Trustworthy Machine Learning through Data-Specific Indistinguishability

Thank you for your positive assessment and valuable suggestions.

1. Differential Trustworthiness (DT) vs. Differential Privacy (DP)

We fully agree with your insightful comment: “Data-Specific Indistinguishability (DSI) and Differential Privacy (DP) (or Input-Independent Indistinguishability (III)) even under the same parameters $(\epsilon, \delta)$ do not imply the same level of trust or privacy.” In Section 1.2 and our Response 1 to Reviewer 53pn, we explain why III is necessary for privacy—instance-based privacy statements or noise mechanisms themselves may already leak information.

The goal of this paper is not to relax existing privacy frameworks (e.g., DP) or improve privacy solutions. Instead, we focus on a broad class of trust concepts that are not primarily concerned with preventing information leakage but rather with controlling data usage (e.g., copyright protection), mitigating influence (e.g., backdoor defense), and governing model behavior (e.g., memorization mitigation).

A key insight from our Differential Trustworthiness (DT) framework is that if we can bound the distinguishability between a target model and a set of safe reference models—such as a model trained on clean data (for backdoor defense) or a dataset excluding memorization-sensitive information—we can obtain a probabilistic guarantee that the target model meets objective trustworthiness requirements.

More importantly, when information leakage is not a concern, input independence is not necessary to derive DT guarantees. This explains why DP (which enforces III) is a sufficient but not necessary condition for DT. Our framework establishes an optimal noise mechanism to achieve DSI, significantly more efficient than existing distinguishability control methods in DP. But, in Section 1.2 including Footnote 2, we clarify DSI noise mechanisms cannot be used to address privacy concerns or provide DP-like guarantees, and we will emphasize this further in our revision.

2. Efficiency of Indistinguishability Control

The goal of our experiments is indeed to compare the efficiency of achieving indistinguishability using DP's standard methods—clipping (sensitivity control) and isotropic noise—versus our DSI framework for black-box data processing. By targeting the same divergence bound (measured in Hockey-Stick divergence with $(\epsilon, \delta)$ parameters), we demonstrate the significant utility improvement enabled by our optimized noise. We will clarify and emphasize that we are not comparing the privacy guarantees ensured by DP-SGD in the revision.

3. Baseline and Additional Experiments Comparing Isotropic and DSI Noise

We apologize for any misunderstanding regarding our notation. In all experiments, we include a baseline (see the column with $\epsilon = \infty$), which represents the original case without any perturbation.

Additionally, we provide new experiments in Exp 3 in the attachment (https://anonymous.4open.science/r/4722-1FD5). Specifically, in a backdoor defense scenario (see Appendix F.3) where training data is collected from $m$ sources (one of which is malicious), we compare the trust-utility tradeoff achieved via DP-SGD (gradient clipping + isotropic noise) versus the DSI framework.

• 3-1) Matching Utility Performance: For $m$ varying from $10$ to $80$, we first adjust the variance of isotropic noise to match the test accuracy obtained with the DSI framework under $(\epsilon=4, \delta=10^{-5})$ and $(\epsilon=8, \delta=10^{-5})$, respectively. We then compare the adversarial success rate (ASR), showing to produce the same empirical test accuracy, DSI outperforms isotropic noise (leading to lower ASR rates) in defending against backdoor attacks in most cases.

• 3-2) Evaluating Provable Indistinguishability: We also report the provable $(\epsilon, \delta)$ guarantees achievable by isotropic noise in this setup. As expected, given that the number of data sources $m$ is relatively small, the worst-case sensitivity $O(1/m)$ remains high. Even with $m=80$, the resulting bounds $(\epsilon=114, \delta = 10^{-5})$ for low-frequency attacks [1] and $(\epsilon=179, \delta = 10^{-5})$ for Blended attacks [2] are too weak to provide meaningful guarantees, to match the same performance from DSI framework with $(\epsilon=4, \delta=10^{-5})$.

These results will be included in our revision, further supporting our claim that standard mechanisms in DP-SGD cannot produce usable divergence bounds in our experimental settings (especially for small dataset and scaling/high-dimensional models).

Finally, we would greatly appreciate your feedback on whether we have adequately addressed your concerns. Please let us know if you have any additional questions.

[1]. Zeng, Yi, et al. "Rethinking the backdoor attacks' triggers: A frequency perspective."

[2]. Chen, Xinyun, et al. "Targeted backdoor attacks on deep learning systems using data poisoning."

Thank you for your positive assessment and helpful suggestions.

1. Relationship Between Privacy and Differential Trustworthiness (DT)

We fully agree with your insightful comment that (differential) privacy is recognized as a stronger guarantee, which, as a sufficient condition, can produce many other (differential) trust guarantees (e.g., reducing memorization and protecting copyright). However, privacy is costly (we also acknowledge your point that "privacy" is a broad concept; here, we specifically use "privacy" to refer to confidential protection). This motivates us to explore more efficient solutions to address “weaker” trust guarantees—those aimed at controlling the influence of training data on models without necessarily preventing data leakage.

Strictly speaking, privacy is not directly comparable to the DT (memorization, backdoors, copyright). One key insight we highlight (see Sections 1.1 and 1.2) is that privacy based on Input-Independent Indistinguishability (III) form a sufficient but not necessary solution: when the goal is not to prevent information leakage, input independence is unnecessary and technically only indistinguishability guarantee is needed to build provable trust. Thus, we propose and justify a relaxed version, Data-Specific Indistinguishability (DSI), and construct the optimal DSI noise mechanism. Due to space constraints, please refer to our Response 1 to Reviewer 53pn for more details on why input independence is necessary for privacy, how DSI compares to per-sample/individual Differential Privacy (DP).

2. Comparison to Distinguishability Control Tools rather than DP Definition

Our key motivation and contribution are not to improve or relax existing DP frameworks but rather to develop better methods for constructing indistinguishability for DT guarantees. As demonstrated above, DSI is strictly weaker than III and is also incomparable to DP guarantees. What our experiments actually target is the efficiency of different mechanisms in achieving indistinguishability.

As the baseline, the classic method to control distinguishability is through clipping (sensitivity control) combined with isotropic noise, as widely explored in DP literature. DP-SGD is a representative. To compare, we propose a new anisotropic noise mechanism to determine the minimal noise necessary for specific indistinguishability. Our framework also enables black-box algorithm analysis without requiring sensitivity control, thereby eliminating clipping bias. Furthermore, we establish accounting methods, such as composition (Theorem 3.4) and grouping (Lemma 3.5), to more tightly convert indistinguishability guarantees into probabilistic DT guarantees.

In summary, our baseline comparison focuses on the efficiency of existing DP methods in achieving the required indistinguishability or statistical divergence bounds rather than their privacy guarantees themselves. To clarify, we will revise claims such as "comparison with DP-SGD" to "comparison with clipping + isotropic noise methods in DP-SGD."

3. Comparison to other Works

• PAC Privacy [1]: PAC Privacy leverages secret entropy and models privacy risk by an adversary’s posterior success rate in recovering the secret. Different from examining the correlation between secrets and leakage from a privacy perspective, DT and DSI do not assume or rely on any input distribution. Instead, we optimize noise based on reference safe models.

• Data-Dependent DP: Sticking to classical DP definitions, [2] utilizes public knowledge of graph model structures to optimize privacy budget allocation across multiple releases. However, the noise mechanism for each release remains the standard Laplace mechanism.

• Sub-space and Public-private DP: Assisted by public data, existing works have considered optimizing clipping strategies, subspace projection, or data augmentation. Still, all existing approaches apply standard isotropic noise. In Exp2 ( https://anonymous.4open.science/r/4722-1FD5), we include a more detailed comparison and show even without assuming public data, DSI-SGD has outperformed all prior benchmarks training on CIFAR10 from scratch. In addition, public data tricks can also save DSI noise, e.g. mixing training data with public samples [3] can reduce the divergence between the target and reference models.

4. Limitations

We will expand the discussion on limitations. As you noted, when both provable privacy and DT guarantees are required simultaneously, the weaker DSI is not applicable and III remains the only known method.

Finally, please do not hesitate to let us know if we have addressed your concerns or you have additional questions.

[1] PAC Privacy: Automatic Privacy Measurement and Control of Data Processing

[2] Data-Dependent Differentially Private Parameter Learning for Directed Graphical Model

[3] DP-Mix: Mixup-based Data Augmentation for Differentially Private Learning

Thank you for your positive assessment and insightful questions.

1. Comparison to other Noise-based Solutions

We appreciate your references to prior works that apply noise for trustworthy machine learning. Existing noise-based solutions typically face two key challenges: a) High utility overhead and b) Lack of provable guarantees.

As a representative of (a) and the primary baseline in our paper, Differential Privacy (DP) mechanisms can be used to ensure provable indistinguishability, which can further imply other trust guarantees such as memorization prevention and unlearning. However, ensuring a meaningful indistinguishability bound (i.e., small security parameters $\epsilon, \delta$) usually requires adding prohibitively-large isotropic DP noise, which is well-known to impose a significant utility cost.

On the other hand, a long line of empirical work explores noise-based defenses without formal guarantees. For example, [1] proposes adding noise to training data as a general defense against unknown textual backdoor attacks; [2] introduces random smoothing, which augments training samples with noisy versions to improve adversarial robustness of trained models; [3] adds small noise to gradients during training to resist membership inference attacks. While these approaches demonstrate strong empirical success, they lack theoretical bounds on how much noise is needed to provably prevent attacks or ensure trust guarantees.

Our Data-Specific Indistinguishability (DSI) noise framework overcomes these limitations by determining the minimal necessary noise to ensure provable trustworthy guarantees while maintaining low utility overhead. The key idea can be summarized below.

• First, given an algorithm $\mathcal{F}$, we identify a safe reference set $R$ whose output $\mathcal{F}(R)$ is highly likely to satisfy the desired trust properties. For example, if $\mathcal{F}$ is a standard deep learning algorithm and $R$ is a clean dataset, then the trained model $\mathcal{F}(R)$ is likely robust to backdoor attacks.

• We then control distinguishability by adding minimal noise to reduce the statistical divergence between the target model and the reference model. This provably ensures that the target model has a high probability of satisfying the desired trustworthiness properties (see Lemma 2.3 and Section 3.2(i)). More importantly, unlike DP, which adds noise isotropically, DSI selectively adds noises only in necessary directions to minimize utility overhead.

DSI also enjoys operational benefits, which enables black-box algorithm analysis and the noise only needs to be added to the final output of a data processing procedure.

2. Additional Experiments on Copyright/Contribution

We have added a set of experiments for the copyright or contribution control using our DSI framework. Please refer to Experiment 1 in attachment (https://anonymous.4open.science/r/4722-1FD5). In this experiment, we finetune a stable diffusion model (v1-4 from https://huggingface.co/docs/diffusers/v0.8.0/en/training/text2image) on a collection $U$ of 425 paintings from 10 artists. We select the reference sets $R_i$ as $U$ excluding one painting, i.e., leaving-one subsets. Thus, the $(\epsilon, \delta)$ DSI parameters characterize the per-sample contribution to the trained model.

In the figure, we present: (1) the original painting, (2) the generated image from the pre-trained diffusion model before fine-tuning, (3) the generated image after fine-tuning without noise, and (4) the generated images after fine-tuning with DSI noise in different indistinguishability budgets ($\epsilon$). From the plot, it is easy to observe that with a larger $\epsilon$, the fine-tuned diffusion learns the art style better and the generated pictures look more similar to the training data.

In practice, the DSI parameters can be used to quantify and determine the contribution of each sample or data source, which provides economic solutions to data usage in collaborative learning.

Finally, we highly appreciate if you could let us know whether we addressed your concerns or you have additional questions.

[1] Zhai, Shengfang, et al. "NCL: Textual Backdoor Defense Using Noise-Augmented Contrastive Learning."

[2] Cohen, Jeremy, Elan Rosenfeld, and Zico Kolter. "Certified Adversarial Robustness via Randomized Smoothing."

[3] Aerni, Michael, Jie Zhang, and Florian Tramèr. "Evaluations of Machine Learning Privacy Defenses Are Misleading."

Thanks for your valuable comments and suggestions!

1. Comparison with (Per-Sample/Individual) Differential Privacy (DP)

We believe there are at least three fundamental differences between Differential Trustworthiness (DT) and Differential Privacy (DP).

1-1) First, on motivation, though both DT and DP leverage distinguishability or statistical divergence, they target very different problems. Distinguishability measures the closeness of two distributions, which can be interpreted in two ways: a) It is hard to distinguish the source of a randomly drawn sample; b) The behavior of two random variables, $a \sim A$ and $b \sim B$, is similar in probability: $\Pr(a \in O) \approx \Pr(b \in O)$ for any set $O$.

Privacy (confidential protection) typically adopts a) to model preventing adversaries from recovering secrets from leakage. If the leakages from different secrets are indistinguishable, no useful information is revealed to an adversary.

In contrast, DT adopts b), using divergence between a target and a set of safe references to characterize the probability that an algorithm’s output satisfies certain trust properties. Imagine, for a set of reference training datasets $R_1, R_2, \dots, R_m$, if each trained model $\mathcal{F}(R_i)$ has a 99% probability of not memorizing sensitive information $q_i$, then the divergence between $\mathcal{F}(U)$ and each $\mathcal{F}(R_i)$ provides a bound on the probability that $\mathcal{F}(U)$ does not memorize all $q_i$ simultaneously.

1-2) Conceptually, DT justifies the role of data dependence in trust-preserving mechanisms, whereas this remains a challenge in privacy. If a specific sensitive data point $x_0$ becomes a parameter in a privacy guarantee (e.g., achieving $(\epsilon, \delta)$-DP for the membership of $x_0$), this statement itself already leaks information about $x_0$. Section 1.2 discusses why input independence is essential for privacy and why most DP mechanisms rely on worst-case sensitivity.

Since DP mechanisms calibrate noise based on the worst case, regular (average-case) data points may enjoy stronger privacy guarantees—motivating prior work on per-sample [1,2] or individual privacy [3]. However, formally quantifying this average-case amplification from the provable, global security parameters remains an open question. Existing works [1,2,3] can only estimate per-sample/individual privacy loss, which itself is sensitive and cannot be disclosed.

A key insight from DT is that many trust concerns focus on the use (e.g. copyright) or influence (e.g. backdoor) of public data to train a model, or governing models (e.g. memorization) to prevent certain behaviors, and it suffices to ensure indistinguishability with respect to safe reference models. More importantly, unlike privacy-preserving operations, when leakage is not a primary concern, input-independence is not necessary, and thus we propose the more efficient DSI.

Another distinction is that DT only requires one-way divergence—specifically, the $f$-divergence from the target output $\mathcal{F}(U)$ to the reference outputs $\mathcal{F}(R_i)$. In contrast, DP treats two adjacent datasets $X$ and $X'$ symmetrically, requiring two-way divergence to prevent leakage. See Section 3.2(i) for further details.

1-3). Operationally, we demonstrate how to determine the optimal anisotropic noise to achieve the required DSI guarantees. However, isotropic noise in prior works [1,2,3] cannot provide controllable per-sample/individual privacy loss. Moreover, our composition theorem (Theorem 3.4) is tight, whereas the composition approach in [2] involves complex expectation-based estimates, and [3] still relies on worst-case individual guarantees.

We will incorporate those comparisons in our revision.

2. Comparison with State-of-the-art Results

Please refer to Table 1. We have compared with the DP-SGD benchmarks (De et al., 2022, https://arxiv.org/pdf/2204.13650) you suggested. By optimizing DSI noise, we achieve 7-10% improvement in test accuracy in all cases with varying indistinguishability budget.

3. Subsampling Amplification

We appreciate your insightful comments and acknowledge that our current results do not exploit subsampling-based randomness, which could further enhance the trust-utility tradeoff—particularly in applications like SGD. We highlight this as a promising future direction in Section 6. But it is also noteworthy that even without subsampling amplification, DSI mechanisms already significantly outperform best-known DP methods that has incorporated subsampling (Table 1).

Finally, we highly appreciate if you could let us know whether we addressed your concerns or you have additional questions.

[1] Y.-X. Wang. Per-instance differential privacy.

[2] V. Feldman and T. Zrnic, Individual privacy accounting via a Renyi filter.

[3] Thudi, Anvith, et al. "Gradients look alike: Sensitivity is often overestimated in DP-SGD.