Data Taggants: Dataset Ownership Verification Via Harmless Targeted Data Poisoning

We thank the reviewer for the thorough and attentive review of our work, we highly appreciate the effort that was put in your review. Allow us to address the above-mentioned weaknesses:

1. This is an interesting point.
   First, Table 3 shows that our approach still works when replacing the random keys with test images (accounting for an actual data poisoning), other forms of keys could be experimented with, if needed, to avoid detection.
   Second, what OOD detection would you have in mind? Most of them (if not all) rely on having a set of inliers and a set of outliers. If you do not know beforehand what the outliers (here the keys) will look like, it is unclear how well it would work.
   Finally, such a countermeasure to data taggants could have downfalls on the provided service. The model provider would likely reduce the utility of their model to any user sending images that would be detected as being OOD.
2. We thank the reviewer for the relevant remark. Watermark/poisoning collisions can indeed be observed in classical settings targeting actual data points. By “collision”, we mean that a data poison (or backdoor watermark) supersedes the initial attack or gets a higher priority when training a model which can disable it.
   This can be the reason why targeted data poisoning can display difficulty to scale in terms of number of targets (e.g. Table 9 in [1] showing their method’s failure when targeting several images). Backdoor attacks however prove to be effective even in the case of multiple attacks as shown in [2].
   Our very method shows that several independent attacks can coexist, since we generate the data taggants for a given key independently from the others. To disable data taggants, an adversary would need to modify at least part of them. Given they only amount to 0.1% of the whole dataset (and already require a non-negligible amount of compute to be crafted), it would be really difficult for an adversary to find them and craft another poisoning on top of them.
3. The lower the $k$, the lower the measured top-$k$ accuracy, and the less effective our approach is. Top-$k$ prediction is still far less information required compared to other approaches (e.g. radioactive data). Also, one could mandate a model provider to give access to the top-$k$ prediction to their model to run the verification, protecting the model from being disclosed.

Regarding your questions:

1. Radioactive data in black-box setting without distillation amounts to a membership inference attack and has none of the theoretical guarantees radioactive data approach offers in white-box or black-box distillation settings. Also, in the black-box without distillation setting, they only claim that "We can see that the use of radioactive data can be detected when a fraction of q = 20% or more of the training set is radioactive". Overall, radioactive data in black-box setting without distillation is a different approach that can hardly be said to work. We thank you for your remark and will make sure to change the description to explicit this distinction.
2. Their use of a t-test depends on the dataset that is chosen to run the test on, which in turn, becomes a factor of confusion. We will make sure to change the manuscript to clarify our criticism of backdoor watermarking’s statistical testing.
   One simpler point we would like to make is that the hypothesis they test for (Proposition 1 in [3] - $H_{1}: P_{b} + \tau < P_{w}$) has no theoretical grounding. As we show below, a benign model can as well display the same behavior they measure as a watermarked model.
3. The imperceptibility is mainly a matter of tradeoff between the gradient-matching loss and the perceptual loss. The dimension of the input space has a role in the effectiveness of the method. We found for instance that the method overall is less effective on CIFAR-10 but we can keep the perturbation relatively imperceptible with the perceptual loss weight. Other experiments show that the approach can be made both effective and imperceptible on 1-second 16kHz audio samples. Future work will be coming on other modalities.

Thank you for engaging in the discussion.

It is not straightforward OOD detection can detect keys when Bob has no idea what the keys might even look like (i.e. if Bob has no idea what the distribution of the keys might be). While we experimented with OOD keys where pixels are uniformly sampled, Table 3 shows that the method still work for in-domain data. Follow-up work can consider many form of keys and explore which ones are the most effective and stealth to OOD detection.

It appears to us that we addressed all your concerns. What could make you reconsider and increase your score?

We thank the reviewer for their review of our work. To address the above-mentioned weaknesses:

• Regarding the novelty:
  [1] relies on creating samples that should be difficult to classify when a model is trained on a benign dataset but easy to classify when trained on the protected dataset. Our approach, on the other hand, relies on randomly sampled out-of-distribution samples that have no ground truth label, and then force the model to make a decision on these samples when trained on our protected dataset.
  Even though [1] also uses gradient matching to craft the perturbations, it is used after they trained a domain adaptation model to generate their “domain watermarked samples”, which is the main contribution of their work. That part (subsection 3.3 in [1]) is not straightforward and requires training a model to generate “new domain” samples.
  The authors of [1] open sourced part of their code on Sept. 16th of this year, without the code to train a domain adaptation model, making their work extremely difficult to reproduce as such.

• Regarding the harmfulness argument:
  Caption of Figure 1 in [1] states that:

"Existing backdoor based methods make the watermarked model (i.e., the backdoored DNN) misclassify ‘easy’ samples that can be correctly predicted by the benign model and therefore the verification is harmful"

Given a sample $x$ which is correctly classified as $y_{true}$, adding the trigger signal $t$ makes the watermarked sample $x+t$ which is supposed to be classified as $y_{w} \neq y_{true}$ by a watermarked model. Given that a trigger only makes sense if it does not alter too much of the data, a cleanly trained model should classify $x+t$ as $y_{true}$ (e.g. a $224 \times 224$ fish picture with a $16 \times 16$ patch in the corner is reasonably still a fish picture). Hence, the backdoor is here to introduce errors.
This is exactly what [1]'s measures of harmfulness (Harmful $H$ and Relatively Harmful Degree $\hat{H}$ in Definition 1) measure and what their argument is based on.

• Regarding the intuition behind the use of out-of-distribution samples as keys:
  As we explain on line 235:

  "Since no natural behavior is expected from the model on these keys, enforcing a specific behavior on them should not induce particular errors, as opposed to backdoor watermarking approaches."

  Could you clarify what you would like us to add? Would you be ok with the following rephrasing:

  "Since no natural behavior is expected from the model on these keys, as they are made of random pixels, enforcing a specific behavior on them should not induce particular errors, as opposed to backdoor watermarking approaches, which relies on modifying the behavior of a model on actual images."

• Regarding choosing Sleeper Agent:
  Even though Sleeper Agent is not the simplest backdoor attack, it is an effective approach. Please notice that Table 1 also includes “Data isotopes” [2] which uses a more traditional backdoor watermarking approach relying on blending a visible trigger into the image.
  (1) Because Sleeper Agent similarly leverages gradient-matching, it allows us to compare fairly the backdoor watermarking approach which uses triggers against data taggants which uses keys.
  (2) The values presented for method [3] were obtained in our setting and following the same detection protocol as the one described in Algorithm 1 of [3], with a margin $\tau = 0.2$. When varying the margin and running the detection on 4 models, we observe a decreasing p-value that plateaus at 0.8 for a ridiculously low margin. The dashed red line is the 0.05 threshold of significance. Error bars represent max/min values.
  Plot image: p-value for the detection of a model watermarked with Sleeper Agent
  When similarly running the detection on benign models (hence checking for potential false detection), we obtain the following results:
  Plot image: p-value for the detection of a benign model
  This means that Sleeper Agent fails altogether on our setting.
  We explain the discrepancy with what the authors reported in [4] by the differences in settings. Most notably, we use a far more challenging training recipe (with much more aggressive data augmentations).
  (3) As requested, we ran the same experiments (same model, dataset, poisoning budget) using the same detection method (Algorithm 1 in [1]) using the BadNet backdoor attack and obtained the following results on watermarked and benign models:
  Plot image: p-value for the detection of a model watermarked with BadNet and a benign model
  The p-value for the detection of benign models being lower than that of watermarked models indicates that running [3] detection algorithm can lead to a higher number of false positives than true positives.\

We thank the reviewer for their review and encourage you to engage in the discussion, replying to our rebuttal.

We have carefully addressed the main concerns in detail and updated the manuscript accordingly. Is there any remaining concern before you can consider increasing your score? We would be glad to clarify any further concerns (if any)

Best regards,
Authors

Allow us to address your questions:

1. The reason why gradient matching works even when only crafting the gradients from a fully trained model is still not understood. [1] suggest to retrain the model during the poison crafting to avoid overfitting to a clean-trained model. This approach induces high training cost when dealing with large-scale datasets such as ImageNet1k.
   The idea of introducing randomly initialized models when crafting poisons have not been explored to the best of our knowledge. Some experimental results we obtained when reproducing witches’ brew [2] experiments on CIFAR-10 showed that the more trained Alice’s model is, the better the poisoning works.
   Our intuition is that neural networks could be using similar features, even at different initializations (and even architecture as per our stress-test experiments). As such, optimizing the data taggants on Alice’s trained surrogate model is enough to have features emerging in the data that can be learned as expected by a newly initialized model. Conversely, when crafting data taggants from a poorly trained model, because the feature extractor has yet to fully emerge, it fails to properly allow to derive relevant features that can be transferred to different models.
2. You are right. Here, in our experiments, we consider any non-zero accuracy on the keys to be suspicious, which leads to a 100% TPR and 0% FPR.

Finally, regarding your point on our related work mentioning the persistance of watermarks, it appears that we need to clarify our point:
Watermarking is traditionally not made to radiate through the processes, only to hold information. While Sander et al. shows that watermarked text (via controlled sampling) can impact a model during fine-tuning, this behaviour is a fortunate byproduct of the initial goal: having detectable text. On the other hand, data taggants are hard to detect from clean data and their whole purpose is to impact models during training. This discussion is indeed interesting and we will make sure to clarify it.

We sincerely hope that the reviewer can kindly consider raising the score if our response helps address some of the concerns.

[1] Souri, Hossein, et al. "Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch." Advances in Neural Information Processing Systems 35 (2022)
[2] Geiping, J., Fowl, L., Huang, W. R., Czaja, W., Taylor, G., Moeller, M., & Goldstein, T. (2020). Witches' brew: Industrial scale data poisoning via gradient matching.
[3] Li, Y., Zhu, M., Yang, X., Jiang, Y., Wei, T., & Xia, S. T. (2023). Black-box dataset ownership verification via backdoor watermarking.

We thank the reviewer for the thorough and attentive review of our work, we highly appreciate the effort that was put in your review. We would like to first address the above-mentioned weaknesses:

• Regarding the evaluations and related works positioning:
  In your review, you say:

  the position of the paper seems to be "there may be DOV methods with strictly better TPR but they come with problems such as unrigorous guarantees or perceptible data changes"

  Could you please elaborate on the elements that made you believe this was the position of our paper? Especially given that we already show in Table 1 that our method achieves better TPR than baselines.
  We added another baseline in the very same setting as Table 1 that we initially discarded: Backdoor watermarking using the BadNet approach (a visible fixed trigger) and the detection method from [3] on 4 watermarked models (to compute the TPR) and 4 benign models (to compute the FPR). This approach relies on an additional hyper-parameter $\tau$ that controls the sensitivity of the test:
  Plot image: p-value for the detection of a model watermarked with BadNet and a benign model
  Given that the p-value of the benign models is lower than that of the watermarked models, for any threshold of significance, BadNet would lead to a higher FPR than TPR in this example, making the method unreliable.
  To improve the clarity of the comparison with previous work, we plan to add a table to the paper to draw a comparison between our work and the related works across the relevant dimensions of comparisons and further explain which baselines we found to be relevant to compare against and which ones were discarded. We hope that would be sufficient to address your concern.

• Regarding the claims of technical contribution:
  We thank the reviewer for taking our contributions into consideration. We believe the current version of the paper already lists the contributions at the end of the introduction. We nonetheless would like to address your point and will update the manuscript to properly highlight our contributions in the rest of the paper.

• Regarding our theoretical guarantees:
  We need to clarify that we do not make any assumption on the classifications/predictions of models on the keys (randomly sampled OOD data points). Regardless of the model, if it is benign, it was not exposed to information about the keys (i.e. either the keys or the data taggants). Because the keys’ labels are random, accuracy on random labels can only amount to chance level. To detail what was shown in the proof of the Proposition 1:

  • the accuracy of a benign model on 1 key must follow a Bernoulli distribution with parameter $\frac{1}{|\mathcal{Y}|}$;
  • hence the top-$k$ accuracy on 1 key must follow a Bernoulli distribution with parameter $\frac{k}{|\mathcal{Y}|}$;
  • since the labels of the $K$ keys are sampled independently, the number of correct top-$k$ predictions on the $K$ keys follows a binomial distribution with parameters ($K$, $\frac{k}{|\mathcal{Y}|}$).

  This allows us to have a theoretical FPR for any observed performance displayed by a model. Given its level (as low as $10^{-60}$), we unfortunately cannot empirically validate it as it would require us running at least thousands of measures to expect one of them to be a false positive. Each of these measures requires training a model from scratch on ImageNet1k (each of them requiring roughly 200 GPU-hours). This would amount to an unreasonably large compute time, making empirical validation of the FPR infeasible. After running our detection procedure on a dozen models, we found a FPR of 0 as reported in Table 1. Backdoor watermarking, on the other hand, cannot provide any theoretical guarantee on the FPR because they cannot characterize the expected behavior of a benign model in their setting. Your remark on the choice of the OOD samples is also relevant. If there was a “TV static” class, then the keys we used in our experiments would hardly be OOD anymore and would amount to choosing the keys among the test images (from the “TV static” class) as shown in Table 3.

• Regarding the exploration of the key technical contribution:
  The exploration you mention would be interesting but falls in a much broader study about gradient matching which seems out of the scope of this paper. Future work on understanding training dynamics should definitely consider addressing this question.

We thank you very much for noticing typos and we made sure to correct them in the manuscript right away.

We thank the reviewer for their time and help.
We appreciate you found our work novel and recognise the stronger theoretical guarantees we provide compared to previous work.

To address the above-mentioned weakness regarding a formal security analysis: could you please give precisions on what you would expect from such analysis and clarify what you call adaptive attacks?

We are glad to address your questions:

1. We believe that by “the exact verification technique”, you mean the keys and the data taggants. If Bob:
   • had knowledge of the keys, he could simply train on them with random labels;
   • had knowledge of the data taggants, he could simply remove them from training.
     We remain at your disposal to include any other component you think could be considered as the exact verification technique.
2. Table 6 in appendix shows the performance of our method and baselines for different budget values (0.001%, 0.01%, 0.1%). The chosen budget of 0.1% corresponds to poisoning roughly 100 samples per key and is enough to be effective. Higher poisoning rates make the computation time too high to repeat them and run thorough experiments with standard deviations.
3. Given that we acknowledge tackling an image classification task with an image classification dataset, we fail to see what you would consider to be a “downstream task”. Could you please elaborate on this?

We will be glad to address any remaining questions and concerns.