Collapsing the Learning: Crafting Broadly Transferable Unlearnable Examples

Thanks for your comments and your appreciation of our findings. We address your concerns below:

W1: The writing of the paper is extremely poor. The paper is not well-organized and it is difficult to follow.

Thank you for your comment regarding the writing quality and organization of our paper. We have taken your feedback into account and have thoroughly revised and improved the manuscript based on the suggestions from you and other reviewers. The latest version has been submitted for your consideration.

W2: The paper does not motivate the proposed method. What is the need for transferable unlearnable examples? On that same note, the contributions of the paper and the proposed method are not well-justified. Why does the proposed method work?

Thank you for your question regarding the motivation and justification for our proposed method. In practical scenarios, it is desirable for unlearnable examples generated by different surrogate models to provide protective effects across various targeted model architectures. This is why we focus on the issue of transferability. To achieve this goal, we first propose to induce data collapse. Using the Score Function [1], we obtain the log gradient of the data distribution. By updating the data with this log gradient, we make the original data more concentrated, thereby reducing the information contained in the data. As this process is model-agnostic, it possesses high transferability. Furthermore, to maintain protection under adversarial training conditions, we propose a modified adversarial training strategy for unlearnable examples. Ultimately, our method can achieve robust transferability. The effectiveness of our approach is demonstrated through numerous experiments in the paper. We hope this response addresses your concerns about the motivation and justification for our proposed method.

W3: How does the proposed modified adversarial training help with effectiveness of the generated examples for an adversarial trained model?

Thank you for your question regarding the effectiveness of our modified adversarial training. The modified adversarial training process helps make the surrogate model robust. In cases where the poisoned model undergoes adversarial training, it will also become robust. As a result, by utilizing robust surrogate models, we aim to generate robust unlearnable examples that remain effective even for adversarially trained models.

If you have any specific question, I would be very willing to address your question.

$1$ A connection between score matching and denoising autoencoders. Neural computation 2011

We are truly grateful for your feedback. In our responses, we have endeavored to address every concern raised with thorough explanations and evidence. As we approach the conclusion of this phase of the review process, we are keen to ascertain whether our rebuttal has successfully resolved the issues highlighted. We invite any additional comments or questions you may have regarding our responses. Your expertise and perspectives are vital in guiding the refinement and understanding of our work. We eagerly await your further guidance and insight.

Thanks for your valuable comments and your appreciation of our findings. We address your concerns below:

Summary: The paper proposes to induce data collapse, before the standard REM procedure.

We would like to clarify that our approach is notably different from the REM method. The key distinctions are as follows:

1. To enhance the transferability of unlearnable examples, we propose Data Collapse. By employing data collapse, we reduce the information contained in the data. The ablation experiments in Table 3 of the paper demonstrate that data collapse can indeed improve transferability, which is a feature not present in other related works.
2. To enhance the robustness of transferability, we propose Modified Adversarial Training (MAT). This method specifically modifies traditional adversarial training to address the unique characteristics of unlearnable examples. The optimization objective of MAT is (min-max)-min, while the optimization objective of REM is min-(min-max). The two objectives are fundamentally different. Our optimization objective aims to make the substitute model $f_{\theta}'$ robust, thereby making the generated unlearnable examples robust as well. In contrast, REM's goal is to generate unlearnable examples for adversarial samples, and its substitute model is not robust.
3. Our method achieves robust transferability through two steps: Data Collapse and Modified Adversarial Training. We hope this clarification addresses your concerns regarding the differences between our approach and the REM method.

W1: The perturbation budget of adversarial training $(ℓ≤4/255)$ is too low, larger perturbations need to be considered. Especially, unlearnable perturbations can often be much less effective by simply increasing the adversarial perturbation budget.

Thank you for your comment regarding the perturbation budget. In response to your concern, we have increased both $\rho_u$ and $\rho_a$, and the experimental results are presented in the table below.

As you pointed out, increasing $\rho_a$$ does indeed reduce the effectiveness of unlearnable examples. To clarify, we define the party generating unlearnable examples as the Attacker and the party training models using unlearnable examples as the Defender. In classification tasks, the Attacker's goal is to minimize the test accuracy of the Defender's model trained on unlearnable examples, while the Defender's goal is to achieve higher model accuracy.

1. From the Attacker's perspective, increasing $\rho_u$ has little cost, as there is no need to use unlearnable examples. However, for the Defender, undermining the effect of unlearnable examples requires increasing $\rho_a$, which is costly. As seen in the table above, as $\rho_a$ increases, the performance upper bound of the model declines significantly. This is actually an arms race, where the Attacker forces the Defender to raise $\rho_a$ by increasing $\rho_u$. Once $\rho_a$ reaches a certain level, the model's training difficulty increases, and the performance upper bound is significantly reduced. In this way, the Attacker achieves their goal.
2. In this scenario, Table 1 in the paper indicates that our method is superior to other methods. our method provides valuable insights into the dynamics between the Attacker and Defender and showcases its effectiveness in various settings, rather than being limited to a specific perturbation budget. We hope this response addresses your concerns and provides a clearer understanding of our approach.

W2 : This paper lacks a comparison of OPS [1]. A significant advantage of OPS is its strong resistance against $ℓ_{2,∞}$ adversarial training, even under large adversarial perturbation budgets.

Thank you for your comment regarding the comparison with the OPS baseline. We acknowledge the importance of including relevant baselines for a comprehensive evaluation. In practice, data augmentation methods are often employed during the model training process, and their cost is significantly lower than that of adversarial training. OPS is prone to being compromised by data augmentation techniques([2,3]). The augmentation techniques almost have no effects on our method. The table below shows the protect effects on CIFAR10 with data enhancements.

We hope this response addresses your concerns and provides a clearer understanding of our approach in comparison to the OPS baseline.

We are truly grateful for your valuable insights and comprehensive feedback. In our responses, we have endeavored to address every concern raised with thorough explanations and evidence. As we approach the conclusion of this phase of the review process, we are keen to ascertain whether our rebuttal has successfully resolved the issues highlighted. We invite any additional comments or questions you may have regarding our responses. Your expertise and perspectives are vital in guiding the refinement and understanding of our work. We eagerly await your further guidance and insight.

Thanks for your valuable comments and your appreciation of our findings. To clarify, we did not merely compare CIFAR-10 and CIFAR-100; we also compared a subset of the first 100 classes of ImageNet. This covers data with different categories and resolutions. Subsequent transferability experiments were conducted only on CIFAR due to resource constraints. We address your concerns below:

W1. As pointed out by the authors in the last section, the proposed method requires significant computational cost and is currently unscalable to large dataset like ImageNet.

Thank you for your comment. Detailly, in our setting, we employed 4 V100 GPUs to train ResNet-18 on the ImageNet-Subset for 5,000 steps, which takes only 39 hours. In the future, we will further explore how to improve the speed of our methods.

W2. The proposed Algorithm 1 has many parameters. Can you provide more discussion on how to set them in practice based on data, and how are they related to the performance or robustness against adversarial training?

Thank you for your question. For the data collapse step,
$\alpha_d$ is based on commonly used default parameters from some score-based generation methods. As for $K_{\alpha}$, it is a hyperparameter, and its value is determined based on computational cost and the value of $\rho_u$. For the Adv step, the step sizes $\alpha_a, \alpha_u$ are derived from the step counts $K_a, K_u$ and the perturbation ranges $\rho_a, \rho_u$. We usually ensure that $K_a \cdot \alpha_a > \rho_a, K_u \cdot \alpha_u > \rho_u$.

The larger the $\rho_u$, the better the protection effect. The larger the $\rho_a$, the greater the range of adversarial training perturbations that can be protected, but it must be ensured that $\rho_a < \rho_u$. When the dataset resolution is large, $K_a, K_u$ typically choose smaller steps to reduce computational cost, and under constant $\rho_a, \rho_u$, $\alpha_a, \alpha_u$ need to be larger. Generally speaking, the more steps $K_a, K_u$ have, the better the effect.

Q1. In Section 3.3, the predefined distribution is set as the normal distribution $\mathcal{N}(\hat{x};x,\sigma^2I)$. I am wondering what is the reason or intuition for such setting. Can we use a different distribution?

Thank you for your insightful question. Firstly, our training objective is to estimate the true data distribution $p_D(x)$, but we do not have the ground truth of the real data. Secondly, according to Stochastic Gradient Langevin Dynamics, what we actually need is $\nabla_x \log p_D{(x)}$, which is referred to as the Score, with the full name being Stein Score. Therefore, we estimate the Score using the denoise score matching [1] method.

In the Denoise Score Matching process, we add Gaussian noise $\epsilon = \mathcal{N}(0, \sigma^2)$, resulting in $\tilde{x} = x + \epsilon$. According to the properties of the Gaussian distribution, we obtain the distribution $q(\tilde{x}|x)= \mathcal{N}(\tilde{x};x,\sigma^2I)$ as mentioned in the paper. The optimization objective of Denoise Score Matching is Equation 3: $\frac{1}{2} \mathbb{E}_{q_\sigma(\tilde{x} \mid x) p_D(x)}\left[\left\|s_\theta(\tilde{x})-\nabla_{\tilde{x}} \log q_{\sigma}(\tilde{x}|x)\right\|_2^2\right]$Therefore, to make the equation meanfuling, any distribution that satisfies the existence of $\nabla_{\tilde{x}} \log q_{\sigma}(\tilde{x}|x)$ can be used.

Q2. In Algorithm 1, the PGD process in lines 13-16 and lines 17-20 are quite similar. What is the advantage of two-stage PGD over one-stage? Also, is there any trade-off between the magnitude of $\rho_a$ and $\rho_u$?

Thank you for your question. PGD[2] is a standard approach for solving inner maximization and minimization problems. It performs iterative projection updates to search for the optimal perturbation.

In our modified adversarial training, the optimization formula is min-max-min. Our algorithm incorporates two PGD processes. The first PGD process corresponds to the inner min in the optimization formula, solving for $\delta_u$. The second PGD process corresponds to the max procedure in the optimization formula, solving for $\delta_a$. The larger the $\rho_u$, the better the protection effect. The larger the $\rho_a$, the greater the range of adversarial training perturbations that can be protected, but it must be ensured that $\rho_a < \rho_u$.

Q3. Following the previous question, if the two-stage PGD is more powerful than one-stage, can we modify the algorithm to add more stages in PGD to expect a better performance?

Thank you for your question. As mentioned in Q2, the PGD step is used to different function. Each PGD step corresponds to an optimization objective, so it cannot be arbitrarily added.

Thanks much for the response and clarifications. I misunderstood a small part of the algorithm in the beginning. I think my previous questions are addressed properly.

As I go through other reviews, I find another major concern that I did not think of earlier, as raised by reviewer KXx5.

In the numerical studies, for adversarial training it seems only the approach in [1] is considered. I am curious about the performance of the work with adversarial training (or other robust) schemes proposed more recently.

[1] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https://openreview.net/forum?id= rJzIBfZAb.

We are truly grateful for your valuable insights and comprehensive feedback. In our responses, we have endeavored to address every concern raised with thorough explanations and evidence. As we approach the conclusion of this phase of the review process, we are keen to ascertain whether our rebuttal has successfully resolved the issues highlighted. We invite any additional comments or questions you may have regarding our responses. Your expertise and perspectives are vital in guiding the refinement and understanding of our work. We eagerly await your further guidance and insight.

Q4. From my understanding the universailty/transferability relies on the first step of modification of data distribution. This approach to model-independent modification needs to should be positioned in contrast to related works. For instance, can this be combined with other approaches as a pre/post processing step?

Thank you for your insightful question. Yes, the data collaspe step can be combined with other related methods. The following table shows the results on CIFAR10.

Table 3 in the paper also demonstrates that our data collapse step can be combined with other methods. In Table 3 of the paper, w/o(Collaspe + Adv) = EM, w/o(Adv) = EM + Data Collapse. Both Table 3 and the table above show that our data collapse step can be integrated with other methods and can improve transferability.

Thanks for your valuable comments and your appreciation of our findings. We address your concerns below:

W1. The interpretation relative of transferability to Robust Unlearning Methods.

Thank you for your suggestions. We have revised our manuscript to clarify this point. Our transferability refers to the effectiveness of unlearnable examples generated by different surrogate models on various poisoned models, regardless of whether they employ standard training or adversarial training strategies. We have highlight the definition in the introduction with italic type.

W2. From a presentation and readability standpoint, I think including the arguments for “how” this approach tackles transferability upfront would better clarify the contributions. Moreover, it might be worth considering if the aim is to ensure universality rather than transferability.

Thank you for your suggestion. We have summarized the arguments and put them upfront our approach in Section 3.3. $x_t$ represents the sample at step $t$; $\nabla \log p_D(x_t)$ denotes the $\log$ gradient of the data distribution concerning the sample $x_t$; $\alpha$ signifies the step size; $\epsilon$ represents random noise following a distribution. $s_{\theta}$ indicates the data distribution gradient estimator, while $\tilde{x}$ refers to the sample after adding noise. Universality is also an important research direction, and we will further deepen our research in this field in the future.

Q1. Does the sequence of Collapse and Adv matter?

Thank you for your insightful question regarding the sequence of Collapse and Adv. Yes, the order in which they are performed does matter. We first apply Collapse to modify the data, aiming to enhance the transferability of unlearnable examples. Subsequently, we conduct the modified Adv to improve the robustness of transferability. If we were to perform Adv first, the resulting noise would be model-dependent, and the subsequent Collapse would be based on this model-dependent foundation, significantly reducing transferability. Moreover, if the noise generated by Adv varies each time, a new data distribution estimator would need to be trained for each instance. By executing Collapse before Adv, we avoid this issue, as Collapse targets the original data.

Q2. Clearly describe the the second step in term of what f represents and how PGD over adversarially trained model is carried out.<It would be useful to clearly describe the second step in term of what f represents and how PGD over adversarially trained model is carried out. I think the algorithm has the required steps but the description in Section 3.4 wasn’t sufficiently clear.

Thank you for your question.

1. $f_{\theta}'$ represents the surrogate model, which is distinct from the $s_{\theta}$ used in the first step for inducing data exploration. These are two separate models.
2. PGD[1] is a standard approach for solving inner maximization and minimization problems. It performs iterative projection updates to search for the optimal perturbation as follows: $\delta^{(k)} = \prod_{\| \delta \| \leq \rho} \left[ \delta^{(k-1)} + c \cdot \alpha \cdot \text{sign} \left( \frac{\partial}{\partial \delta} \ell(f_\theta(x + \delta^{(k-1)}), y) \right) \right]$ Where $k$ is the current iteration step ($K$ steps at all), $\delta^{(k)}$ is the perturbation found in the $k$-th iteration, $c\in\{-1,1\}$ is a factor for controlling the gradient direction, $\alpha$ is the step size, and $\prod_{\|\delta\|\leq\rho}$ means the projection is calculated in the ball sphere $\{ \delta:\|\delta\| \leq \rho \}$. The final output perturbation is $\delta^{(K)}$. Throughout this paper, the coefficient $c$ is set as $1$ when solving maximization problems and $-1$ when solving minimization problems. This also can be found in Appendix A.3.
3. We employ modified adversarial training to update the surrogate model in three main steps. First, we use the PGD algorithm to generate unlearnable noise for clean samples, obtaining $x+\delta^u$. Second, we apply the PGD algorithm to $x+\delta^u$ to produce adversarial noise, resulting in $x+\delta^u+\delta^a$. Finally, we input the obtained $x+\delta^u+\delta^a$ into the surrogate model and update the surrogate model parameters using stochastic gradient descent. We have modified the Section3.4.

Q3. Is the current notations model f and model s are defined with shared parameters of $\theta$. Please clarify

Thank you for your question. No, the current notations for the models $f_{\theta}'$ and $s_{\theta}$ represent different parameters. For $s_{\theta}$, it represents the log gradient of the data distribution estimator, and we use the U-Net structure. While for $f'_{\theta}$, it represents the surrogate model for generating robust unlearnable examples, and we employ common architectures such as VGG-16, ResNet-18, ResNet-50, DenseNet-121, and WRN-34-10.

$1$ Towards deep learning models resistant to adversarial attacks. ICLR2018

We are truly grateful for your valuable insights and comprehensive feedback. In our responses, we have endeavored to address every concern raised with thorough explanations and evidence. As we approach the conclusion of this phase of the review process, we are keen to ascertain whether our rebuttal has successfully resolved the issues highlighted. We invite any additional comments or questions you may have regarding our responses. Your expertise and perspectives are vital in guiding the refinement and understanding of our work. We eagerly await your further guidance and insight.