Towards Adversarially Robust Condensed Dataset by Curvature Regularization

We really appreciate your constructive feedback. In particular, we are happy that you recognized several strengths of our paper. Below are our responses to your questions and suggestions:

The proposed method should exhibit a performance better than random chance.

We don’t believe it is fair to hold adversarial defense methods for condensed datasets to such standard. Many of the adversarial defense methods that work well under the real dataset setting do not apply in the condensed dataset setting. This is because one of the fundamental premises of dataset condensation is that no real data will be given to the downstream model at training time, which is important to preserve many of the properties of condensed datasets, including size, training efficiency, and privacy. Without real data, the data distribution that the model learns is already largely different compared to the test images, making it much easier for the adversary to find adversarial examples. Therefore, we don’t think it is realistic to expect the same degree of robustness from condensed datasets and real datasets.

In addition, one reason for the low level of robustness is the inherent low level of clean accuracy. For example, many adversarial defense methods can retain a robustness of around 50%, on a model that can normally achieve 90% clean accuracy on CIFAR10. However, DC can only achieve 50% clean accuracy on CIFAR10, so it is not surprising that when applied the same level of attack, the robustness accuracy deteriorates severely.

We are aware of the lack of previous research in this area. This is why in addition to comparison to vanilla dataset condensation methods, in the appendix we also showed a comparison between our results and those obtained from adversarial training, which is recognized as one of the more effective adversarial defense methods for standard datasets. This comparison demonstrates that our method yields superior results.

The enhanced robustness performance doesn’t seem to justify this computational overhead. Has there been an analysis regarding computational cost?

Although we don’t have a formal analysis of the computational cost, it is very negligible. In fact, although the problem of robust dataset condensation is a tri-level optimization problem, we formulated our method in this exact way to reduce the potential computational cost by embedding the procedure in the bi-level optimization of dataset condensation.

You can look at our algorithm in the appendix. Our perturbation $z$ is calculated from the gradients from the real images, where the gradient is already calculated in the original DC algorithm. Then, we simply do another forward pass (without a backward pass) with real images plus $z$, and calculate the final regularization through the difference of the losses. This means the overall computational overhead is only a single forward pass per iteration.

The proposed method shows decline in performance on regular data.

In fact, we show improvements to clean accuracy after applying our method in the appendix. (Our base condensation method is DC). While this increase in accuracy may be due to differences in experimental procedures, and is not an intended consequence of GUARD, it does show that there is no clear decrease in clean accuracy.

Are there performance results under AutoAttack setting with the same epsilon as strong attack?

Unfortunately we haven’t conducted experiments in that particular setting.

Are there any other cases where adding curvature regularization to methods other than gradient matching leads to an improvement?

Yes! In our appendix, we presented results where we utilized SRe2L [1], a condensation method that performs better on high resolution datasets, and tested our method on ImageNette where it showed an improvement.

[1] Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective. Neurips 2023.

We are very thankful for your constructive feedback. We especially appreciate your acknowledgment of the significance of our research topic and our effort to maintain clear writing. Below, we provide detailed responses to your questions and suggestions:

The result seems to only weakly suggest that the proposed method may exhibit better consistency in terms of adversarial robustness.

We do agree the results may seem weak, but improving adversarial robustness on condensed datasets is not a trivial task. Many of the adversarial defense methods that work well under the real dataset setting do not apply in the condensed dataset setting. This is because one of the fundamental premises of dataset condensation is that no real data will be given to the downstream model at training time, which is important to preserve many of the properties of condensed datasets, including size, training efficiency, and privacy. Without real data, the data distribution that the model learns is already largely different compared to the test images, making it much easier for the adversary to find adversarial examples. Therefore, we don’t think it is realistic to expect the same degree of robustness from condensed datasets and real datasets.

In addition, one reason for the low level of robustness is the inherent low level of clean accuracy. For example, many adversarial defense methods can retain a robustness of around 50%, on a model that can normally achieve 90% clean accuracy on CIFAR10. However, DC can only achieve 50% clean accuracy on CIFAR10, so it is not surprising that when applied the same level of attack, the robustness accuracy deteriorates severely.

In the appendix, we conducted a comparison between our results and those obtained from adversarial training, which is recognized as one of the more effective adversarial defense methods for standard datasets. This comparison demonstrates that our method yields superior results.

The authors only briefly mention that “previous work has shown that regularizing gradients give a false sense of security”

Our proposition highlights that both gradients and curvature impact the upper bound of the robust loss function. We are aware of existing research indicating that gradient-based defenses can create a false sense of security and are vulnerable to certain attacks, so we conducted preliminary experiments that suggested gradient regularization generally underperforms compared to curvature regularization. Based on these findings, we chose to focus on curvature regularization in our paper.

I am unsure if this contracts the statement that “it is important that the condensed data enable the model to perform well both in i.i.d test setting and under adversarial attacks

The condensation method employed in our study is DC, and in the appendix we actually demonstrated improvements in clean accuracy after applying our method. Although this increase in accuracy might partially result from differences in experimental procedures, it importantly indicates that our approach does not lead to a clear decrease in clean accuracy.

We sincerely appreciate your constructive feedback. We are particularly grateful for your recognition of our pioneering efforts in studying this specific topic. Below, we present our responses to your questions and suggestions:

DC was a recent technique that has not been recognized by many people.

Dataset condensation has been proven of interest to the community by its popularity and the large body of related works. Our work is extremely relevant in this area, as several papers [1, 2] have discussed the potential application of dataset condensation in trustworthy machine learning (and hence why we study the topic of adversarial robustness).

[1] Zongxiong Chen, Jiahui Geng, Derui Zhu, Herbert Woisetschlaeger, Qing Li, Sonja Schimmler, Ruben Mayer, and Chunming Rong. A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairness. arXiv preprint arXiv:2305.03355, 2023.

[2] Jiahui Geng, Zongxiong Chen, Yuandou Wang, Herbert Woisetschlaeger, Qing Li, Sonja Schimmler, Ruben Mayer, Zhiming Zhao, and Chunming Rong. A Survey on Dataset Distillation: Approaches, Applications, and Future Directions. arXiv preprint arXiv:2305.01975, 2023.

The hypothesis does not make much sense

The hypothesis is based on the premise that synthesizing a smaller training dataset from a larger one inherently involves discarding some information and features. We believe this selective reduction is a necessary part of the data condensation process.

The novelty of the proposed method is not clear.

GUARD is not just a simple combination of the Dataset Condensation with Gradient Matching (DC) and the CURE method. Unlike the CURE regularizer, our method embeds the regularization into the condensed dataset, allowing for no decrease in actual training time efficiency.

That being said, our work serves as an important foundation for robust dataset condensation and is much more than just the proposed GUARD framework. Not only did we provide a formulation of robust dataset condensation and a comprehensive robustness benchmark of SOTA methods, but we are also the first to theoretically prove that minimizing the adversarial loss of the condensed dataset and minimizing the adversarial loss of the real dataset is only different by a constant factor, which is very useful for future works in this area.

The Experiment is Weak.

Our experiments in the main paper focused on MNIST and CIFAR10 because we attached the regularizer to DC, which is notably less effective with high-resolution datasets. However, in the appendix, we extended our experimentation using SRe2L [3], a more suitable condensation method for high-resolution datasets. This allowed us to test our method on ImageNette and present these results for a more comprehensive evaluation

[3] Zeyuan Yin, Eric Xing, and Zhiqiang Shen. Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective. Neurips 2023.

The improvement is marginal

We recognize that our improvement may seem low compared to normal adversarial robustness papers. This is because improving adversarial robustness on condensed datasets is not a trivial task. Many of the adversarial defense methods that work well under the real dataset setting do not apply in the condensed dataset setting. This is because one of the fundamental premises of dataset condensation is that no real data will be given to the downstream model at training time, which is important to preserve many of the properties of condensed datasets, including size, training efficiency, and privacy. Without real data, the data distribution that the model learns is already largely different compared to the test images, making it much easier for the adversary to find adversarial examples. Therefore, we don’t think it is realistic to expect the same degree of robustness from condensed datasets and real datasets.

In addition, one reason for the low level of robustness is the inherent low level of clean accuracy. For example, many adversarial defense methods can retain a robustness of around 50%, on a model that can normally achieve 90% clean accuracy on CIFAR10. However, DC can only achieve 50% clean accuracy on CIFAR10, so it is not surprising that when applied the same level of attack, the robustness accuracy deteriorates severely.

In the appendix, we conducted a comparison between our results and those obtained from adversarial training, which is recognized as one of the more effective adversarial defense methods for standard datasets. This comparison demonstrates that our method yields superior results.

We greatly appreciate your constructive feedback. We are especially thankful for your acknowledgment of our motivation and the value of our objectives. Below, we have provided responses to your questions and suggestions:

The theoretical part (section 5) is not sound enough.

Regrettably, due to space limitations in the main paper, we had to place the bulk of proof in the appendix. Please view the appendix for a more formal proof of the proposition.

The reference is not comprehensive.

Thank you for suggesting papers regarding sharpness aware minimization and other related topics. We agree that it relates to our topic of study, and have included them in our paper.

Is it $\phi$ or $\theta$ in “where $\phi$ is some neural network and D is a distance function in the parameter space" in page 4?

Yes, thank you for pointing out our type. It is indeed supposed to be $\theta$.

Definition of h is missing

h is a feature extractor, which was defined in the formal proof in our appendix. We have added the definition to the proposition for clarity.

Why is $\tilde{l}(x, v)$ convex given that l(x) is convex?

The first term, $\ell(x)$, is convex by assumption. The second term, $\nabla \ell(x)^{\top} v$, is a linear function in $v$ and inherently convex. The third term, $\frac{1}{2} v^{\top} H v$, is a quadratic form. Given that $H$m the Hessian of a convex function $\ell(x)$, is positive semi-definite, the quadratic form is also convex in $v$. As the sum of convex functions retain convexity, the collective convexity of all three terms ensures that $\tilde{\ell}(x+v)$ is convex.

I think eq(12) requires $h^{-1}()$ to be a contraction map

The assumption in eq(12) is adopted from a specific dataset condensation method outlined in [1]. This method specifically optimizes the synthetic dataset to minimize the distance between the distribution of the real and synthetic data.

[1] Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In IEEE/CVF Winter Conference on Applications of Computer Vision, 2023.