Efficient local linearity regularization to overcome catastrophic overfitting

We thank the reviewer e1KG for their feedback. We have made the changes in the text visible in red. We address below their concerns.

• (Q1): Not conceptually novel

(A1): We respectfully disagree with this statement. Admittedly, the spirit of enforcing local linearity has been well applied as a solution to CO. However, there are several key contributions introduced in our paper:

• Our method is the first to avoid the 3x time cost coming from Double Backpropagation while being based on the same local linearity principle.
• We theoretically link our regularization term and LLR [1] as approximants of second-order directional derivatives (Prop. 1 and 3). Compared to LLR, ELLE poses the advantage of not requiring double backpropagation.
• We develop the first method to adaptively regularize local linearity and showcase its benefits.
• We showcase the appearance of Delayed CO for several other methods and demonstrate that adding our regularization can resolve this issue.

While our approach maintains simplicity, the results in the paper demonstrate the robustness of the method to tackle CO and outperform SOTA for large $\epsilon$. If the reviewer has noticed any of the previous contributions in related papers, we are happy to reevaluate this list.

• (Q2): Why not use Finite Differences to approximate LLR or GradAlign?

(A2): Let us explain why Finite Differences don’t fit well in this setup. We theoretically link LLR and ELLE by showing both regularization terms approximate second-order directional derivatives (see Prop. 3 and 1). Therefore, using a Finite Differences approximation of the gradient in the LLR case is unnecessary.

We elaborate on the approximation of GradAlign with Finite Differences in Appendix B.14. Estimating the gradients via Finite Differences is computationally expensive with $d+1$ function evaluations needed for estimating every gradient ($d = 248,832$ for ImageNet). Additionally, when more efficient approximations are involved [2], the regularization term becomes non-smooth due to the appearance of the sign function and it is not directly differentiable, as we would desire for a regularization term. Moreover, it requires one more function evaluation than ELLE. If the reviewer has in mind an alternative way to compute the finite differences, we would be happy to discuss this further.

• (Q3): Missing References.

(A3): We are thankful for the references that link curvature and robust generalization. We accordingly cite [3] and [4] in the introduction. However, we emphasize that there exist distinct differences between these papers and our method.

[3] analyze the role of the curvature of the activation functions in robust generalization when performing AT. Our method is not based on architectural design, but the optimization method used to train a given architecture. The combination of low curvature architecture design and regularization terms to enforce local linearity is an interesting future direction.

[4] propose a method for controlling the global curvature of twice differentiable neural networks to improve robust generalization. Our method controls the local curvature of the network and is not restricted to twice-differentiable classifiers.

• (Q4): Experiments ignore a canonical adversarial defense: PGD

(A4): AT is an effective but expensive defense. This is precisely the reason that faster single-step alternatives flourished [6,7], trading off performance for a 10x speed up. The main target of our work is handling critical failures and weaknesses of single-step adversarial training methods. Ideally, we would like single-step methods as reliable as AT PGD-10 but without the computational burden of PGD.

To address the reviewer’s concern, we have included the AT PGD-10 evaluation in Fig 1, Fig 2, Fig 3 and Fig 5. The ability of AT PGD-10 to generate locally linear models is well known and displayed in Fig. 2, where both the gradient misalignment and our local linear approximation error are controlled during training. N-FGSM+ELLE-A is the method that attains the closest performance to AT PGD-10.

• (Q5): Isn’t enforcing local constancy (GAT and NuAT) a special case of local linearity?

(A5): We thank the reviewer for the attentive reading. We have revised the text in section 3.3.

Let us clarify that in the original submission, we already had comparisons against GAT. As studied in [5], GAT and NuAT are not able to overcome CO for large values of $\epsilon$. In Fig 5, we show that GAT suffers from CO for all cases except PRN at $\epsilon=8/255$. When combined with ELLE-A, GAT training is stabilized and CO does not appear.

We will be happy to answer any other comments or questions the reviewer might have.

We thank the reviewer vQnz for their feedback. We have made the changes in the text visible in red. We address below their concerns.

• (Q1): Can you include the evaluation with other SOTA adversarial attacks?

(A1): Our models are evaluated with AutoAttack [1] (See Tab. 1 and 2 and Fig. 5), which contains the SOTA attacks FAB [2] and Square [3]. Moreover, AutoAttack is the standard evaluation metric in the RobustBench benchmark. We are not aware of more effective adversarial attacks than the AutoAttack ensemble. We will be happy to include any other SOTA adversarial attacks in the evaluation if the reviewer has any concrete methods in mind.

• (Q2): Can you integrate your regularization term with other SOTA adversarial training methods other than FGSM?

(A2): Yes, please let us explain how we have already integrated ELLE to other methods. In this paper, we present a thorough evaluation against SOTA single-step adversarial training methods. ELLE(-A) can be integrated as a plug-in regularization term into any other adversarial training method. This is demonstrated in the paper by combining ELLE-A with FGSM, GAT [4], and N-FGSM [5] (See Fig. 3 to 5). As we demonstrate, our method is effective to overcome CO when combined with all these methods.

Additionally, during the rebuttal we have included an analysis of the integration of ELLE-A with single-step variants of AdvMixUp [6] and TRADES [7]. These two methods are only able to overcome CO when in combination with ELLE-A, see Tab. 3 and Fig. 20.

We will be happy to answer further comments the reviewer might have.

References:

[1] Croce and Hein, Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks, ICML 2020

[2] Croce and Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack, ICML 2020

[3] Andriushchenko et al., Square Attack: a query-efficient black-box adversarial attack via random search, ECCV 2020

[4] Sriramanan et al., Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses, NeurIPS 2020

[5] de Jorge et al., Make Some Noise: Reliable and Efficient Single-Step Adversarial Training, NeurIPS 2022

Dear reviewer vQnz,

We are thankful for the time invested in reviewing our work. The reviewer was concerned about a) the attacks used for evaluating our models b) the possibility of combining our regularization term with other SOTA robust training methods. Our rebuttal further motivates the use of AutoAttack, clarifies the usage of our method in combination with GAT and N-FGSM and extended to combine with AdvMixUp and TRADES.

If the concerns of the reviewer are addressed, we would appreciate it if the reviewer increases their score. If the reviewer still has any remaining concerns, we are more than happy to clarify further.

Best regards,

Authors

Dear Reviewer vQnz,

we are grateful for your feedback and appreciation of our work. We are thankful for increasing the score.

Best,

Authors

We thank the reviewer 4XDh for their feedback. We have made the changes in the text visible in red. We address below their concerns.

• (Q1): Equation (1) is overly excessively simplistic, additional explanations are needed. Deep models and training losses do not conform to linear mappings.

(A1): We thank the reviewer for highlighting this issue. We agree with the reviewer in that deep models and training losses are usually non-linear. However, even though they are globally non-linear, it has been observed that the loss of robust models behaves linearly in small regions around data points, see [1] and related citations in the third paragraph of the introduction. As an example, the output of Deep NNs with the ReLU activation function is highly non-linear, but piecewise linear, and thus they are locally linear in small regions.

We have clarified this in the preamble of equation (1) and accordingly updated equation (1) to better capture this intuition.

• (Q2): The loss function usually remains constant throughout the training process, how does this transition from linear to non-linear occur in the loss?

(A2): “The loss function usually remains constant”: We believe the reviewer refers to the minimization objective, e.g., the cross entropy loss. This is indeed kept the same throughout the training process in AT [2]. Nevertheless, the behavior of the loss landscape with respect to the input data evolves in time depending on the parameters of the model [1]. Let $h(\theta_t, x, y) = L(f_{\theta_t}(x), y)$, we are interested on the local linearity of $h$ w.r.t. $x$. The loss transitions from locally linear to non locally linear in the sense that for some $t = t_1$, we have that $h$ behaves linearly with respect to $x$, but it does not for some other $t = t_2 > t_1$. This behavior is clearly displayed in Fig. 2 (b-c), where the local linear approximation error suddenly increases at epoch 15 when training with FGSM.

• (Q3): The local, linear approximation error resembles the AdvMixUp formulation, why do you not compare against it?

(A3): We are thankful to the reviewer for the suggestion. We conduct the related experiment in Appendix B.13. The convex combination used in the design of the input points and labels given by AdvMixUp resembles the convex combination used for obtaining the point $x_c$ in our method. However, there are fundamental differences between AdvMixup and ELLE. Let $\delta_{\text{adv}}$ be the adversarial perturbation given by some adversarial attack:

• AdvMixUp assigns the one-hot vector corresponding to the true class to $x$ and uniform class probabilities to the point $x + 2\delta_{\text{adv}}$. Then, every point in the convex combination of $x$ and $x + 2\delta_{\text{adv}}$ is assigned a convex combination of the aforementioned labels.
• ELLE enforces the training loss to be locally linear in the whole region $x + \delta: ||\delta|| \leq \epsilon$ via a loss involving a convex combination of terms.
• In the single-step scenario, i.e. $\delta_{\text{adv}} = \epsilon \cdot \text{sign}(\nabla_{x}L(f_{\theta}(x), y))$, AdvMixUp suffers from CO while our method does not.

We include an experimental evaluation of AdvMixUp and its combination with ELLE-A in Appendix B.13. Showing that combined with ELLE-A, CO is avoided, see Tab. 3 and Fig. 20.

• (Q4): ELLE-A is worse than ELLE in the long schedule but it is not present in the text.

(A4): We thank the reviewer for highlighting this missing explanation in the text. This is expected because the ELLE-A regularization is weaker. So overall, the model can still overfit in long schedules. ELLE is a stronger regularization and test performance decreases much less with time. We have included a related sentence in section 4.4.

• (Q5): The proposed regularization method does not perform well in small epsilon values and short schedules.

(A5): Guarding against attacks with a small perturbation budget doesn’t guarantee the resistance against attacks with a larger budget. Additionally, large perturbation radiuses can still produce imperceptible perturbations as we show in Fig. 6 for ImageNet. Small radiuses are also not so relevant when studying CO because it does not appear. In fact, even standard FGSM with $\epsilon=2/255$ can be used for ImageNet training without the appearance of CO, see Tab. 1.

The biggest improvements with our method occur when $\epsilon$ is large. However, when combining ELLE-A with N-FGSM, AA accuracy is even improved for $\epsilon \leq 8/255$ in SVHN and is comparable for CIFAR10/100, see Table 2 in the appendix.

• (Q6): Why use long schedules and large $\epsilon$?

(A6): Long schedules have been used for other existing robust training methods [4,5]. Other works argue the avoidance of CO in short schedules gives a false sense of robustness [6,7]. Additionally, the evaluation in long schedules is a standard practice [1,3]. The avoidance of CO in longer schedules is an additional benefit of our method. When training in less studied datasets where the appropriate length of the schedule is unknown, our method is safer against CO.

Regarding larger $\epsilon$, we should aim at being robust to the largest possible perturbations. Our attack visualization (See Fig. 6) does not reveal a change in the human prediction with the $\epsilon$ values employed.

• (Q7): ELLE appears to be an approximation of LLR, LLR should be included in the main comparisons.

(A7): We thank the reviewer for the suggestion. Both LLR and ELLE approximate the second order derivatives up to constant factors. This is proven in Prop. 3 (End of the appendix) and Prop. 1 respectively. Considering this fact, the lower computational cost of ELLE favors its usage. The comparison against LLR is usually not included in the literature [1,3]. Moreover, our results in Fig. 1 show that LLR attains a significantly lower adversarial accuracy than ELLE.

We are currently running additional experiments on CIFAR100 and SVHN. We will post those results and revise the respective tables during the rebuttal period. We include an ablation on the $\lambda$ parameter for LLR in Appendix B.12, Fig. 19, and the corresponding CIFAR10 results in Fig. 3 (b). LLR does not match the performance of ELLE(-A), especially for large $\epsilon$.

We will be happy to answer any other comments or questions the reviewer might have.

References: [1] Andriushchenko and Flammarion, Understanding and improving fast adversarial training, NeurIPS 2020

[2] Madry et al., Towards Deep Learning Models Resistant to Adversarial Attacks, ICLR 2018

[3] de Jorge et al., Make Some Noise: Reliable and Efficient Single-Step Adversarial Training, NeurIPS 2022

[4] Rice et al., Overfitting in adversarially robust deep learning, ICML 2020.

[5] Xu et al., Exploring and exploiting decision boundary dynamics for adversarial robustness, ICLR 2023

[6] Kim et al., Understanding catastrophic overfitting in single-step adversarial training, AAAI 2021

[7] Li et al., Towards Understanding Fast Adversarial Training, arXiv 2020

Dear reviewer 4XDh,

We are thankful for your effort to review our work. The main concerns expressed are the a) non-linear mappings learning, b) comparisons with AdvMixUp, and c) comparisons with LLR and d) the need for long schedules and large $\epsilon$. Our rebuttal addresses those core concerns along with the rest of the questions of the reviewer.

In addition, we present have concluded the LLR experiments promised in our first response and include them in (Fig. 19) and its evaluation (Fig. 3). We find LLR is more sensitive to hyperparameters and is not able to match the performance of ELLE(-A).

If the concerns of the reviewer are addressed, we would appreciate it if you re-evaluate our submission. If the reviewer still has any remaining concerns, we are more than happy to clarify further.

Best regards,

Authors