Robustness Reprogramming for Representation Learning

W4: We want to emphasize that our proposed method is orthogonal to the existing methods including adversarial training. It can be combined with adversarial training to further improve the robustness. Our proposed method can be considered as a post-hoc approach for any well-trained models, which reprograms the vanilla architecture into a robust one.

In terms of training time, we pre-train the model with adv-train technique for 200 epochs, after which we fine-tune the hybrid architecture for only 3-5 epochs. They are not directly comparable as they operate at different stages. We also acknowledge that our architecture includes additional computational costs; however, this is not the primary focus of our work. Most importantly, we would like to highlight the significant contributions of our work to the field of robust machine learning as explained in W1.

In terms of the number of parameters, we include only a few additional {${\lambda}$}, which are negligible in large-scale models.

W5: We believe we have explained and discussed the suitable scenarios for each paradigm in Section 4. We are happy to provide further clarification here:

• Paradigm 1 (without fine-tuning): is suitable when the model is large-scale and pre-trained, and there is no available training data. In this case, we can simply adjust the hyperparameters {${\lambda}$} to achieve the desired balance between clean and robust performance.

• Paradigm 2 (fine-tuning ${\lambda}$ only): is suitable when the large-scale backbone models have substantial parameters, but only limited data is available. In this scenario, we can freeze the model parameters and fine-tune only {${\lambda}$} to obtain the optimal layer-wise combination of the hybrid architecture.

• Paradigm 3 (overall fine-tuning): is suitable when excellent robustness is required, and it is acceptable to fine-tune the models for a few epochs.

W6: To provide an overview of the scaling cost, we also include a comparison of the inference running time between the vanilla LPM and our NRPM across various layers in the table below. As shown, our NRPM incurs only 3-4 times the inference cost of the LPM, which we consider acceptable. For the question "if/how this is better than AT", please refer to W4.

Table 15: Running time with ResNet as backbone.

W1: Thanks for pointing out. Our idea of this work comes from the robust statistics and optimization. We are glad to further explain and summarize the idea and contributions here.

The idea logic of Nonlinear Robust Pattern Matching (NRPM):

(a) We first reveal the common module in all the deep learning models, that is, the linear feature transformation, $z = \mathbf{a}^\top \mathbf{x} = \sum_{d=1}^D a_d \cdot x_d$.

(b) The linear feature transformation $z = \mathbf{a}^\top \mathbf{x}$ has a underlying inverse optimization problem, Ordinary Least Squares (OLS) problem:

where the first-order optimality condition $\frac{\partial \mathcal{L}(z)}{\partial z} = 0$ yields the optimal solution $z^* = \sum_{d=1}^D a_d \cdot x_d$.

(c) OLS estimation is highly sensitive to outlying values due to the quadratic penalty, so we choose to use linear penalty which give a robust alternative Least Absolute Deviation (LAD) estimation:

(d) LAD estimation is non-smooth, which requires an Newton-IRLS algorithm to approximate the estimator and unroll it as the neural network layer as:

The significant contributions can be found in the common response.

W2: Thank you for your comments. We have corrected the typos and provided clearer explanations in the revised version accordingly.

(a) Consider a feature data points {${a_d x_d}$}$_{d=1}^D$. $X$ represent the random variable of feature empirical distribution $F$.

(b) $y = z_{LPM}/D$ is defined as in the following formulation after “=>”. This definition is introduced for the sake of notational simplicity in the proof of Theorem 3.2 in the appendix. We will further clarify it in the revised version.

(c) {${\lambda}$} represents a set of layer-wise hyperparameters. Each layer will have one $\lambda$.

(d) $N$ is a typo and should be $D$, which is the dimension of the input data.

W3: We have included the detailed explanation in the Hidden Embedding Visualization part in Section 5.3. And we are glad to explain it again here: In Figure 5, we visualize the clean embeddings ($x$ or $z_i$, first row) and the attacked embeddings ($x'$ or $z_i'$, second row) of LPM (first column) and NRPM (second column). We then compare the clean-attacked embedding differences between LPM and NRPM.

It is evident that under attack, in the vanilla LPM, the hidden embeddings are more seriously destroyed, and the output probability changes significantly, which explains its reduced robustness. We will revise the caption with more detailed explanations following your kind suggestions.

Thank you for clarifying these details and for updating the manuscript with corrections/clarifications. I am raising my score since the contributions of this work are significant and are well-supported by empirical results.

W1: We emphasize that our proposed method is orthogonal to existing methods and can be combined with them to further improve performance. Additionally, PGD, FGSM, C&W, and AutoAttack are commonly used and reliable evaluation methods in this field, as demonstrated in most related papers [1][2][3] including the two you mentioned.

To further validate the effectiveness of our method, we also include the baselines and attacks in our evaluation, as shown in the following table. For DeepFool, we also include the results across various attack iterations. The experiments demonstrate excellent robustness compared to the baselines under various attacks.

References:

1. Tack, Jihoon, et al. "Consistency regularization for adversarial robustness." 2022.
2. Liu, Zhenyu, et al. "Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks." 2024.
3. Wu, Dongxian, Shu-Tao Xia, and Yisen Wang. "Adversarial weight perturbation helps robust generalization." 2020.

Table 7: Adversarial robustness on CIFAR10 with ResNet18 as backbone.

Table 8: Adversarial robustness on CIFAR10 with ResNet18 as backbone.

W2: We want to emphasize that the tasks we try are not easy since robustness is a very challenging problem (for instance, robust accuracy on CIFAR10 is not yet high yet), and currently existing works suffer from a clear performance limitation. This work proposes fundamental research to explore new strategies for a transformative breakthrough. The size of the backbone and dataset are not the focus of this paper. Instead, we focus on delving deep into the investigation, even starting from very basic deep learning units such as MLPs and LeNets, hoping to provide clear evidence to demonstrate the advantages of our technical ideas, excluding the impact of irrelevant designs.

Following your suggestions, we have included additional experiments on CIFAR-100, Tiny-ImageNet, and WideResNet28-10, as shown in the following tables. Due to time and computational constraints, we only present the results for Paradigm 1.

From the results, we observe that, even without fine-tuning (Paradigm 2), our method can significantly improve over the vanilla backbone, achieving gains of up to 15.94%, 19.3%, and 20.2% in the experiments on CIFAR-100, Tiny-ImageNet, and WideResNet28-10, respectively.

Table 9: Robustness reprogramming on CIFAR100 with ResNet18 as backbone.

Table 10: Robustness reprogramming on Tiny-Imagenet with ResNet50 as backbone.

Table 11: Robustness reprogramming on CIFAR10 with WideResNet28-10 as backbone.

W3: Following your suggestion, we include a comparison of the inference running time between the vanilla LPM and our NRPM in the table below. As shown, our NRPM incurs only 3-4 times the inference cost of the LPM, which we consider acceptable.

Table 12: Running time with ResNet as backbone.

Additionally, we also report the training time of different methods with the same device. As shown in the table, our paradigm 3 requires 2-5 times training time of other methods per epoch. however, we can only slightly finetune the pretrained model for only 5 epochs, which largely reduces the total training time.

Table 13: Training time with ResNet18 as backbone in RTX A6000.

Last, we want to point out that our main contribution lies in proposing a novel approach to improve robustness, which is orthogonal to existing methods. Regarding efficiency, while we acknowledge that our proposed architecture introduces additional, yet acceptable, computational costs, our robustness reprogramming improves robustness in a lightweight manner, either without tuning or with slight tuning.

Q1: We believe our evaluation is already adaptive and comprehensive enough to avoid the issue of gradient masking:

• In our proposed NRPM layer, there is no non-differentiable operator, which mitigates concerns about differentiability issues.
• Our evaluation using AutoAttack already includes the black-box square attack, which does not rely on gradients.
• In Section 5.4, we also evaluate the certified robustness using randomized smoothing, which theoretically guarantees robustness under any attack.
• We include a visualization analysis of the hidden embeddings in Section 5.3 to further validate the robustness of our proposed robust layer.
• Following your suggestions, we evaluate our model with extremely large attack strength and observe that the robust accuracy is almost zero.

Table 14: Gradient Masking Test on ResNet18

Q2: We believe we have explained and discussed the suitable scenarios for each paradigm in Section 4. We are happy to provide further clarification here:

• Paradigm 1 (without fine-tuning): is suitable when the model is large-scale and pre-trained, and there is no available training data. In this case, we can simply adjust the hyperparameters {${\lambda}$} to achieve the desired balance between clean and robust performance.

• Paradigm 2 (fine-tuning ${\lambda}$ only): is suitable when the large-scale backbone models have substantial parameters, but only limited data is available. In this scenario, we can freeze the model parameters and fine-tune only {${\lambda}$} to obtain the optimal layer-wise combination of the hybrid architecture.

• Paradigm 3 (overall fine-tuning): is suitable when excellent robustness is required, and it is acceptable to fine-tune the models for a few epochs.

Second, our method already achieves the best balance between clean and robust performance among the existing adversarial defenses. From the results shown in the following table, we can observe the following:

• Compared to SAT and MART, our method significantly improves both clean and robust performance.

• Compared to PGD-AT, TRADES-2.0, AWP, and DYNAT, our method demonstrates a significant advantage in robustness while achieving comparable clean performance.

• Although TRADES-0.2 and Consistency yield better clean performance than our method, their robust performance is significantly inferior. In fact, we can use the hyperparameter in the loss of TRADES to adjust our model fine-tuning to achieve similar clean performance but much better robustness compared with these methods.

From these observations, we can conclude that our method significantly improves robustness while maintaining comparable clean performance compared with existing approaches.

Table 2: Adversarial robustness on CIFAR10 with ResNet18 as backbone.

Third, there is an inherent trade-off between clean and robust performance in any model. Compared to standard training, adversarial training will reduce clean performance while enhancing robustness. As demonstrated in Table 4 of TRADES [1], the hyperparameter in the loss of TRADES can be adjusted to balance clean and robust accuracy through model training. Orthogonal to these training strategies, our method can adjust this balance by simply adjusting hyperparameters $\lambda$ in the inference process, which is also a notable contribution. This provides unprecedented opportunity for adaptive inference (without fine-tuning), which is a promising future direction for this field.

We hope this can fully address your concern. Please kindly let us know if any further clarification is needed.

[1] Zhang, Hongyang, et al. "Theoretically principled trade-off between robustness and accuracy." International conference on machine learning. PMLR, 2019.

W1: {${x_d}$}$_{d=1}^D$ represents the initial input, and $x_0$ denotes the perturbation introduced to the data. To avoid confusion, we have changed $x_0$ to $\Delta x$ and updated the notation accordingly in the revised version.

W2: The influence function measures the sensitivity of model predictions to introduced perturbations, which is highly correlated with adversarial robustness. Our theoretical justification builds on this perspective to further validate the robustness implications of our methods, in addition to empirical experiments.

In fact, the influence function is well established in robust statistics and has recently been applied to adversarial robustness. Here, we list several papers [1][2][3] that highlight the close relationship between influence functions and adversarial robustness.

References:

1. Lai, Lifeng, and Erhan Bayraktar. "On the adversarial robustness of robust estimators." IEEE Transactions on Information Theory 66.8 (2020): 5097-5109.
2. Deng, Zhun, et al. "Interpreting robust optimization via adversarial influence functions." International Conference on Machine Learning. PMLR, 2020.
3. Cohen, Gilad, Guillermo Sapiro, and Raja Giryes. "Detecting adversarial samples using influence functions and nearest neighbors." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.

W3: Thanks for your insightful comments. The influence function is introduced to measure the sensitivity of the model to the perturbation, which is useful no matter under bounded or unbounded $\ell_p$ adversaries. This work indeed focuses on $\ell_p$ bounded attack like PGD, FGSM, and AA, since they are the most commonly used evaluation.

To validate the effectiveness of our method under unbounded attack, we include the experiments under DeepFool with different iterations, which do not restrict the bound on the attack budget. As shown in the following table, our method can outperform other baselines across all the iterations. When the number of iteration goes into extremely large (unbounded), all the model will have nearly zero accuracy.

Table 3: Adversarial robustness on CIFAR10 with ResNet18 as backbone.

Additionally, we believe the papers you mentioned [1][2] fall into the category of non-$\ell_p$ attacks, which we consider as a very interesting topic. We also conducted some experiments in this area earlier, but we did not include them in the initial submission. As shown in the table below, our NRPM architecture demonstrates improvements over the vanilla LPM in most cases.

Table 4: Robustness under non-$\ell_p$ attack on CIFAR-10

W4: Thanks for pointing out, we have included more information in the revised version.

Q1: Following your suggestion, we also include a comparison of the inference running time between the vanilla LPM and our NRPM in the table below. As shown, our NRPM incurs only 2-4 times the inference cost of the LPM, which we consider acceptable.

Table 5: Running time with MLP as backbone.

Additionally, we want to point out that our main contribution lies in proposing a novel approach to improve robustness, which is orthogonal to existing methods. Regarding efficiency, while we acknowledge that our proposed architecture introduces additional, yet acceptable, computational costs, our robustness reprogramming improves robustness in a lightweight manner, either without tuning or with slight tuning. Most importantly, we want to emphasize again the significant contributions of our work to the current field of robust machine learning, as highlighted at the beginning of our response.

Q2: Refer to Q1.

Q3: We believe our evaluation is already adaptive and comprehensive enough to avoid the issue of gradient masking:

• In our proposed NRPM layer, there is no non-differentiable operator, which mitigates concerns about differentiability issues.
• Our evaluation using AutoAttack already includes the black-box square attack, which does not rely on gradients.
• In Section 5.4, we also evaluate the certified robustness using randomized smoothing, which theoretically guarantees robustness under any attack.
• We include a visualization analysis of the hidden embeddings in Section 5.3 to further validate the robustness of our proposed robust layer.

Q4: The NRPM approach is suitable for training from scratch, but it cannot improve over Paradigm 3. As shown in the table, NRPM sacrifices clean performance, which limits its ability to enhance robustness. In contrast, our Paradigm 3 can simultaneously learn the optimal combination of {${\lambda}$} and model parameters to improve both natural and robust performance, outperforming NRPM. Certainly, we believe that training a hybrid model from scratch would be comparable to Paradigm 3, but the computational cost would be prohibitively expensive. This further highlights the necessity of our proposed robustness reprogramming.

Table 6: Adversarial robustness on MNIST with 3-layer MLP as backbone.

Q5: Refer Q3.

W1: In the first equation, no assumption is made about the prediction error. The OLS formulation $L(z)$ is directly derived from the inverse optimization problem of the linear feature transformation $z = \mathbf{a}^\top \mathbf{x} = \sum_{d=1}^D a_d \cdot x_d$. Specifically, the first-order optimality condition, $\frac{\partial L(z)}{\partial z} = 0,$ yields the linear transformation $z^* = \sum_{d=1}^D a_d \cdot x_d.$

W2:

• The $w_d$ is not the parameter to be learned; instead, it is computed as $w_d^{(k)} = \frac{a_d x_d^{(k)}}{\sum_{d=1}^D a_d x_d^{(k)}}$ at the $k$-th layer in the neural network, where $x_d$ represents the input value, $a_d$ is the parameter to be learned, and $z^{(k)}$ is the hidden embedding at the $k$-th layer.

• In Alg 1, the entire {${x_d}$}$_{d=1}^D$ is one data sample, and $d$ is the index of dimension. So the approach will find independent $w_d$ for each dimension $d$ in each sample.

• The parameters/hyperparameters in the model to be optimized include two parts: (1) model parameters {${a_d}$}$_{d=1}^D$, same as backbone model, (2) balance hyperparameter {${\lambda}$} introduced in hybrid parameters. The parameters can be optimized into two phases:

  • Phase 1 (Pre-training): the model parameters {${a_d}$}$_{d=1}^D$ is pre-trained and saved in backbone model.
  • Phase 2 (Fine-tuning): this phase can be divided into three paradigms as in Section 4, including manually selecting the hyperparameter {${\lambda}$} with fine-tuning (paradigm 1); only fine-tuning {$\lambda$} while freezing the {${a_d}$} (paradigm 2); fine-tuning both the {${a_d}$} and {${\lambda}$} at the same time (paradigm 3).

• The pseudo-code for implementation details have been included in the Alg 2 in Appendix A.1 due to space limit in the submission. Moreover, we have provided the available code in https://anonymous.4open.science/r/NRPM-322C/.

W3: Following your suggestion, we also include a comparison of the inference running time between the vanilla LPM and our NRPM in the table below. As shown, our NRPM incurs only 3-4 times the inference cost of the LPM, which we consider acceptable.

Table 1: Running time with ResNet as backbone.

Additionally, we want to point out that our main contribution lies in proposing a novel approach to improve robustness, which is orthogonal to existing methods. Regarding efficiency, while we acknowledge that our proposed architecture introduces additional, yet acceptable, computational costs, our robustness reprogramming improves robustness in a lightweight manner, either without tuning or with slight tuning. Most importantly, we want to emphasize again the significant contributions of our work to the current field of robust machine learning, as highlighted in our common response.

W4: We compare the baselines including: PGD-AT [1], TRADES [2], MART [3] and SAT [4] and AWP [5]. For fair comparison, we conduct adversarial training on the baseline from scratch. We apply the same training setting as ours by setting training batch size as 128, weight decay as 2e-5, momentum as 0.9. We train the models with 200 epochs with adjusted learning rate (0.1 for 0-100 epochs; 0.01 for 100-150 epochs; 0.001 for 150-200 epochs). For TRADES, TRADES-2.0 and TRADES-0.2 means that we set the sensitivity of regularization hyperparameter $\beta$ in TRADES as 2.0 and 0.2, respectively.

References:

1. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, "Towards deep learning models resistant to adversarial attacks.", 2017.
2. H. Zhang, Y. Yu, J. Jiao, E. Xing, L. El Ghaoui, and M. Jordan, "Theoretically principled trade-off between robustness and accuracy.", 2019.
3. Y. Wang, D. Zou, J. Yi, J. Bailey, X. Ma, and Q. Gu, "Improving adversarial robustness requires revisiting misclassified examples.", 2020.
4. L. Huang, C. Zhang, and H. Zhang, "Self-adaptive training: beyond empirical risk minimization.", 2020.
5. D. Wu, S.-T. Xia, and Y. Wang, "Adversarial weight perturbation helps robust generalization.", 2020.

W5: Thank you for pointing that out. We apologize for the mistake made while copying the results of AutoAttack. We have double-checked all the experimental results and have updated the correct values in the revised version. The conclusions remain the same after correcting the results.

Q1: This technique is a well-established theory in robust statistics, so we will briefly explain it here. I am happy to provide more details as follows:

• Intuitively, when adversaries perturb the input, the residual $z / D - a_d \cdot x_d$ tends to increase. Consequently, the quadratic penalty on the residual will dominate the Ordinary Least Squares (OLS) estimator, resulting in a shift towards those dominating input pixels and significantly impacting the representation of the output. By replacing the quadratic penalty (OLS estimator) with a linear alternative (LAD estimator) on the residual $z / D - a_d \cdot x_d$, the impact of outliers can be mitigated. The robustness of OLS and LAD estimators has been well validated in the field of robust statistics.

• For a simpler example, consider the mean estimator as the OLS estimator and the median estimator as the LAD estimator. If there are perturbations in the data points, the median is generally more robust and less sensitive to such perturbations.

Q2: Yes. ”robust training” means ”adversarial training”.

Q3: Simply replacing LPM with NRPM will result in clean performance drop. As shown in the following table, when $\lambda$ is set to 0, our model becomes equivalent to NRPM, leading to a reduction in natural clean performance to 18.8%. This degradation in clean performance can be significantly mitigated through fine-tuning, which improves the natural performance to 78.7%. However, this is still not the optimal case. As demonstrated by Paradigm 3 (tuning all), we can simultaneously learn the optimal combination of {${\lambda}$} and model parameters to enhance both natural and robust performance. Therefore, it is necessary to construct a hybrid architecture and learn the optimal combination of LPM and NRPM.

Table: Robustness reprogramming under 3 paradigms on MNIST with 3-layer MLP as backbone.

Q4: The {${x_d}$}$_{d=1}^D$ is the initial input, and $x_0$ represents the perturbation introduced to the data. To avoid confusion, we have changed $x_0$ to $\Delta x$ and updated the notation accordingly in the revised version.

Q5: The numbers at the top of each column are the budgets of attack. The attacks used are FGSM (Table 1 & 2) and PGD (Table 4). The budgets are both measured with the $L_\infty$ norm.

Q6: $w_i$ is not a parameter to be learned but is computed from features $x_d$ and pre-trained model parameters $a_d$. The only parameter to be learned is $\lambda$. Each layer has a single parameter $\lambda$ that needs to be learned.

Q7: The bar plots show the probabilities of the data being classified into different categories by the models.

Q8: Thank you for pointing that out. (1) and (2) correspond to (a) and (b). We have corrected the errors in the updated version.