Adversarial Robustness via Deformable Convolution with Stochasticity

Thank you for your detialed comments and your interest in the content of our experiments. We summarize and rebut your 8 major concerns in your comments.

Re 0. The notion of "data independence" needs clarification.[Claims,Q3]

Thank you for correcting our statement. The "data independence" means the hyperparameters at the DCS layer are data independent. However, the DCS layer still needs to be trained. We will revise it in the final version.

Re 1. Clearer training process and hyperparameter choices.[Evaluation]

Thank you for this nice concern. The hyperparameters of the experiments for all baselines and datasets are listed separately in the Sec. 5.1. In terms of the locations of replaced DCS layer, we conduct an ablation study on DCS locations in Sec. B.3, where we changed the location of DCS and find that the second layer is the best for DCS. For clarification, we will add this setting to Sec. 5.1. in the final version as:

• In our experiments, unless specifically labeled, DCS replaces the second convolutional layer. All other layers keep the original settings.

Re 2. Explanation of Lemma 2.[Theorem,Supplementary]

Thank you for your advice. We noticed 2 major concerns for the explanation of Lemma 2, we will improve them in the appendix as follows.

(1) We note that there is no explanation for the symbols $\lfloor\cdot\rfloor$ and $\lceil\cdot\rceil$. This will be explained in line 570 when it first appears as:

• $\lfloor\cdot\rfloor$ for rounding down and $\lceil\cdot\rceil$ for rounding up.

(2) Regarding the proof of Lemma 2, we note that the step from Eq. (23) to Eq. (24) need to be expanded:

For the 3 terms to the right hand side in Eq. 23, the goal is to leave only $n$ into the numerator part. Thus the relationship between $p_u$ and $n$ can be clear. We loose the terms by considering $-\lceil\cdot\rceil$ as $-(\cdot)$ and $\lfloor\cdot\rfloor$ as $(\cdot)$

• For the first term $pu \leqslant 1-\frac{n\alpha^2}{\lceil\alpha\lceil^2k^2}$
• For the second term $p_u\leqslant 1-(\frac{m^2\alpha^2}{k^2}+\frac{n\alpha}{k^2\lceil\alpha\rceil}-\frac{m^2\alpha}{k^2\lfloor\alpha\rfloor})\leqslant 1-(\frac{n\alpha}{k^2\lceil\alpha\rceil})$
• For the third term $p_u\leqslant 1-\frac{n\alpha^2}{k^2\lfloor\alpha\rfloor^2}$
• We set the strictest bounds for $p_u$. Bringing in $\alpha=\frac{k}{S}$, the third term is found to be the strictest: $1-\frac{n}{S^2\lfloor\alpha\rfloor^2}$, as is shown in Eq. 24.

After bringing the bounds in Eq. 24 into Eq. 22 and simplify it, Eq. 25 can be obtained. After transforming the rounding up and rounding down to the corresponding limits, Eq. 26 can be obtained. Finally, considering $\Delta^g \in [0,1]$, Eq. 26 become Eq. 27, which is consistent with Lemma 2.

Re 3. Terminology[Experiment,W1]

Thank you for the constructive suggestion. We will change "pixel" to "point" when coresponding to the DCS kernels in the final version.

Re 4. Explanation of Figure 1.[Experiment,Q2]

Thank you for your constructive comments. The circle marked as $\mathbb{K}$ in Figure. 1 represents the potential distribution of all DCS kernels. DCS automatically samples a random kernel from K at each forward propagation.

Re 5. Connections to other relevant domains.[RelatedWorks,Q4]

Thank you for your valuable comments. We have compared DCS against other stochastic adversarial defenses methods in Table 2. We divided the stochastic adversarial defenses into random weights and random structures. We are happy to add certified defenses and stochastic activation functions to the table and cite related work. Table 2 will then be expanded as the following table (only expanded part and ours).

Re 6. Is Step 4 trained alternately or in a unified manner?[Q1]

Thank you for your valuable questions. To avoid misunderstanding, we first claim that the randopm sampling processes in Figure 1 are not trainable.

For the sampled kernels, we trained each kernel alternatively. Only one kernel is trained during each forward propagation, regardless of whether it is normal adversarial training or GSAT. Therefore we consider this to be alternating training.

Re 7. Would combining DCS with adversarial training further improve robustness?[Q5]

Yes. AT will further improve robustness. Corresponding results were compared in our experiments by Sec. 5.3.2 and Fig. 4(a). The results show that using AT or GSAT imporves the performance of DCS.

Reference

[1] Certified Defenses for Adversarial Patches, ICLR 2020.

[2] Adversarial Defense Via Data Dependent Activation Function and Total Variation Minimization, ICLR 2019.

Your expert comments are constructive for our paper. We summarize and rebut your 4 major concerns in your comments.

Re 0. Claim of generalization and data independence equires empirical support.[Claims,W2]

Thank you for suggesting additional experiments to validate our claim. To verify the sensitivity of DCS hyperparameters to more distributions, we extended our experiments on STL-10[1] using ResNet18 as the baseline. The input size keeps the same as raw image as $96$. The results are shown in the following table:

This results show the effectiveness of DCS on different distributions without changing the settings. We will include this experiment in the final version.

Re 1. Additional experiments against attacks.[Evaluation,W1]

Thank you for helping us to refine our experiments.

(1) We conducted adversarial robustness tests on CIFAR-10 using ResNet18 for query-based, transfer-based and adaptive attacks that you mentioned. We expanded the results to Table 6 as

In the table, SQUARE and Pixel attacks are included in the initial paper. We added 4 columns to the table. The added attacks used the following settings:

transfer-FGSM:          base model=pretrained WRN50_2, epsilon=16/255
transfer-FGSM-base:     base model=pretrained baseline model, epsilon=8/255
BPDA:                   epsilon=8/255，max steps=20, learning rate=0.5
BPDA+EOT:               epsilon=8/255，max steps=20, learning rate=0.5, EOT steps=3.

It can be seen that DCS appears to be robust against black-box attacks. We attribute this to the fact that the random masking of the convolutional kernel by DCS also shields a portion of the adversarial perturbations. For ataptive attacks like BPDA and EOT-BPDA, DCS also shows high robustness. We attribute this to randomness in conjunction with gradient masking.

We will use the expanded Table 6 alone and analysis in the final version.

(2) For attacks where adversarial samples are added to the training set, some predefined adversarial samples are added to the training set in both traditional adversarial training(AT) and GSAT. Related results of normal AT and GSAT can be found in Table 1 and Table 3.

Re 2. Connections to other relevant domains like certifiable defenses could be drawn.[RelatedWorks]

Thanks for this nice suggestion. We will add certifiable defense to Table 2 in the final version as:

Re 3. Concern about computational overhead.[Q1]

Thanks for this nice concern. We recorded the training and reasoning times for DCS+GGSAT vs. baseline+AT. We choose RseNet18 as the baseline. For training, we use CIFAR-10 training-set (50000 examples) following the hyperparameters:

epochs: 200
batch size: 128
optimizer: SGD
weight decay: 5e-4
initial learning rate: 0.1
scheduler: multiste (lr/10 at epoch 60 and 120)

For reasoning, we use CIFAR-10 test-set (10000 examples) with batch size=1024. We recorded the entire time cost for training and reasoning in the table below.

Despite the difference in training time, their time consumption during inference is very similar. The time gap in training is due to the introduction of additional parameters in the DCS layer. However, during inference, the extra time consumption caused by the extra parameters is not as dramatic as training due to the absence of backpropagation.

We are delighted to add this comparison in the appendix in the final version.

Reference

[1] An analysis of single-layer networks in unsupervised feature learning, Journal of Machine Learning Research - Proceedings Track 15, 215–223 (01 2011).

[2] Explaining and harnessing adversarial examples, in International Conference on Learning Representations, 2015.

[3] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples, ICML 2018.

[4] Certified Defenses for Adversarial Patches, ICLR 2020.

Thank you for your pertinent review and your interest. Your review is summarize and rebut by 3 major concerns.

Re 0. Ablation study by replacing downsampling layers.[Experiment,Q1]

This is an interesting problem. Your subtle experimental design help us to study the effect of stride in a limited network structure. We compare the results by replacing the DCS to the downsampled layer (layer 6) V.S. the closest normal convolution layer (layer 5, 7). The results are shown in the table below.

We notice that the performance of DCS on downsampling layer is bettter than the normal convolution layers. According to the lemmas, larger sreide $S$ helps to reduce the assumed small numbers $\epsilon_c$ and $\epsilon_l$ in the lemmas. It results in a smaller gradient similarity and output distance, and finally increase the robustness and clean accuracy.

In addition, larger $S$ also helps the bounds in both lemmas to be numerically looser in practice. This helps to find a suitable $n$ easier.

Re 1. Explanation regarding the instability of GSAT in Table 3 and potential connection to the Equations.[W1,Q2]

This is a constructive question. We will first explain the source of instability in GSAT and with Eq. 13, and then analyse the potential connection of Table. 3 with Eq. 12.

(1) According to Eq. 13, GSAT modifies the sample space sampled by the DCS layer in forward propagation during training by selection. Below we explain the reason for the instability of GSAT shown in Table 3.

We have demonstrated both theoretically and experimentally that paths in $\mathbb{X}^g$ and $\mathbb{X}^u$ are not easy to attack, while paths in $\mathbb{X}^s$ are very easy to be attacked by gradient-based algorithms (see Eqs. 7, 10 and Sec. 5.3.3 for details). This means that when the network is attacked, the gradients generated by the paths in $\mathbb{X}^g$ and $\mathbb{X}^u$ will be in different direction from the attacked gradients generated by the paths in $\mathbb{X}^s$.

To minimize the influence of inaccurate attacked gradients in training, we select and remove paths in $\mathbb{X}^s$ from $\mathbb{X}$, and build the selected random space, as demonstrated in Eq. 13. This corresponds to step 11 in Algorithm 1. The selected random space is used for sampling paths in the forward and backward propagation for each training step sepreately.

We understand that using GSAT the network are only limited optimized on the paths in $\mathbb{X}^s$ under each adversarial examples. So when the DCS samples paths in $\mathbb{X}^s$ in infernce, there are performance decrement compared to the other sampled paths. The decrement will cause the instability in GSAT. The larger probability the performance drop in DCS, the greater instability in GSAT.

(2) For the relationship of results in Table 3 and Eq. 12, it mainly connected by $n$. From Table. 3, we can find that with larger $n$, GSAT will be more unstable. We give an explanation as: when Eq. 12 gives larger bounds for $n$, which refers to larger $n$ in practice, the percentage paths in unselected random space will increase. This increases the probability for DCS performance decrement in the networks using GSAT for training, which leads to greater instability. Thus, as shown in Table 3, $n$ is suggested to be small for a stable GSAT.

The explanation of instability will be added in the appendix and the analysis of connection between Table 3 and Eq. 12 will be added in experiment section in the final version to helping audiances in understanding the nature of GSAT.

---

In addition, we would like to argue that the paths in $\mathbb{X}^s$ are inherently difficult to optimize, given the difficulty of obtaining the correct gradients after a precision gradient-based attacked. Together with the results in Table 3 demonstrating that the network remains robust even with performance decrement. So we consider the performance decrement to be acceptable.

Re 2. Unreported robust accuracy of Non-AT baselines.[W2,Q3]

Thank you for your valuable suggestions.

We supplemented the results of the NON-AT baseline in the table below. Cos in the table means gradient cosine similarity.

We will add these results as bars in Figure 4(a) in the final version.

Thank you for your expert comments and your interest in the content of our experiments. We summarize and rebut your 6 major concerns in your comments.

Re 0. Evaluate under using BPDA+EOT attack.[Method,Experiment,W2,Q3]

Thanks for this nice concern. We evaluated DCS under BPDA and BPDA+EOT attacks on CIFAR-10, following the hyperparameters:

epsilon=8/255，max steps=20, learning rate=0.5, EOT steps=3.

In the final version, we will expand Table 6 to include:

DCS is robust under BPDA+EOT attacks. We attribute this to randomness in conjunction with gradient masking.

Re 1. Both proofs assume certain distributions and independence properties of the gradients that might not hold exactly in practice.[Theorem]

Thanks for this nice concern. In practice, the data should be normalized. We assume the worst case of all normalized data to ensure that both lemmas stands in any data distribution after normalization. This does not mean that the worst case will necessarily occur. Therefore, in practice, the values of these two bounds is looser.

We would like to emphasize that the worst case assumptions in proofs are specified on data distributions with infinite data points. This avoids the dependence of the bounds on specific data sets. This is the reason why the DCS setup can be data-independent.

Re 2. Significant adaptations to apply DCS to other model architectures such as Transformers.[W1]

Thanks for this nice concern. We believe this question reveals the future direction in randomized structure defense.

(1) We confirm that DCS is tailor-made for convolutional operations. We believe that convolution is still an important tool in image processing. Transformer is out of the research scope of this work.

(2) However, we briefly explored that DCS fits Vision Transformer(ViT). ViT uses a $16\times16$ convolution in patch embedding. Large kernel size makes lower bound of $n$ increases and hinders finding a suitable $n$. To avoid this, we notice that patch embedding can be split into multiple concatenated $3\times3$ convolutions[2]. Our baseline follows the settings in [2] and then replace the second $3\times3$ convolution with DCS. We obtained the following results

DCS works well with ViT-tiny and we will add this experiment in appendix. We would like to argue that due to time constraints, the network is roughly trained on CIFAR-10 with two stages:

Stage 1: Fix a pretrained ViT-tiny and train the FC layer and convolutions from scratch with:

-epochs: 200
-batch size: 128
-optimizer: SGD
-weight decay: 5e-4
-initial learning rate: 0.01
-scheduler: multiste (lr/10 at epoch 50 and 100)

Stage 2: Adversarial finetune the entire network using GSAT with:

Stage 2:
-epochs: 90
-batch size: 128
-optimizer: SGD
-weight decay: 5e-4
-initial learning rate: 0.01
-scheduler: cosine

Re 3. How to ensure that the randomness introduced by DCS and GSAT is properly controlled?[Q1]

Thanks for this nice question.

(1) For DCS, we control randomness by $n$ through Lemma1,2. Other setting in DCS are decided by the replaced convolution layer, to keep the data dimensions constant. The stride keeps the same. The kernel size is added by $2$, while padding is adapted accordingly.

(2) For GSAT, the size of random space is smaller but fixed after removing the masks, since the number of removed masks is unchanged in each step.

Re 4. How to ensure robustness improvements are not a result of gradient masking?[Q2]

Thank you for your constructive comments on the additional experiment. To validate this, we manually cancel randomness. Instead, we selected two fixed deformable convolutional kernel $X^i$ and $X^j, n=4$ with no repeated points. $X^i$ is attacked and $X^j$ is used for reasoning. The results are

The robustness is mainly from randomness. We will add this observation into the appendix.

Re 5. How is selective masking performed and how this impacts the clean Acc and robustness?[Q4]

Thanks for this nice concern.

(1) According to Algorithm 1, GSAT record the included points in the DCS kernel when generating the adversarial training examples. Then, all masks that unmask the recorded points are banned, until the end of current forward propagation. With the other masks, DCS will be trained with kernels sampled in $\mathbb{X}^g$ and $\mathbb{X}^u$.

(2) From Eq(7,10), $\mathbb{X}^g$ and $\mathbb{X}^u$ helps to enlarge the gradient cosine similarity and minimize the output distances, which leads to higher robustness and clean Acc.

Reference

[1] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples, ICML 2018.

[2] Early convolu-tions help transformers see better, NIPS 2021.