Training Robust Ensembles Requires Rethinking Lipschitz Continuity

We thank the reviewer for their insightful comments. Below are our responses to their questions and concerns:

1. The soundness of the claim in this paper and Missing related work and discussion:

We want to emphasize that we do not claim that Lipschitz continuity does not help with making the ensembles more robust (they are indeed more robust than ensembles without any modification). However, the increased transferability rate might hinder their effectiveness, which shows itself in results such as the ones presented in Table 2.

This, in fact, can be explained by theoretical results in the paper [4] you mentioned. That paper has become available after submitting this work, but we believe the connections to their result can improve the discussion of our paper. Similar to their theoretical observation, there is a trade-off between complexity and diversity. Enforcing Lipschitz continuity works as a regularizer to decrease the complexity but decreasing the complexity leads to reduced diversity. That is why we need an additional modification, such as LOTOS, to further boost the diversity despite the reduced complexity caused by Lipschitz continuity. There might be other better methods for dealing with this trade-off, but we believe LOTOS will set a reasonable base-line for that. The connections between how LOTOS increases diversity and the diversity term introduced in [4] will also be useful for our discussion in the revised version of our manuscript.

Paper [3] would be a great addition to our discussion to justify the better generalization of adversarial examples generated by models with controlled operator norm, which completely matches our observations (e.g., Figure 1), and motivates the necessity of further training strategies, such as LOTOS, to counteract this improved generalization which potentially leads to higher transferability rates.

The lower-bounds on the probability of transferability in [2] (similarly in [1]) show the connection with the smoothness of the model in terms of the Lipschitz continuity of the gradients (not Lipschitz continuity of the outputs which is a milder assumption), but we believe their results are aligned with our observations and will be good additions to our discussions.

We think [5] is an insightful work that motivates avoiding the extension of basic intuitions and observations for simple models to neural networks when they are used in the form of an ensemble, as it might not lead to the expected benefits one expects from an ensemble of models. This aligns with our observations (e.g., Table 2) and further motivates a full understanding of neural network models when used in ensembles to find better strategies that are worth the additional resource requirements for training these models.

We would like to thank the reviewer for pointing us to the listed papers. We believe all the mentioned papers are useful in our work and improve our motivation and discussion sections.

2. Experiments:

We are running experiments for adding other ensemble training methods (DVERGE). We will post the results here once they are ready. We also added results for other attack methods and $p$-norms as well as other types of noise (blur, pixelated, etc.) (please see the response to reviewer 1).

3. What is the definition of $\Re$ in the appendix (page 14)?

We used it to denote the real part of the complex numbers. We will add to our notations. Thanks!

4. Do you think the method in this paper requires significant work on hyperparameter tuning?

For the hyper-parameters of the models and the training of the ensemble (learning rate, optimizer, scheduler, etc.) we used all the default hyper-parameters of these repositories, as they are, in our experiments:

[1] https://github.com/kuangliu/pytorch-cifar [2] https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble

For the hyper-parameters introduced in LOTOS loss ($mal, k, w_i$), after experimenting with the effect of $k$ and noticing that a value of $1$ is enough, we performed all our experiments with $k=1$, which automatically sets $w_1=1$ (no other $w_i$). For the ablation study on $mal$, we tried different values (Appendix B.6.2), but for all other experiments, we did not do hyper-parameter tuning and fixed it as $0.8$. We think the results can be improved by tuning these hyper-parameters but we did not invest the time as we think we got the results that showcase the effectiveness of our method.

Thank you for your detailed response! It seems I had some misunderstandings about the derivation. I’ll raise my score to 6, good luck!~

I really appreciate the authors for their hard work, especially the extensive experiments comparing to other methods (as suggested by other reviewers). I'm also happy to see LOTOS perform better than others. My major concerns have been addressed.

While some reviewers have considered the empirical part, I have a question for the theoretical part (the proof of Proposition 3.3, which is a core theoretical part in your motivation). Please let me know how the fourth line is derived in Appendix A.1, i.e., $\mathbb{E}\_x\left[\sup \_{\|\delta\|_2<r}\left(\ell\_{\mathcal{F}}(x, y)-\ell\_{\mathcal{G}}(x, y)\right)+2 L\|\delta\|_2\right] \leq \mathbb{E}\_x\left[\left(\ell\_{\mathcal{F}}(x, y)-\ell\_{\mathcal{G}}(x, y)\right)+2 L r\right].$ I'm not sure why $\sup\_{\|\delta\|_2<r}$ has been cancelled.

Dear authors,

I deeply appreciate your work, especially the experimental contributions. However, I regret to note that the novelty of the proof in Appendix A.1 appears to be quite limited, leading me to believe that it provides relatively little new insight. The motivation of this paper is about Lipschitz constant, which may be considered by [1] (see below). In other words, the motivation in this paper has already been hinted at by previous work [1]. That is the reason that why I do not raise the score anymore.

Specifically, the proof relies on the global Lipschitz constant, which closely resembles the first-order Taylor expansion. However, [1] considers second-order Taylor expansion. Also, since the definition of transferability in this paper is similar to that in [1], a more comprehensive discussion of [1] may be warranted. I have several questions and comments:

• Motivation for Lipschitz continuity and its comparison to [1]: Proposition 3.3 provides a strong motivation to investigate the role of Lipschitz continuity in transferability rates for this paper. However, [1] offers a much more extensive theoretical exploration than Proposition 3.3 in this paper. Notably, [1] employs second-order Taylor expansion to conduct a fine-grained analysis of gradients, smoothness, and other related properties. In contrast, the proof in this paper simply leverages the Lipschitz constant to simplify the analysis.
• Insights on gradient magnitude: The findings in [1] emphasize the critical role of gradient magnitude in transferability (see Theorem 2 in [1]), which is closely tied to the Lipschitz constant. In particular, Lipschitz constant is the upper bound of gradient magnitude. This connection suggests that deeper insights into gradient behavior could enrich the theoretical results in this paper. In other words, the analysis in Proposition 3.3 is less interesting than [1] since it only consider the upper bound of gradient magnitude. In contrast, [1] considers the gradient magnitude itself as well as the consine similarity, smoothness, etc.
• Transferability bounds: The proof of Lemma 5 from [1] provides valuable insights into the upper bound of transferability. It demonstrates that transferability is constrained by the disagreement between two classifiers and the total variation distance between adversarial and clean data distributions (See $\operatorname{Pr}(f(\mathcal{A}(x)) \neq g(\mathcal{A}(x))) \leq \operatorname{Pr}(f(x) \neq g(x))+\rho$ in the proof of Lemma 5). Likewise, Proposition 3.3 in this paper bounds transferability using the product of the Lipschitz constant and radius, along with a disagreement term, i.e., $\left|R\_{\mathcal{F}}\left(\mathcal{A}\_{\mathcal{F}}(x), y\right)-R\_{\mathcal{G}}\left(\mathcal{A}\_{\mathcal{F}}(x), y\right)\right| \leq \left|R\_{\mathcal{F}}(x, y)-R\_{\mathcal{G}}(x, y)\right| + 2 L \epsilon$. I mean, they may bring similar insights. You may explain the connection between them.

Given these points, I find that both the findings and proof techniques in Proposition 3.3 is covered by [1]. This overlap diminishes the novelty and theoretical significance of Proposition 3.3 in this paper. However, I still appreciate the authors' hard work and I still like to keep my score as 6 if there is no other concern raised by other reviewers.

[1] TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness. NeurIPS 2021.

We thank the reviewer for their insightful comments. Below are our responses to their questions and concerns:

1. How about the white-box case?

We performed the same attack we used for the black-box setting, PGD-50 with $\ell_\infty \leq 0.04$, as well as two other stronger attacks AutoAttack [1] and MORA [2]:

However, we note that since our focus is on the transferability of adversarial examples from surrogate models and diversifying the models in the ensemble, our experiments were mainly on black-box attacks. For the black-box attacks, using Definition 3.2., we disentangle true transferability from the attack difficulty by conditioning on the examples that the attack is already successful on the surrogate model and computing the ratio of them that transfers to the target model. The same disentanglement is done from the accuracy of the models. This is in contrast to the previous definitions of transferability (e.g., [3]), in which using a stronger attack can lead to a higher transferability rate.

Also, another motivation behind focusing on black-box attacks aligns with some recent efforts on training ensemble methods that are robust against transferability attacks because in most scenarios whitebox attacks are impractical [4].

2 Using LOTOS as the surrogate model?

Here are the results that can be compared to the ensembles of three models in Table 2.

As expected, using ensembles trained with LOTOS as the surrogate models leads to a stronger black-box attack against clipped models. Still, the ensembles trained with LOTOS are more robust against these attacks, as well.

3 All definitions in this paper seem to build on the $\ell_2$ norm.

Robustifying the model by enforcing layer-wise Lipschitz constant has been utilized to guarantee $\ell_2$ robustness, which itself is an upper-bound for $\ell_\infty$ as well. Although LOTOS is proposed as a solution to this trade-off, by orthogonalizing the transformations, it promotes different behavior from the corresponding layers of the models with respect to any perturbation in the inputs. This leads to the diversity of the models regardless of the norm that has been used for computing the adversarial samples. In fact, in our experiments, we have used the default setting of the attacks from [5], which uses PGD-50 attack with $\ell_\infty$ norm.

4 Are all models in the ensemble trained simultaneously?

The definition of the loss given in equation 4, as is, is applicable to simultaneous training of the models of the ensemble; however, we can also train them sequentially, and for training each new model make sure that its layers are orthogonal to the top-$k$ singular vectors of the corresponding layers of the previously trained models. We have not performed any experiment with the latter case, though, and as it is common in other robust ensemble training methods (TRS, DVERGE) only considered the case where the models are trained simultaneously. Still what you pointed out might be another benefit of using our method since it makes it easier to add additional models to the previously trained ensembles and reduces the required resources.

5 Line 285 says... Are these not contradictory?

In line 285, we are discussing the general affine layers. In fact, in fully connected layers, we do need a high value of $k$ to ensure satisfactory orthogonalization because even if the top singular vector of a layer is orthogonal with respect to the other layer, still the other singular vectors can be aligned. Later in that section, we show that for “convolutional” layers, because of their intrinsic properties, even $k=1$ can be effective because it also induces orthogonality with respect to other singular vectors, to some extent (Theorem 4.1.). And that is what makes our approach highly efficient for convolutional layers. We will make this clear to avoid confusion.

6 In Theorem 4, which one is dominant?

By using LOTOS, the value of $\epsilon$ can be forced to be very small (complete orthogonalization), but because of the limitations of convolutional layers and also minimizing the cross-entropy loss, it might not become zero for some layers. (see Figure 7 in Appendix). On the other hand, the second term can be forced to be smaller by choosing a higher weight decay. Also, it becomes naturally smaller for the more significant singular vectors by decreasing $\frac{p}{n}$. So the dominating factor depends on different choices of the hyper-parameters and we have not evaluated that.

7 Do you think the method will extend to other popular architectures such as vision transformers?

Since prior works on ensemble robustness focused only on ResNet (or similar) models, we also chose them for our experiments. However, since our approach is applicable to any affine layer and is specifically much more effective for convolutional layers, it is interesting to see the effect of our approach on more complex models that contain these layers. That being said, that will be a stretch because for controlling the spectral norm, we have to train ensembles of these models from scratch and cannot use pre-trained models. Thus, it is unlikely that we will be able to provide these results in the brief rebuttal window.

8 Line 215 -- How good is this approximation?

Proposition 3.3. is general and does not consider any specific loss function as long as the Lipschitz continuity condition holds. The point of this proposition is just to motivate that if we have the Lipschitz continuity property, the population loss on the adversarial samples would not differ much between the surrogate model and the target model, and that difference can depend on the Lipschitz constant. As you mentioned, the difference in the empirical loss would depend on the specific data samples and might vary to some extent, however, our empirical results show the same effect (e.g., Figure 1).

9 In Equation (3), how would we determine $w_i$?

They are just hyper-parameters that can be tuned in hyper-parameter search. As the reviewer suggested, the singular values can be a reasonable and natural choice and that is what we considered as well. However, in our experiments, because we used $k=1$, that choice does not matter as we only have one singular vector and $w_1$ can be absorbed by $\lambda$ and be set to $1$.

10 Line 260...

We should have replaced the index $i$ that goes from $1$ to $d$ in the summation to something else like $l$ to avoid confusion and make it like $|| \sum_{l=1}^d \sigma_l u_l v_l^\top v_i' ||_2 = 0$. Note that when $k=1$ we are orthogonalizing the largest singular vector of each layer with respect to the whole transformation of the other layer, which includes all its singular vectors. So the largest singular vector of each layer cannot be aligned with the second singular vector of the other layer or any of its other singular vectors. However, with $k=1$, we are only enforcing this orthogonalization for the first singular vectors of each layer with respect to the other layer, and still, the second singular vector of a layer can be aligned with the second singular vector of the other layer or any of its other singular vectors (except the first one). This in fact can be true for fully connected layers and is related to your other question and that is why for fully connected layers we need to consider higher values for $k$ for LOTOS to be effective. However, Theorem 4.1. proves that for convolutional layers by just orthogonalizing the first singular vectors, we get orthogonalization of the other singular vectors, to some extent, for free.

[1] Croce et al., Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML 2020.

[2] Yu et al., MORA: Improving Ensemble Robustness Evaluation with Model-Reweighing Attack. NeurIPS 2022.

[3] Yang, Z., Li, L., Xu, X., Zuo, S., Chen, Q., Zhou, P., ... & Li, B. (2021). Trs: Transferability reduced ensemble via promoting gradient diversity and model smoothness. Advances in Neural Information Processing Systems, 34, 17642-17655.

[4] Sitawarin, C., Chang, J., Huang, D., Altoyan, W., & Wagner, D. PubDef: Defending Against Transfer Attacks From Public Models. In The Twelfth International Conference on Learning Representations.

[5] https://github.com/AI-secure/Transferability-Reduced-Smooth-Ensemble

We thank the reviewer for their insightful comments. Below are our responses to their questions and concerns:

1 What are the white-box attacks that you used in this paper?

As detailed in Appendix B.1, the white-box attack that we use is set up to evaluate the transferability rates among the models within an ensemble, and therefore it is not an attack on the whole ensemble. The attacks that we use for the whole ensemble are black-box attacks using surrogate models. The white-box attack, which is used only for measuring the transferability rate among the models within the ensemble, attacks each individual model using PGD-50 in a white-box manner to generate adversarial samples and then evaluates the other models of the ensemble on those samples to see what portion of them are transferred to them (according to Definition 3.2). We called it white-box to emphasize the use of one of the models within the ensemble as the surrogate model, but it is as if we are performing a black-box attack on the other models of the ensemble.

2 TRS was shown to have an overestimated robustness

We would like to emphasize that LOTOS does not depend on TRS and the purpose of combining that with TRS is merely to show that it can be combined with other ensemble robustness approaches to further improve their robustness. LOTOS is a tangent to other defense mechanisms and has been proposed as a solution for increased transferability rate when Lipschitz continuity is used for the robustness of individual models.

Lipschitz continuity has been used by prior works to make individual models more robust. Now assume that someone decides to use Lipschitz continuity to increase the robustness of each individual model in the ensemble and potentially incorporate that with another approach such as TRS to boost the robustness of the ensemble. We have shown in our work that in contrast to the expected outcome, the Lipschitz continuity that leads to more robust individual models also increases the transferability rate among them and prevents the desired boost in the robustness of the ensemble. LOTOS provides a solution to this increase in the transferability rate while allowing the ensemble to benefit from the robustness effect of Lipschitz continuity on individual models.

Similarly, in Appendix B.8, we showed that LOTOS does not interfere with the robustness gains from other approaches designed for boosting the robustness of individual models, such as adversarial training. Our goal for the two experiments in 5.5 and B.8 is to show that LOTOS can be used alongside other defense mechanisms for adversarial robustness.

3 It is not clear to me why DVERGE was not considered in the baseline comparisons

Thank you for bringing DVERGE to our attention. We will perform our experiments with DVERGE as well and present the results shortly.

4 For eq. (3):

• Where is $w_i$ introduced?

The line right after the equation explains that “$w_i$’s are arbitrary weights which are non-increasing with $i$ to emphasize “more importance” for the singular vectors corresponding to top singular values”

• Why does this equation measure similarity?

It measures how similar the transformations of two layers are to one another. The more aligned the singular vectors of the transformation matrices of affine layers are, the more similar they behave on the input space. Also, for each layer, the top singular vectors explain the majority of the transformation. Also note that as mentioned after the equation, the maximum value is attained when both layers are the same, and the minimum value (0) is attained when each layer is orthogonal with respect to the top $k$ singular vectors of the other layer. In general, the more aligned their top singular vectors are with one another, the higher the value will be.

• What are $A$ and $B$ matrices, are they model parameters?

As mentioned in the line before the equation they are the corresponding matrix transformations of two corresponding affine layers $f^{(j)}$ and $g^{(j)}$.

• Also given that $A$ and $B$ are trainable, do you backprop through the SVD of them?

When backpropagating through the loss in equation 4, the singular vectors are fixed. The backpropagation computes the gradient with respect to the parameters of layers $f$ and $g$ and updates them to minimize $S$. Then the new singular vectors of both layers are computed to recompute the Loss in equation 4 for the next iteration. Note that since the change in the parameters of the layers are small in each iteration, for computing the singular vectors, by reusing the singular vectors of the previous iteration during the PowerQR iteration, we converge to the correct singular vectors in a few steps of power iteration [4].

8 Could you compare your methods with other ensemble defenses, especially with DVERGE

We are running experiments for training DVERGE ensembles to compare the effect of incorporating LOTOS using our current attacks as well as AutoAttack and MORA to compare with the ones trained with TRS. We will report the results here once they are ready.

[1] Sedghi, H., Gupta, V., & Long, P. M. (2018). The singular values of convolutional layers. arXiv preprint arXiv:1805.10408.

[2] Senderovich, A., Bulatova, E., Obukhov, A., & Rakhuba, M. (2022). Towards practical control of singular values of convolutional layers. Advances in Neural Information Processing Systems, 35, 10918-10930.

[3] Delattre, B., Barthélemy, Q., Araujo, A., & Allauzen, A. (2023, July). Efficient bound of Lipschitz constant for convolutional layers by gram iteration. In International Conference on Machine Learning (pp. 7513-7532). PMLR. [4] Boroojeny, A. E., Telgarsky, M., & Sundaram, H. (2024, April). Spectrum Extraction and Clipping for Implicitly Linear Layers. In International Conference on Artificial Intelligence and Statistics (pp. 2971-2979). PMLR.

[5] Gray, R. M. (2006). Toeplitz and circulant matrices: A review. Foundations and Trends® in Communications and Information Theory, 2(3), 155-239.

5 For eq. (4): There is no $k$ in the LHS.

Equation 4 shows the training loss which constitutes the regular cross-entropy loss for training the models (left side of the ‘+’ sign) and LOTOS loss (right side of ‘+’ sign). $k$ is a hyper-parameter chosen apriori which affects the computation in equation 3 and defines the number of top singular vectors that we want to consider for orthogonalization.

6 In Theorem 4.1. it is not clear to me how it is proven for $k = 1$

To make it more clear, first let's rewrite a simplified version of equation (3) (without some of the hyper-parameters, such as $w_i$ and $mal$ for only one layer (so we can drop the $(j)$ superscript for clarity):

$S_k(f,g) = \sum_{i=1}^k ||f(v_i^\prime)||_2 + ||g(v_i)||_2$.

With the definitions in the theorem, we can simplify this further to:

$$ S_k(f,g) = \sum_{i=1}^k || A v_i^\prime ||_2 + ||M_2(v_i)||_2 $$

Focusing on only the sum of the first term, the theorem proves that if we only enforce $||A v_1^\prime ||_2$ to be small, that also entails an upper-bound for $||A v_i^\prime ||_2, i \geq 2$, therefore, we do not need to use a large $k$ in the computation of equation 3 to ensure the orthogonalization; even using $k=1$ automatically implies small value for $||A v_i^\prime ||_2, i \geq 2$ and that makes it much more computationally efficient.

7 Why circular padding?

The choice of circular padding for proving theoretical results on convolutional layers is a common practice ([1,2,3,4]) and allows for theoretical reasoning about these layers. Convolutional layers with other types of padding are expected to act similarly to the corresponding convolutional layers with circular padding ([1,3,5]). That being said, we emphasize that in our experiments we did not make any modification to the padding type (e.g., zero padding or no padding) or other configurations of the convolutional layers of the models that we used, which verifies the extension of our theoretical results to other types of padding as well.

8 It is not clear to me how this theorem links with eq. (4).

As mentioned in section 4, the higher values of $k$ entail a larger orthogonalization among the two corresponding affine layers, but increasing this value would make the computation of equation 4 more costly. This Theorem entails that, for convolutional layers, unlike dense layers, the value of $k$ does not need to be large for $S_k$ (defined in equation 3), and a small value of $k$ (e.g., 1) would imply the orthogonalization of the other singular vectors to some extent. On the contrary, for dense layers, this is not true; if one chooses $k=1$ for dense layers (e.g., fully connected layers), the only thing that LOTOS does is that it enforces the first singular vectors of the two layers to be orthogonal. However, the second singular vectors might be completely aligned with each other because they are independent of the directions of the first singular vectors.

8 Could you consider stronger attacks such as AutoAttack [b] and MORA [a]?

Here are the results on the robust accuracy against whitebox attacks on the whole ensemble (all the models of the ensemble are provided to the attacker):

The two new attacks are indeed more powerful in the whitebox setting. We also tried these two attacks in the black-box setting we used for our experiment, by attacking the surrogate ensemble and using the generated adversarial examples to evaluate the target ensemble:

It is interesting to see that although these two attacks were stronger in the white-box setting, the adversarial examples they find might be too specific to the surrogate ensemble preventing them from generalizing to the target ensemble. Both results can be insightful and we will add them to our revised manuscript.

Thank you for your detailed response. I have raised my score as most of my confusion are addressed. It is promising that by combining DVERGE with LOTOS, it can achieve a much higher robust accuracy. However, the question regarding its competitiveness wrt DVERGE remains.

We would like to thank the reviewer for going through our responses to their constructive comments and increasing their score. As the reviewer mentioned, using LOTOS with DVERGE ensembles increases their robust accuracy against MORA white-box attacks by $9.5$ percentage points which we believe is very notable and makes a great difference.

DVERGE has proved to be an effective method for increasing ensemble robustness and it has introduced novel ideas for increasing model diversity through their distillation loss and cross-adversarial training. However, we believe comparing LOTOS alone with DVERGE is not a fair comparison because LOTOS increases the training time of ensembles of ResNet18 only by $2.4$ times per epoch (see Appendix B.5) whereas DVERGE increases the training time by a factor of $14.3$ because of all the different techniques they use in their algorithm. If we only add adversarial training to LOTOS, which leads to almost the same (a factor of 14.6) increase in the training time (see Appendix B.8) for a fair comparison, then the robust accuracy against MORA becomes $49.5$ ($\pm 1.45$), which is $29.8$ percentage point higher than DVERGE alone. We reported results of LOTOS alone without its combination with adversarial training because we wanted to disentangle its effectiveness from other common techniques for boosting adversarial robustness. But we will add this result to the revised version of our manuscript to highlight the advantages of our proposed method.

We want to emphasize that still LOTOS without adversarial training achieves high adversarial robustness, with much lower running time than other techniques such as TRS and DVERGE, and this is a very important factor because it makes LOTOS practical for larger models. So we believe all both DVERGE and LOTOS can have their place in the world and be used depending on the available resources and needs.

With all of this additional experimentation we have conducted and the additional analysis we have provided on the run-time of different methods and our new result on robust accuracy of LOTOS with adversarial training against MORA attack, we hope the reviewer will consider increasing their score, particularly since all other reviewers have found our work an example of “empirical observation of interesting deep learning phenomena” and see LOTOS as “a solution that uniquely focuses on orthogonalizing affine layers, setting it apart from existing methods that target different aspects of model diversity”. Also, we believe the concerns raised by the reviewer have been experimentally resolved (including this particular note on fair comparisons between DVERGE and LOTOS) in this rebuttal window.

4. Limitation 4: Weak models?

As the reviewer mentioned, the models that have a lot of room for improvement from the robustness perspective are easier to modify to increase their robustness. Although we used ResNet and DLA models in our experiments (which is common practice for prior works on ensemble robustness), we tried to show LOTOS can improve the robustness, whether it is used for non-robust models (Table 1), or when used with models that are made individually robust using adversarial training (Table 6 in Appendix B.8), or when used with ensembles that are made robust using other ensemble robustness methods (Table 3). All these models have different robustness levels to begin with, but we showed that LOTOS consistently makes an improvement in the robustness of these models. Note that in addition to these experiments, we also performed some experiments on ResNet models without batch norm layers, which has been shown to improve the robustness of the models (Table 5).

Furthermore, we have shown that LOTOS improves over models that are Lipschitz continuous, which are shown to be much more robust and target reaching the maximum robustness by limiting the change in the output for small changes in the input [5,6,7]. However, this approach has the robustness of individual models as the goal and what we have shown is that they will be less effective when used in the form of an ensemble.

To further address the reviewer’s concerns, we believe the rebuttal timeline would allow us to perform some of our basic experiments with ResNet50 models as well, which have been shown to be much more robust than ResNet18 models [8]. We will report the results here once they are ready. Also note that we already have some experiments that include ResNet34 models, which are more robust than ResNet18 models as well (see Section 5.4 and Appendix B.7).

[1] Croce et al., Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML 2020.

[2] Yu et al., MORA: Improving Ensemble Robustness Evaluation with Model-Reweighing Attack. NeurIPS 2022.

[3] Hendrycks, D., & Dietterich, T. (2018, September). Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. In International Conference on Learning Representations.

[4] Yang, G., Duan, T., Hu, J. E., Salman, H., Razenshteyn, I., & Li, J. (2020, November). Randomized smoothing of all shapes and sizes. In International Conference on Machine Learning (pp. 10693-10705). PMLR.

[5] Szegedy, C. (2013). Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199.

[6] Cohen, J., Rosenfeld, E., & Kolter, Z. (2019, May). Certified adversarial robustness via randomized smoothing. In international conference on machine learning (pp. 1310-1320). PMLR.

[7] Boroojeny, A. E., Telgarsky, M., & Sundaram, H. (2024, April). Spectrum Extraction and Clipping for Implicitly Linear Layers. In International Conference on Artificial Intelligence and Statistics (pp. 2971-2979). PMLR.

[8] Peng, S., Xu, W., Cornelius, C., Li, K., Duggal, R., Chau, D. H., & Martin, J. (2023). Robarch: Designing robust architectures against adversarial attacks. arXiv preprint arXiv:2301.03110.

We thank the reviewer for their insightful comments. Below are our responses to their questions and concerns:

1. Limitation 1:

We performed additional experiments using AutoAttack [1] (used in evaluations of RobustBench) as well as MORA [2] (which is specifically designed for white-box attacks on ensembles and suggested by reviewer uLeN) to generate the adversarial examples on the surrogate models and used Definition 3.2 to compute the robust accuracy against transferability-based attacks (similar setting to the PGD-50 with $\ell_\infty \leq 0.04$ reported in Table 2).

2. Limitation 2:

In general, adversarial samples can be seen as the worst-case change within the used $\ell_p$ norm ball, and robustness against these samples entails accuracy against other changes within that $\ell_p$ norm ball. Still, to evaluate these methods on other types of noise, we used the benchmark introduced by [3] for CIFAR-10 and report the results in the table below:

As the results show C=1 ensembles work better than the original ensembles because they are trained to be less sensitive to changes in the inputs. LOTOS ensembles are more robust than the other two because, in addition to benefiting from the Lipschitz continuity introduced in C=1 ensembles, they further diversify the models for how to react to changes in the input.

3. Limitation 3:

The enforced Lipschitz continuity on the models of the ensemble provides robustness against a certain $\ell_2$ norm radius of the samples. We have used a surrogate ensemble and used PGD-50 with $\ell_\infty \leq 0.04$ to generate adversarial examples. The choice of $\ell_\infty$ allows the adversarial attack to be more flexible and therefore more difficult to defend against [4]. We then measure the transferability rate using Definition 3.2. Here, we have performed some experiments with other attack sizes, as well as other norms ($\ell_2$ and $\ell_1$). As we can see with larger attack sizes the effectiveness of our method becomes more clear.

Results for PGD-50 with different attack sizes:

Results for PGD-50 with $\ell_1 \leq 0.04$ and $\ell_2 \leq 0.04$: