Revisiting Adversarial Examples from the Perspective of Asymptotic Equipartition Property

Lack of Comparative Analysis: The paper provides naive empirical results without adequately comparing the proposed adversarial training method to current state-of-the-art approaches (as shown in Table 2 and Figure 5). Additionally, there is insufficient comparison between the results of this method and standard adversarial training (which does not introduce extra noise), as well as a lack of discussion on the balance between clean and adversarial training sets in regular AT (also in Figure 5).

Insufficient Theoretical Discussion: The paper lacks a discussion of existing theories that address the notion of adversarial examples being off the data manifold. There is little exploration of how the concept of non-typical examples relates to the similarity between off-manifold examples and those deemed non-typical.

This paper is not very clearly written and therefore difficult to evaluate. I think even if all technical issues are addressed, the paper would need to re-written to be reproducible, which, in my opinion, is necessary prior to being publication-ready.

• The notion of "natural" samples and "adversarial" samples is not formally defined, despite being used formally (such as in Lemma 1 & 2). Related, Lemmas 1 & 2 are not stated formally.
• The definitions of Non-Typical I and Non-Typical II are not given.
• The definition of Typical perturbations is not clear, see Questions section.
• It's not precise how the methods were implemented, how exactly is adversarial training performed and how is adversarial accuracy evaluated? (See questions)
• In Table 1, it's not clear what the rows represent—on the top, you have listed three different datasets, and on the bottom, four different models. Which model is used for which dataset? It seems as if the first row of the datasets & of the models is copied.
• Figure 5 doesn't specify which dataset is being used (the numbers seem to indicate it is CIFAR-10, by checking with Table 2, but please confirm this)

The theory in this paper is not rigorous. The main novel theoretical contributions of this paper are Lemma 1 and Lemma 2.

• As discussed above, neither lemmas are stated formally and so it is unclear what is being stated.
• There is no proof of Lemma 1. (In the appendix, there is a proof that is labeled "Lemma 1" but this actually refers to Lemma 2 from the main paper).
• The proof of Lemma 2 (labeled "Lemma 1" in the appendix) does not seem not rigorous: specifically the argument that $I(Z;G)>0$ does not seem to be supported by any formal proof (please see Questions section).

Experimental & Implementation details

• While it is impossible for me to verify, the claimed robustness of the AEP-AT trained models in Table 2 raise a red flag for me to the point where I believe they might indicate a possible bug in the code that generates adversarial attacks—for example, the authors are claiming that the adversarial accuracy. It is somewhat difficult for me to believe that the method proposed by the authors—adding a grid pattern to adversarial examples during training (as in Figure 4)—would increase adversarial performance by nearly 30 points. In order to verify that the models are actually as robust as the authors claim, the authors should implement adaptive attacks such as AutoAttack. If this is not a bug and holds up under other attack strategies, this is an impressive result. (Nonetheless, my claims about the writing above still stand.)

1. My main concern is that the existing basic theoretical foundations of the paper are relatively weak (both Theorems 1 and 2 are known results) and are not clearly explained. I'm trying to understand the theoretical results (especially the proof of Lemma 2), but the vague expression (including the definition of notations, the introduction of background knowledge, and the mathematical analysis and discussion of results) made it difficult for me to understand these results theoretically. See Questions for more details.

2. The paper is not well-written, many concepts and symbols are not defined, theorems and lemmas are not stated rigorously, and many notations are used inconsistently. I would suggest the authors go through the paper and polish the languages and statements.

I have the following concerns about this manuscript.

1. My major concern is that the theoretical analyses or explanations in this paper lack some quantitative results or conclusions. From my point of view, normal training (i..e, training on clean inputs) can be considered as a special form of adversarial training when the size of the adversarial budget is $0$. Therefore, any theoretical conclusions concerning the properties of the adversarial perturbations should depend on the size of the adversarial budget. In the formulation of this paper, I can say that the ``adversarial perturbation'' can also be typical when the maximum allowed magnitude is very small. However, the author's analyses and theoretical results do not reflect this.

2. The motivation of AEP-AT is not very clear, the authors need more words to elaborate this part. In addition, the authors claim optimal trade-offs of AEP-AT in line 443, a ROC curve is expected. The advantages of AEP-AT are unclear either.

3. AutoAttack is the strongest adversarial perturbation so far, when evaluating the robustness and reporting robust accuracy in the table, so it is better to evaluate robustness by AutoAttack instead of $10$-step PGD.

4. More empirical simulations are expected for Section 4.3 and 4.4, since the conclusions are already included in existing works.

5. The observations and the conclusions in this paper are already known in this community, making it different to point out the contributions of this paper.