Improving model robustness against noise with safe haven activations

1. The description of the proposed methods starts immediately in Section 2 after the introduction. It is recommended to add a background section to discuss the necessary background details and extensively discuss the related works with their respective challenges, which provide the necessary motivations for this paper.
2. It is recommended to discuss the proposed method in more detail through a detailed top-level algorithm describing all the operations involved.
3. While the experiments have been conducted on various benchmarks. The choice of the benchmarks is questionable. The experiments have been conducted on a combination of relatively old DNN models and adversarial attack methods, and the datasets used CIFAR-10 and SVHN represent a relatively simple classification task. It is recommended to test the proposed method on more challenging benchmarks.
4. While adversarial training is one of the most powerful adversarial defenses, other defenses have been proposed. It is recommended to compare the proposed method with other adversarial defenses.

W1. The presentation in Section 2 is unclear, particularly in the explanation of $d_Q$, which is the most critical component. Specifically, as shown in Eq. (5), it seems that $d_Q$ is treated as a function of $x$, with $\delta_x$ following a uniform distribution. However, in Eq. (6), $d_Q$ is presented as a function of $y$ without any associated distribution. Why does the expectation term $E_{\delta_x}$ disappear in Eq. (6)? Additionally, how are Eq. (5) and Eq. (6) equivalent? The proof does not seem to provide sufficient evidence to support this. Furthermore, I believe it should be $y = Wx + b$ rather than $y = WD + b$, as $y$ represents a value, not a distribution. Due to the issues mentioned above, I became lost starting from Eq. (6) and could only attempt to understand the method through Figures 1 and 2.

W2. I can infer the main point of the paper from Figures 1 and 2, and I believe there is no need to overcomplicate the description of the method. The authors should revise the manuscript to clearly explain the Q-safety, E-safety, and NC-safety values, along with their corresponding $d_Q$.

-The paper does not include evaluations against stronger, modern adversarial attacks like AutoAttack, which limits the validity of its robustness claims​.

-There is insufficient comparison with recent state-of-the-art defensive strategies, especially non-quantization-based methods, which weakens the context of the proposed method’s effectiveness​.

-Dependency on Simplified Attack Models: The focus on simpler attacks, such as FGSM and PGD, may not fully demonstrate the model’s robustness under more complex real-world adversarial scenarios​.

• The additional QH layers introduce complexity and may increase inference time and energy usage, which is counterproductive for edge AI applications where efficiency is crucial​.

• The experiments are conducted primarily on CIFAR-10 and SVHN, which may not fully represent robustness in more complex datasets or tasks. Broader evaluations could provide stronger evidence of the method's effectiveness​.

• The experiments are limited to relatively small datasets like CIFAR-10 and SVHN, which may not generalize to more complex, large-scale datasets like ImageNet. Testing on larger datasets could strengthen the claims of robustness and applicability in real-world scenarios​.

-The paper exhibits some inconsistencies in writing style and terminology, which may make it harder for readers to follow and fully understand the methodology and results. A more polished and consistent presentation could improve readability and comprehension.