Adversarially Robust Anomaly Detection through Spurious Negative Pair Mitigation

Q3&W3:

The question raised concerns a typographical error in our manuscript, where we mistakenly referred to A³ (Adaptive Auto Attack) as A² (Auto Attack) in the column title. We sincerely apologize for this oversight. This error has been corrected in the revised version of the manuscript, and the updated results, reflecting this correction, are as follows:

Nevertheless, the question remains as to why PGD-1000 outperforms AutoAttack in our study. To address this, we kindly refer the reviewer to Appendix K of our manuscript, where we provide a detailed explanation of the adjustments made to AutoAttack for the anomaly detection task. Specifically, AutoAttack is an ensemble of six attack methods:

1. APGD with Cross-Entropy loss,
2. APGD with Difference of Logits Ratio (DLR) loss,
3. APGDT,
4. FAB \cite{croce2020minimally},
5. Multi-targeted FAB \cite{croce2020reliable}, and
6. Square Attack \cite{andriushchenko2019square}.

Among these, the DLR loss-based attacks assume that the target model is a classifier trained on more than two classes. In the context of anomaly detection, the problem is closer to binary classification, as we are only distinguishing between normal and anomaly classes. Therefore, we excluded DLR-based attacks in our adaptation of AutoAttack and instead utilized an ensemble of the remaining attacks. We believe this modification is the main reason for the observed performance difference between AutoAttack and PGD-1000 in our study.

Dear Reviewer mZoT,

Thank you for your valuable comments. Here are our responses:

Q1 & W1:

Analysis of Training Data Size on Performance:

Our experiments encompass various configurations, and the appendix provides comprehensive details about the datasets and settings used (please see Appendix L for complete dataset details). The information relevant to this concern is already available in the manuscript, but to address the reviewer’s specific concern, we have provided additional empirical details here.

As the table below indicates, there is no clear correlation between the training dataset size and the performance of our method. This demonstrates the robustness of our approach, as it performs consistently across datasets with varying sizes and resolutions.

Inference Time:

After training, we freeze our model and compute the features of the training samples to create a feature bank. This bank maps images to a low-dimensional embedding space (e.g., reducing dimensions from $(1000, 3, 224, 224)$ to $(1000, 256)$. This process is performed once and does not need to be repeated.

During inference, we compute the features of each test sample using the frozen model and compare its similarity to the precomputed feature bank. This is computationally efficient because the operations are performed by the frozen model in a low-dimensional space. Leveraging similarity in an embedding space for detection is a well-established and widely used approach in the literature, known for its low computational and resource requirements. Additionally, we provide detailed information about the computational times in our experiments, which are further elaborated in Appendix Section I of the paper.

The average computational times for our model are as follows:

• Embedding computation: 1--3 ms
• Matrix multiplication: 1--2 ms
• Finding the maximum value: $<$1 ms

We also provide the evaluation times for competing methods for a fair comparison.

The table results are averaged over 100,000 inferences.

Q2 & W2:

All adversarial attacks against anomaly detection models in our study are targeted and adaptive, designed specifically to degrade performance. Detailed explanations can be found in the Preliminaries section and Appendix K (please kindly see lines 108–119 and 1350–1364).

Specifically, anomaly detection methods rely on an indicator referred to as the anomaly score, which is used to distinguish normal test samples from anomalous ones by assigning likelihoods. Ideally, an effective anomaly detector produces non-overlapping score distributions for normal and anomalous samples—assigning higher anomaly scores to anomalous samples and lower scores to normal ones.

In our evaluation, we employed targeted adversarial attacks that directly manipulate the anomaly score. These attacks operate end-to-end and are specifically designed to force misclassifications by altering the score distribution. Depending on the test sample's label (normal or anomaly), the attacks aim to either maximize or minimize the anomaly score:

• For anomaly test samples: The attack minimizes the anomaly score, causing the detector to incorrectly classify anomaly samples as normal (false negatives).

• For normal test samples: The attack maximizes the anomaly score, causing the detector to incorrectly classify normal samples as anomalous (false positives).

By targeting the anomaly score directly, these attacks effectively exploit the decision boundary between normal and anomalous predictions, inducing the most detrimental types of misclassifications for the detection system. This approach ensures that the attacks are both highly specific and impactful in the context of anomaly detection.

Dear Reviewer mZoT,

Thank you for your valuable review and positive feedback! We are pleased to hear that your concerns have been addressed.

Sincerely, The Authors

Dear Reviewer qBmY,

Thanks for your constructive and valuable comments on our paper. Below, we provide a detailed response to your questions and comments. If any of our responses fail to address your concerns sufficiently, please inform us, and we will promptly follow up.

W1&Q1:

In Table 6 of our paper , we present an ablation study comparing different loss functions, including the standard Contrastive Loss (CL) and our proposed loss. In this experiment, all components remain the same except for the objective function. The primary distinction between the standard CL and our proposed objective lies in mitigating spurious negative pairs by assigning higher weights to crafted opposite pairs.

The results clearly demonstrate that the inclusion of spurious negative pairs in standard CL weakens the model's robustness against adversarial attacks. By emphasizing opposite pairs and as a result reducing the impact of spurious negatives, COBRA enhances inter-group perturbations, leading to improved robustness.

Moreover, in Figure 3 of our paper, we provide t-SNE visualizations of the learned feature embeddings for both the standard CL and COBRA-trained models:

Standard CL Model: The embeddings of normal and anomaly samples are not well-separated, showing significant overlap between the two groups. This indicates that the model struggles to effectively distinguish normal samples from anomalies under adversarial perturbations.

COBRA Model: The embeddings exhibit a clear separation between normal and anomaly samples. The margin between the two groups is significantly larger, demonstrating enhanced inter-group discrimination and better robustness.

These visualizations qualitatively support our claim that spurious negative pairs in standard CL hinder the model's ability to enforce a strong margin between normal and anomaly distributions.

These visualizations qualitatively support our claim that spurious negative pairs in standard CL hinder the model's ability to enforce a strong margin between normal and anomaly distributions.

Additionally, previous studies, such as [1], have also addressed similar challenges in CL by redefining the pairing mechanism. These works underscore the limitations of standard CL in specific scenarios.

Note that, our study is distinct from [1] in its focus and application. The key difference between COBRA and them lies in the nature of the tasks they address. COBRA is designed for an open-set task, which involves detecting anomalies relative to a normal group. In contrast, FNC is tailored to a closed-set task, clustering the normal group (training set) into various semantic subgroups.

[1] https://openaccess.thecvf.com/content/WACV2022/papers/Huynh_Boosting_Contrastive_Self-Supervised_Learning_With_False_Negative_Cancellation_WACV_2022_paper.pdf

W2&Q2:

We appreciate the reviewer’s valuable suggestion and understand the concern regarding the use of a single architecture in our experiments. To address this, we conducted additional experiments by replacing ResNet-18 with other widely used architectures while keeping all other components of the methodology fixed. Below, we present the results of these experiments across various datasets.

For each dataset, we report results in the “Clean / PGD-1000” format, reflecting performance under both clean and adversarial settings.

While the results show slight variations across architectures, the overall trends confirm that COBRA generalizes well to other feature extractors.

W7 & Q6-(2):

Based on the provided description, we interpret the reviewer's question as an inquiry into the impact of adversarial training and pseudo-anomalies in our proposed method. This interpretation is grounded in the fact that the hard transformations in our pipeline are specifically designed to craft pseudo-anomaly samples.

To address this, we have conducted ablation studies to analyze the individual contributions of these components. These experiments demonstrate the effect of each component on the robustness of the proposed approach. Additionally, we note that these results are also presented in the manuscript, but we provide further discussion here to address the reviewer's concerns in more detail. It is important to emphasize that adversarial training is a critical component of our pipeline, and its combination with pseudo-anomalies and the proposed loss function achieves strong, robust performance. Removing adversarial training significantly compromises the model's ability to safeguard against anomalies.

The table below summarizes the experimental setups and results across multiple datasets:

(Clean/PGD-1000)

We also highlight that our proposed loss function, $L_{\text{COBRA}}$, requires pseudo-anomalies for training. In the absence of pseudo-anomalies, as in Setups A and B, we substitute $L_{\text{COBRA}}$ with a contrastive loss ($L_{\text{CL}}$) to enable training. This substitution is necessary to facilitate the provided experiments and analyze the role of pseudo-anomalies.

W7 & Q6-(1):

In response to the reviewer’s question regarding the role of transformations and adversarial training, we will first review our method and then directly address the concerns raised. To clarify, the use of hard transformations (along with threshold-based filtering) in our pipeline is specifically designed to create pseudo-anomalies, which serves a purpose distinct from that of mild augmentations in our approach. Mild augmentations, commonly used in self-supervised learning, are a standard practice in CL domain, that we also leverage during training with $L_{COBRA}$.

---

Motivation for designing robust anomaly detection

Developing robust anomaly detection methods is crucial due to their application in safety-critical real-world problems such as autonomous driving. While clean results on challenging benchmarks (e.g. MVTecAD) achieve near 100% AUROC, the best performance of previous works under attacks is less than random detection even on tiny datasets. Motivated by this, we propose COBRA, an adversarially robust model that improves robust detection by 26.1%. COBRA is based on three components: using pseudo-anomaly samples, adversarial training, and introducing a novel loss function, which we explain in more detail below.

Motivation behind COBRA

We hypothesized that for effective robust anomaly detection, the model must be exposed to both anomalies and normal samples during training. Otherwise, the detector can be easily fooled by perturbations on anomaly test samples during inference. This hypothesis aligns with observations from related works [1,2,3].

However, in anomaly detection setups, abnormals are unavailable [16,17,18]. Moreover, collecting extra abnormal datasets presents challenges, such as the need to process and remove normals to avoid providing misleading information for the detector. Additionally, extra data may bias the detector model, and obtaining relevant datasets, especially for medical imaging, is costly. As a result, we propose a novel approach for crafting pseudo-anomaly samples. We use hard transformations such as rotation and cut-paste on normals, which have shown to shift normals to an anomaly distribution [9,10,14]. To ensure transformed samples do not still belong to the normal distribution based on the characteristics of the dataset, we develop a threshold mechanism. Adversarial training and its variants have been shown to be the most effective defense for DNNs. Thus, we chose adversarial training as a core component of our method, specifically utilizing the common adversarial training approach, PGD-10.

We also explored the impact of different loss functions on the robustness of anomaly detection. A robust detector should achieve separated embeddings for normal and anomaly groups. Contrastive Learning (CL) has been shown to be more effective than classification (CLS) loss for anomaly detection [14]. CL operates by attracting positive pairs and repelling negative pairs. However, using CL with adversarial training fails to achieve robust detection because negative pairs include normal-normal and anomaly-anomaly pairs, referred to as spurious negative pairs. These weaken inter-group perturbations. To address this, we propose a new loss function that explicitly increases the distance between normal and anomaly distributions by repelling opposite pairs which are confirmed to be inter-group pairs. It is important to highlight that the limitations of CL in anomaly detection are apparent in adversarial scenarios. This stems from the fact that adversarial training requires a high degree of data complexity compared to standard settings, necessitating a broad range of strong perturbations to achieve robust anomaly detection [19, 20].

[1] Chen, Robust OOD, 2020

[2] Azizmalayeri, Your, 2020

[3] Chen, Atom, 2021

[4] Kalantidis et al., Hard, 2020

[5] Li Cutpaste, 2021

[6] Sinha Negative Data, 2021

[7] Miyai Rethinking Rotation, 2023

[8] Zhang Improving, 2024

[9] Chen Novelty, 2021

[10] DeVries Improved, 2017

[11] Yun Cutmix, 2019

[12] Akbiyik Data, 2019

[13] Ghiasi Copy-Paste 2020

[14] Tack CSI, 2020

[15] Cohen Transformaly, 2022

[16] Yang Generalized, 2022

[17] Salehi A Unified, 2022

[18] Perera One-Class, 2021

[19] Schmidt Adversarially 2018

[20] Stutz Disentangling 2019

[21] Golan GT 2018

[22]Hendrycks Using 2019

Q5&W6:

We first review our hard augmentation (aug) strategy: We define a set of hard transforms, $T = T_i$, with each $T_i$ representing a specific type of hard aug. For each hard transformation process, which aims to shift an inlier sample to an outlier, a random subset of $k$ members from $T$ is selected, $T_{j_1}, T_{j_2}, \dots, T_{j_k}$, and sequentially applied, resulting in $T_{j_k}(\dots(T_{j_2}(T_{j_1}(x))))$. We avoid using $k = 1$, which would apply only a single hard transformation because in some cases, it does not significantly alter the semantics. For instance, applying rotation to a Car image yields an OOD sample as it is rare in natural images. However, some semantics are rotation invariant, such as `Flower images. Therefore, we have used $k > 1$. This ensures that the output of this process is sufficiently shifted from the in-distribution.

For creating the set of hard augs., we include Jigsaw, Random Erasing, CutPaste, Rotation, Extreme Blurring, Intense Random Cropping, Noise Injection, Extreme Cropping, Mixup, Cutout, CutMix, and Elastic Transform. These transformations have been extensively investigated in various areas of the literature (e.g., self-supervised learning) and have been shown to be harmful for preserving semantics, often resulting in a significant shift from the original transformation. Two criteria were used for selection of such augs as hard transformations:

• Several studies in OOD detection explore the effect of using augs for crafting outlier samples to provide info about OOD for detector models. After reviewing such works, which have provided independent studies on each hard transformation, we provided a unified view of such transformations and created a set of them instead of just relying on one [1,2,3,4,5,6].

• Moreover, there have been several studies in self-supervised learning research exploring suitable augs for crafting positive pairs from each image and using them in contrastive loss (e.g., InfoNCE loss). Specifically, they choose soft augs that preserve semantics (e.g., mild color jitter). Another rationale behind creating such a set of hard transformations is that we choose transformations that the literature shows are harmful for preserving semantics and avoided using them (e.g., rotation) [7,8,9].

• For mild/soft augmentations in training step, we follow common practices in the self-supervised learning literature (e.g., color jitter) and refrain from making any modifications.

[1] Tack et al. CSI 2020

[2] Kalantidis et al.Hard negative 2020

[3] Li et al.Cutpaste 2021

[4] Sinha et al.Negative data 2021

[5] Miyai et al.Rethinking 2023

[6] Zhang et al.Improving the 2024

[7] Chen et al.SimCLR 2020

[8] Grill et al.BYOL

[9] He et al.MOCO

---

Q7&W8:

We sincerely apologize for the issue, which appears to have resulted from a typographical error in our manuscript. Some equations in the definition were inadvertently omitted, and we have now revised the manuscript to address this oversight (please see line 285). Specifically, after proposing our loss function, we intended to provide an intuition for the objective behind our approach.

We initially claimed that in the case of adv. training with $L_{CL}$, the objective is to ensure that the embeddings of both the normal and pseudo-normal groups become compact. This is achieved by weighting positive pairs to bring them closer together.

Conversely, adv. training with $L_{\text{Opposit}}$ was defined to offer better insight into the mechanics of $L_{\text{COBRA}}$ rather than to serve as a standalone recommended loss function. Specifically, $L_{\text{Opposit}}$ explicitly pushes the distributions of the normal and anomaly groups further apart.

Intuitively, adv. training with $L_{\text{COBRA}}$ integrates these two objectives, producing embeddings that are compact within each group while maximizing the separation between the embeddings of the two groups. The ablation study presented in Table 6 demonstrates that neither $L_{CL}$ nor $L_{\text{Opposit}}$ alone achieves the performance of $L_{\text{COBRA}}$. This underscores the importance of combining both properties: compact representations for the normal and anomaly groups and maximizing their separation.

$L_{\text{COBRA}}= \sum \sum \log \frac{\exp(\text{sim}(f(x_i), f(x_j))/t) - \exp(\text{sim}(f(x_i), f(\Upsilon(x)))/t)}{\sum\limits_{x_k \in \text{P}(x_i) \cup \text{N}(x)} \exp(\text{sim}(f(x_i), f(x_k))/t)},$

$L_{\text{Opposite}}= \sum \sum \log \frac{ - \exp(\text{sim}(f(x_i), f(\Upsilon(x)))/t)}{\sum\limits_{x_k \in \text{P}(x_i) \cup \text{N}(x)} \exp(\text{sim}(f(x_i), f(x_k))/t)},$

$L_{\text{CL}}= \sum \sum \log \frac{\exp(\text{sim}(f(x_i), f(x_j))/t) }{\sum\limits_{x_k \in \text{P}(x_i) \cup \text{N}(x)} \exp(\text{sim}(f(x_i), f(x_k))/t)},$

$L_{\text{COBRA}} \simeq L_{\text{CL}}+ L_{\text{Opposite}}.$

W4&Q4:

We found that it is common in the literature to fix the training $\epsilon$ and evaluate robustness across a range of $\epsilon$ values during testing (refer to Table 14), as we have already done in the manuscript (please see Table 14).

To further address the reviewer's concern, we conducted additional experiments where the training $\epsilon$ was varied to assess its impact on robustness while keeping the test-time $\epsilon$ fixed.

In this experiment, the model was trained using the default epsilon ($\epsilon$) values as specified in the manuscript: $\epsilon = \frac{4}{255}$ for low-resolution images and $\epsilon = \frac{2}{255}$ for high-resolution images. During inference, however, we varied the epsilon for adversarial testing from $0$ (clean images) up to $\frac{8}{255}$. The following table summarizes the model's performance on various datasets under different levels of adversarial perturbations:

To address the reviewer's concern, we conducted new experiments where we varied the epsilon ($\epsilon$) value used for adversarial training, while keeping the evaluation configuration fixed (i.e., $\epsilon = \frac{4}{255}$ for low-resolution images and $\epsilon = \frac{2}{255}$ for high-resolution images). We experimented with different $\epsilon$ values ranging from $0$ (standard training without adversarial examples) to $\frac{8}{255}$. For each $\epsilon$ value used during training, we evaluated the model on both clean images and adversarial images generated with the fixed evaluation $\epsilon$. The results are summarized in the following table, where each cell shows "clean / PGD-1000":

W5:

We acknowledge the reviewer's concern regarding the trade-off between clean and robust performance, and we will address this more thoroughly in our manuscript.

However, we would like to emphasize certain points about comparing ZARND and our method, which we have also discussed in the manuscript (please see Line 890).

ZARND utilizes an adversarially pretrained model on a large dataset (i.e., ImageNet), whereas our approach achieves superior performance without relying on any additional datasets or pretrained models.

While we agree that on smaller datasets, such as CIFAR-10, the differences in performance between our method and ZARND may appear minor, our primary focus in this study is on real-world and industrial datasets, such as MVTecAD. On such datasets, our method demonstrates over a 45% improvement compared to ZARND. It is important to note that ZARND's performance on natural datasets benefits from its pretrained weights on ImageNet but falls short on more challenging benchmark datasets, such as MVTecAD, as discussed in our manuscript.

W3&Q3:

To apply a filter to the crafted pseudo-anomalies, a knowledgeable model is required. Notably, in our problem setting, no pre-trained models or additional data are available. The training set is limited to a single semantic class (e.g., car images) without any supplementary information. This constraint motivates the need to extract meaningful representations from normal data using a self-supervised approach.

For instance, one potential strategy is to train an autoencoder on the normal data with a reconstruction loss and use its embeddings. However, representative studies [1,2] suggest that training a k-class classifier to predict transformations is a simple, more effective, and promising approach for representation learning in one-class classification compared to alternative methods.

We adopt this approach, leveraging the learned embeddings from the classifier to compute the likelihood of input test samples and define a threshold for false anomaly data filtering.

The primary advantage of our approach is that, even without utilizing additional data or external models, we propose a method to define a threshold for crafting data that confidently does not belong to the inlier set, thereby enabling the generation of effective pseudo-anomalies. It is worth noting that many existing methods [3-9] aim to craft anomalies for various tasks but often rely on extremely large generators (e.g., Stable Diffusion) or massive datasets (e.g., LAION-5B). Despite their complexity, these methods underperform compared to our proposed simple and efficient strategy for anomaly crafting. For further details, please refer to Figure 2 and Table 5.

[1] Deep Anomaly Detection Using Geometric Transformations Izhak Golan, Ran El-Yaniv Neurips

[2] Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, Dawn Song ICLR

[3] Lee et al, Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples, ICLR 2018.

[4] Kirchheim et al, On outlier exposure with generative models, NeurIPS ML Safety Workshop, 2022

[5] Du et al, Learning what you don’t know by virtual outlier synthesis. ICLR

[6] Tao et al. Non-parametric outlier synthesis. ICLR 2023

[7] Du et al, Dream the Impossible: Outlier Imagination with Diffusion Models, Neurips 2023

[8]Chen et al, ATOM: Robustifying Out-of-distribution Detection Using Outlier Mining

[9] RODEO: Robust Outlier Detection via Exposing Adaptive Out-of-Distribution Samples ICML 2024

Dear Reviewer qBmY,

thank you again for your review. We wanted to check in to see if there are any further clarifications we can provide. We hope that our updated PDF, including new experiments and explanations, effectively addresses your concerns.

Best, the authors

Dear qBmY,

We apologize for reaching out again.

While we are pleased to know that your concerns have been addressed, the assigned score suggests there may still be unresolved issues. We would like to engage further to address these concerns.

we hope the new experiments and revisions to the manuscript allow you to reevaluate your score.

Thank you!

Dear Reviewer 1DsR,

We appreciate your constructive feedback on our manuscript. Below, we provide a detailed response to your questions and comments.

W1:

Thank you for your feedback. We have carefully addressed all your suggestions and would greatly appreciate it if you could kindly see our revised manuscript.

W2&Q1:

In Figure 1, an augmented sample and its corresponding opposite sample (e.g., $A_1$ and $A_1^\Upsilon$) are used in $\mathcal{L}_{\text{COBRA}}$. However, in equation (2), the opposite sample of the original sample (e.g., $A^\Upsilon$) is used instead.

Thank you for noting this point. The $A^\Upsilon$ in equation (2) could also be replaced by $A_1^\Upsilon$, as both $A$ and $A_1$ are positive pairs and share similar semantics. Applying hard transformations to these samples results in closely related pseudo-anomalous samples, a behavior we have also observed experimentally. To further clarify this point, we explicitly mention it in line 286.

In Figure 1 and the "Adversarial Training Step" paragraph on page 6, a single adversarial example $x_{\text{adv}}$ is generated for the augmented samples $x_i$ and $x_j$. However, the generation process is unclear and needs a step-by-step explanation, especially on how the perturbation is optimized in $\mathcal{L}_{\text{COBRA}}$.&If possible, include pseudocode for generating the adversarial example.

Thank you for pointing this out. To clarify, the perturbation $x_{\text{adv}}$ is computed with respect to the entire batch $X$, rather than specifically the augmented samples $x_i$ or $x_j$. This is achieved using the PGD-10 algorithm and the defined objective function. We will work on improving Figure 1 and adding further clarification to better illustrate this point. Additionally, we would like to highlight that we already provide an extensive pseudocode on page 16, which includes the computation of adversarial perturbations. Nevertheless, we will make additional efforts to enhance the clarity of this explanation.

W3&Q2:

We thank the reviewer for their insightful suggestions. In response, we have revised the structure of the EXPERIMENTS section to enhance clarity and coherence. Additionally, we will provide a more detailed analysis of Figure 3 and the experimental results in our manuscript.

Figure 3 presents visualizations of features extracted by the encoder, trained on CIFAR-10 in a one-class setup with 'Automobile' as the normal class. The visualizations compare representations learned under various loss functions: Contrastive Loss (CL), Classification Loss (CLS), Supervised Contrastive Loss (SupCL), and the proposed COBRA method, while keeping other components constant.

The COBRA method demonstrates superior performance, achieving compact clustering for normal samples and a significant separation from pseudo-anomalies, even under adversarial perturbations. In comparison, alternative loss functions such as CL, SupCL, and CLS fall short in these aspects.

Q3: To address the reviewer’s concern, we present the results for both scenarios : with and without adversarial training. Notably, replacing adversarial training with standard training leads to improved clean performance, albeit at the cost of reduced robustness. This demonstrates that, in environments where reliability is guaranteed and adversarial agents are absent, our method provides the flexibility to prioritize clean performance over robustness. This adaptability is a key feature of the COBRA framework, enabling it to be tailored to the specific requirements of the deployment environment.

(Clean/PGD-1000)

Q4:

For adversarial training, as noted in the manuscript, we use PGD with 10 steps. However, for adversarial testing, we employ PGD-1000 to ensure the attack is sufficiently strong. To address the reviewer’s concern, we have conducted additional experiments where all other components are fixed, and the number of PGD steps during testing is varied. The results of these experiments are provided below.

As the results indicate, reducing the number of PGD steps during testing leads to a minor weakness in the adversarial evaluation, reflecting slightly less effective attacks. These findings validate our choice of using PGD-1000 for robust evaluation during testing to ensure rigor.

Dear Reviewer 1DsR,

We sincerely appreciate the thoughtful feedback you’ve offered on our manuscript. We have carefully reviewed each of your comments and have made efforts to address them thoroughly. We kindly ask you to review our responses and share any additional thoughts you may have on the paper or our rebuttal. We would be more than happy to accept all your criticisms and incorporate them into the paper.

Sincerely, The Authors

Thank you for your feedback. We're glad your comments have been addressed!

Sincerely, The Authors

Q2:

We appreciate your thoughtful suggestion regarding expanding the set of positives for normal samples to include all other normals in the batch. To address the reviewer's concern, we conducted additional experiments where we fixed all other components of our method but replaced our loss function with Contrastive Loss (CL). In this setup, we treated all normal samples within a batch as positive pairs during training. The results of these experiments are provided below.

However, this approach resulted in a significant decline in performance.

The primary reason for this performance degradation is the occurrence of mode collapse and over-clustering within the embedding space for normal samples. Specifically:

Mode Collapse: Treating all normal samples as positives forced the model to align dissimilar normal samples and map them to a similar embedding space, disregarding their inherent variability. For example, in the CIFAR-10 vs. CIFAR-100 setup (where CIFAR-10 represents the normal set), this alignment reduced the diversity of embeddings within the normal class, which in turn impaired the model’s ability to generalize to unseen normal and anomalous samples.

Over-Clustering: Compressing all normal samples into an overly tight cluster reduced the margin between the normal and anomaly classes. This diminished the model's ability to distinguish between normal and pseudo-anomalous samples, particularly for near-boundary anomalies.

These issues negatively impacted both clean and adversarial robustness.

We believe these findings highlight the critical importance of balancing compactness within the normal class with the preservation of inter-class margins.

Q3:

Pseudo-anomaly samples, especially those located near the normal distribution (near-anomalies), provide valuable information for anomaly detection. Many methods focus on crafting such anomalies. Our approach stands out because, without relying on additional data or external models, we propose a threshold-based method for crafting data that confidently lies outside the inlier set. This enables the effective generation of pseudo-anomalies. While existing methods [1-7] often rely on large generators (e.g., Stable Diffusion) or extensive datasets (e.g., LAION-5B), they lack thresholding or filtering strategies. This increases the risk of including false anomalies—normal samples mistakenly treated as anomalies. Despite their complexity, these methods underperform compared to our simple and efficient strategy, as shown in Figure 2 and Table 5.

Key Contributions of Our Approach:

1. Crafting Pseudo-Anomalies from Normal Samples:Instead of utilizing external anomaly datasets, we generate anomaly samples through hard augmentations of normal data. This avoids challenges associated with incorporating real anomaly data, such as ensuring it excludes normal concepts that could mislead the detector.

2. Auxiliary Task for Effective Representation:In the absence of labels, we employ an auxiliary task for learning meaningful embeddings, using transformation prediction as the encoder.

Our method demonstrates superior performance with a simpler and more efficient approach, addressing the limitations of existing anomaly-crafting techniques in domains like medical imaging, where gathering real near-anomaly samples is costly and time-consuming.

3. Threshold-Based Filtering for False Anomalies:While hard augmentations can shift normal samples toward anomaly-like regions, they may still produce falsely crafted anomalies. To mitigate this, we use a Gaussian Mixture Model (GMM) to estimate likelihoods based on the embeddings of training samples. These embeddings are extracted using the transformation predictor encoder. A likelihood-based threshold, such as the bottom 5% of the distribution, is then applied to filter out samples with a high likelihood of being normal.

On page 1 (line 051), the authors refer to a “novel thresholding approach (19).” It’s unclear why this is considered novel, especially since the cited work is from 2013.

We cited the GMM paper, which presents a general and widely used classic method for modeling low-dimensional embedding spaces. It is important to note that GMM is not a filtering-based method.

---

[1] Lee et al, TraininClass , ICLR 2018.

[2] Kirchheim et al, On outlier with, 2022

[3] Du et al, Learning virtual outlier synthesis. ICLR

[4] Tao et al. Non-parametric outlier synthesis. ICLR 2023

[5] Du et al, Dream the: Diffusion Models, Neurips 2023

[6]Chen et al, ATOM: Robustifying Mining

[7] RODEO: Robust Outlier via ICML 2024

Q5:

Maximizing the margin between normal and anomaly distributions creates a larger separation in the feature space, making it harder for adversarial perturbations to cause misclassification. Adversaries would require stronger perturbations to shift samples across the margin, which often exceeds allowable perturbation limits, thus thwarting attacks. Compactness of normal samples further enhances robustness by reducing intra-class variability, ensuring that small perturbations are unlikely to push normal samples into the anomaly space. Together, these properties strengthen the model’s decision boundaries and reduce overlap between classes, minimizing both false positives and negatives under adversarial conditions.

This separation improves generalization by enabling the model to learn distinct and discriminative features, making it resilient to unseen samples and adversarial manipulations. Additionally, robust feature learning ensures the model focuses on essential characteristics that differentiate normal and anomaly samples, further bolstering resistance to attacks.

In our method, this principle is implemented through the construction of opposite pairs between normal samples and their pseudo-anomalies. Our COBRA loss function explicitly pulls positive pairs together to ensure compactness while pushing apart opposite pairs to maximize the margin. Adversarial training then reinforces this separation by generating perturbations that directly target the inter-group boundary, enhancing robustness against attacks near the decision threshold.

Notably, the ablation studies (Table 6) and the visualization (Figure 3) provide supportive evidence to these claims

Q6:

We will work on refining the manuscript to ensure these concepts are presented more clearly before the final submission. For further clarification, we have provided a brief explanation here:

In our setup where only normal samples are available during training, pseudo-anomalies are generated by applying hard transformations (e.g., blurring, intense cropping, random erasing transformations) to normal samples. For a normal sample $x$, the transformed version $x'$ is created as:

$x' = T_m(\ldots T_2(T_1(x)))$

where $T_i \in T$ is a transformation, and $m$ is the number of transformations applied.

To ensure $x'$ sufficiently deviates from normal samples, a thresholding mechanism is applied:

• Feature Extraction: Embeddings are obtained using a pre-trained model $C$.
• Likelihood Estimation: A statistical model (e.g., Gaussian Mixture Model) fits the embeddings of normal samples.
• Threshold ($\lambda$): Based on a significance level, samples with a likelihood below $\lambda$ are accepted as pseudo-anomalies.

An opposite pair consists of a normal sample and its corresponding pseudo-anomaly:

• Opposite Pair $(x, x')$: The normal sample $x$ and the transformed sample $x'$ that is accepted as a pseudo-anomaly.

The purpose of creating negative pairs is to mitigate the inclusion of opposite pairs and maximize the margin between normal and anomaly samples, ensuring greater separation in the feature space for robust anomaly detection.

In the contrastive learning framework, for any given normal sample, positives and negatives are defined as follows:

Positive Pairs

• Definition: Pairs of augmented versions of the same sample from the same group (either both normal or both pseudo-anomalies).
• Construction:
  • Apply light augmentations $\tau_1$ and $\tau_2$ to the normal sample $x$ to obtain $x_1$ and $x_2$.
  • Positive Pair: $(x_1, x_2)$.
• Purpose: Encourage the model to learn invariant features within the same class, promoting compactness.

Negative Pairs

• Standard Approach: Traditionally, negatives are all other samples in the batch, which can include samples from the same class, leading to spurious negative pairs (e.g., normal-normal pairs).
• Our Approach:
  • Opposite Pairs as Negatives: Specifically use opposite pairs $(x, x')$ as negative pairs.
  • Definition: Negative pairs are formed between a normal sample and its corresponding pseudo-anomaly.
• Purpose: Focus on maximizing the margin between normal and anomaly samples without being misled by spurious negatives.

Q7:

Thank you for your observation. The loss formulation indeed aligns with NT-Xent as introduced in SimCLR. We have revised the manuscript and updated the terminology and references accordingly to reflect this.

Q8&W3&W4:

Thank you for the feedback. We will revise the manuscript to align with the proposed suggestions and enhance the clarity of the definitions. We appreciate your input and will apply it to improve the manuscript.

Q1:

We should highlight that our study includes a comprehensive ablation study that examines the effects of each component of our method. This includes the use of extra anomalies (Table5), the loss function (Table6), and the impact of adversarial training (Table 9). To address the reviewer's concerns, we have provided those experiments here.

(Clean/PGD-1000)

Dear Reviewer YXiZ,

Thank you for your constructive and valuable comments on our paper. First, we provide a brief overview of the motivation behind our method, followed by detailed responses to each of your comments.

Motivation for designing robust anomaly detection

Developing robust anomaly detection methods is crucial due to their application in safety-critical real-world problems such as autonomous driving. While clean results on challenging benchmarks (e.g. MVTecAD) achieve near 100% AUROC, the best performance of previous works under attacks is less than random detection even on tiny datasets. Motivated by this, we propose COBRA, an adversarially robust model that improves robust detection by 26.1%. COBRA is based on three components: using pseudo-anomaly samples, adversarial training, and introducing a novel loss function, which we explain in more detail below.

Motivation behind COBRA

We hypothesized that for effective robust anomaly detection, the model must be exposed to both anomalies and normal samples during training. Otherwise, the detector can be easily fooled by perturbations on anomaly test samples during inference. This hypothesis aligns with observations from related works [1,2,3].

However, in anomaly detection setups, abnormals are unavailable [16,17,18]. Moreover, collecting extra abnormal datasets presents challenges, such as the need to process and remove normals to avoid providing misleading information for the detector. Additionally, extra data may bias the detector model, and obtaining relevant datasets, especially for medical imaging, is costly. As a result, we propose a novel approach for crafting pseudo-anomaly samples. We use hard transformations such as rotation and cut-paste on normals, which have shown to shift normals to an anomaly distribution [9,10,14]. To ensure transformed samples do not still belong to the normal distribution based on the characteristics of the dataset, we develop a threshold mechanism. Adversarial training and its variants have been shown to be the most effective defense for DNNs. Thus, we chose adversarial training as a core component of our method, specifically utilizing the common adversarial training approach, PGD-10.

We also explored the impact of different loss functions on the robustness of anomaly detection. A robust detector should achieve separated embeddings for normal and anomaly groups. Contrastive Learning (CL) has been shown to be more effective than classification (CLS) loss for anomaly detection [14]. CL operates by attracting positive pairs and repelling negative pairs. However, using CL with adversarial training fails to achieve robust detection because negative pairs include normal-normal and anomaly-anomaly pairs, referred to as spurious negative pairs. These weaken inter-group perturbations. To address this, we propose a new loss function that explicitly increases the distance between normal and anomaly distributions by repelling opposite pairs which are confirmed to be inter-group pairs. It is important to highlight that the limitations of CL in anomaly detection are apparent in adversarial scenarios. This stems from the fact that adversarial training requires a high degree of data complexity compared to standard settings, necessitating a broad range of strong perturbations to achieve robust anomaly detection [19, 20].

[1] Chen, Robust OOD, 2020

[2] Azizmalayeri, Your, 2020

[3] Chen, Atom, 2021

[4] Kalantidis et al., Hard, 2020

[5] Li Cutpaste, 2021

[6] Sinha Negative Data, 2021

[7] Miyai Rethinking Rotation, 2023

[8] Zhang Improving, 2024

[9] Chen Novelty, 2021

[10] DeVries Improved, 2017

[11] Yun Cutmix, 2019

[12] Akbiyik Data, 2019

[13] Ghiasi Copy-Paste 2020

[14] Tack CSI, 2020

[15] Cohen Transformaly, 2022

[16] Yang Generalized, 2022

[17] Salehi A Unified, 2022

[18] Perera One-Class, 2021

[19] Schmidt Adversarially 2018

[20] Stutz Disentangling 2019

[21] Golan GT 2018

[22]Hendrycks Using 2019

---

W1:

While we adopt a single-pairing strategy for forming positives, similar to common contrastive learning (CL) frameworks, our method addresses the issue of class collision (spurious negative pairs) by introducing a new category of pairs. By emphasizing opposite pairs rather than uniformly treating all negatives, we ensure that the model focuses on maximizing the margin between normal and anomaly groups, thereby avoiding intra-class conflicts.

We believe that our proposed framework effectively addresses the mentioned challenges, resulting in a robust anomaly detection method. As demonstrated in our ablation study (please kindly see Table 6), $L_{COBRA}$ outperforms $L_{CL}$ in terms of performance, with all other components remaining identical except for the objective function.