Thank you for taking the time to review our paper and provide your valuable comments. We have conducted experiments and provided results to address your questions. We have also revised our paper with these additional results. Please find our response to your comments below:

W1&Q1:

We respectfully request the reviewer to consider our "Common Response" to this question.

W2&Q2:

We understand the reviewer's concerns regarding the fairness of comparisons due to the use of large pre-trained model. To address this, we would like to provide some key points that highlight our approach's novelty and justify the fairness of our comparisons:

• Motivation: Developing robust OOD detection methods is a crucial task due to their application in safety-critical real-world problems such as autonomous driving. While the clean results on challenging benchmarks are near 100% AUROC, the best performance of previous works under attacks is less than random detection even on tiny datasets.

• Robust Methods: Previous robust OOD detection methods mostly focused on a specific setup of outlier detection, such as multi-class outlier detection. For instance, ALOE, ATD, and ATOM are only applicable in the multi-class setup and cannot handle other outlier detection setups like one-class outlier detection. In contrast, our method can generalize across different detection setups and datasets, including medical imaging and industrial datasets, despite the limitations of previous works.

• RODEO vs. SOTA: Several studies have utilized pre-trained models to enhance outlier detection. These studies have employed models such as ViTs (trained on 21M samples), BERT (1B data), and CLIP (400M data)[1,2] in the outlier detection domain. It is worth noting that we utilized a smaller version of CLIP, trained on 65M data, which is less data-intensive than the previously mentioned models. Moreover, while previous works directly leveraged the rich features of CLIP to develop a detector, we have adopted a different approach by implicitly using CLIP's guidance.

Furthermore, we conducted an experiment to compare our method with EXOE[2], a recent SOTA method based on CLIP-400M. The results indicate that our method slightly outperforms EXOE in clean performance, albeit with a minor margin. On the other hand, the significant superiority of our robust detection method highlights our contribution.

W3&Q3:

OOD detection can be formulated as:

$$g_\lambda(\mathrm{x}) = \begin{cases} \text{ID} & \text{if } O(\mathrm{x}) \leq \lambda \\\\ \text{OOD} & \text{if } O(\mathrm{x}) > \lambda \end{cases}$$

where $O(x)$ is the detection score, and $\lambda$ is the threshold. In PGD attacks, as stated in the Appendix, instead of maximizing the loss value, we try to increase $O(x)$ if $x$ belongs to in-distribution samples and decrease it otherwise. The formulation of the attack would be:

x^{t+1} = x^t + \alpha\cdot \text{sgn}( \nabla_{ x}( y.O_{\boldsymbol{\theta}}(x^t)) )$$ Where $y=1$ for in-distribution samples and $y=-1$ for OOD samples. The same setting holds for AutoAttack. AutoAttack is an ensemble of six attack methods: APGD with CE loss, APGD with DLR loss, APGDT , FAB, multi-targeted FAB, and Square Attack. Due to the presumption in DLR loss-based attacks that the model's output size must be three or greater, we had to exclude these specific attacks in our adaptation of AutoAttack, resulting in a total of four attacks. Addressing your concerns regarding the strength of the PGD attack, we conducted additional experiments with a PGD attack with 1000 steps and 10 random restarts, with results indicating negligible performance reduction. To further validate our method's adversarial robustness, we employed the Adaptive Auto Attack(A3)[3], which has been shown to be a stronger attack than AutoAttack. These enhancements substantially strengthen our novelty detection method's robustness evaluation. ND: |Attack\Dataset|CIFAR10|CIFAR100|MNIST|FMNIST|SVHN|MVTecAD|Head-CT|BrainMRI|Tumor Detection|Covid19| |-|-|-|-|-|-|-|-|-|-|-| |Clean|87.4|79.6|99.4|95.6|75.2|61.5|87.3|76.3|89.0|79.6| |PGD-100|71.1|62.8|95.7|88.1|35.4|15.9|70.0|71.1|67.5|59.4| |PGD-1000|70.2|62.1|94.6|87.2|33.8|14.9|68.6|68.4|67.0|58.3| |AutoAttack|69.3|61.0|95.2|87.6|33.2|14.2|68.4|70.5|66.9|58.8| |A3|70.5|61.3|94.0|87.0|31.8|13.4|68.1|67.7|65.6|57.6| OOD Detection: |Dataset\Dataset|Clean|PGD-100|PGD-1000|AutoAttack|A3| |-|-|-|-|-|-| |CIFAR10|93.2|70.4|69.5|69.0|68.8| |CIFAR100|88.1|66.4|64.7|65.3|63.2| OSR: |Attack\Dataset|MNIST|FMNIST|CIFAR10|CIFAR100| |-|-|-|-|-| |Clean|97.2|88.8|79.6|48.5| |PGD-100|87.7|67.1|64.1|37.7| |PGD-1000|85.0|65.3|62.7|35.3| |AutoAttack|86.4|66.8|63.5|36.9| |A3|84.1|62.9|63.0|35.4| [1] S Esmaeilpour Zero AAAI 2023 [2] P Liznerski Exposing TMLR 2023 [3] Y Liu Practical CoRR 2022 If our responses address your points, we kindly ask for a re-evaluation of your first score to reflect the paper's enhancements.

We thank the reviewer for the positive review! Thank you for the thoughtful review and valuable comments. Responses to specific points are provided below:

The paper could benefit from a more thorough...

It should be noted that many of the advanced methods developed for robust detection tasks, including ATD, ALOE, and ATOM, are founded on a data-centric approach. These methods typically employ an outlier exposure (OE) technique, which incorporates an auxiliary random outlier set, like ImageNet, during the process of adversarial training. Our method has been benchmarked against these in Tables 2 and 3 of the paper, and for your convenience, which are provided here as well for your review:

Results on OSR setup:

Results on OOD detection setup:

The method involves several complex steps...

We have provided details on computational cost in Appendix 10.3. To address your concern, we have added further details about the trade-off between performance gain and the additional computational cost here and have updated the appendix accordingly.

Computational cost:

To mitigate the computational costs, one alternative is to employ the ImageNet dataset for OE in adversarial training. We have examined this approach in our ablation study. Below, we present the average performance of our detector across two scenarios: (1) using ImageNet as OE for adversarial training, and (2) generating OE with our proposed pipeline and then leveraging it for adversarial training:

The paper could provide...

In response to your comment, we have conducted further experiments and included the results here and in the appendix. This includes scenarios where our detector was trained in a non-adversarial setup, with evaluation under both clean and adversarial settings

ND-setup:

OSR-setup:

OOD-setup:

The paper assumes that ...

We address these considerations in our ablation study, which examines scenarios where auxiliary outlier datasets are either distant from the inlier distribution or lack diversity. The results, extracted from Table 3, help clarify these situations.

Regarding Near-Distribution, the results show that both Adaptive OE and Vanilla OE (ImageNet) enhance clean and robust detection for the CIFAR10 dataset. However, for MNIST, Vanilla OE (ImageNet) is much less effective compared to our Adaptive method. This difference is likely due to the closer distribution between ImageNet and CIFAR10, in contrast to the larger gap between ImageNet and MNIST distributions. Regarding diversity, our results suggest that using Gaussian noise as OE is less effective due to its lower diversity compared to other techniques we evaluated. With respect to FDC metric, which serves to assess the diversity and how close is the OE dataset to in-distribution, it is evident that Adaptive OE outperforms the other two methods in this respect. The Ablation Studies section contains tables that compare the AUROC and FDC metrics of our approach with all the OE methods evaluated in our research, and the respective values have been presented here as well:

The approach is only ...

Your suggestion indeed underscores an important avenue for future research in applying our approach to diverse modalities.

We deeply appreciate your insights, which have been instrumental in elevating our work; we humbly believe these enhancements might resonate in an uplifted assessment.

We appreciate your insightful review and sincerely apologize for the typos. These issues have been addressed in the revised version, and we will enhance the clarity of our paper. Here is our detailed response:

Originality :

We understand concerns about using large pre-trained models akin to extra datasets. Yet, notable methods like ATD and ALOE also use external datasets like TinyImageNet as OE dataset. To highlight our work's novelty, here are some key points:

• Motivation : Developing robust outlier detection methods is essential for safety-sensitive real-world applications, like autonomous driving. Although clean results on challenging benchmarks are near 100% AUROC, the top results from previous works drop to below random levels of detection under attack scenarios, even with tiny datasets.

• Data Complexity : We agree with the reviewer that utilizing CLIP may provide supervision for our pipeline, however, the real-world application's demand for reliability necessitates robust methods, even at the expense of data complexity. As the reviewer noted, several studies have utilized pre-trained models to enhance outlier detection. These studies have employed models such as ViTs (trained on 21M samples)[1], BERT (trained on 1B data)[2], and CLIP (trained on more than 400M data)[3,4,5] in the outlier detection domain. Despite their notable performance in clean settings, they are vulnerable to adversarial. In contrast, our approach employs a smaller CLIP variant, trained on 65M samples, which not only withstands strong attacks more effectively but also delivers competitive performance in standard settings.

• Robust Methods : Prior robust OOD detection techniques were largely tailored to certain scenarios like multi-class outlier detection, with examples like ALOE, ATD, and ATOM being restricted to this setup. Our method, however, is versatile, effectively working across various detection scenarios and datasets, including those in medical imaging and industry, overcoming the constraints of earlier approaches.

• Different usage of CLIP : The utilization of CLIP in previous OOD detection methods differs from our approach. Previous works directly leveraged the rich features of CLIP to develop a detector. However, we implicitly utilize CLIP's guidance to generate near and diverse auxiliary outliers for utilizing adversarial training in subsequent steps.

In order to better address the reviewer's concern, we also compare our method with EXOE[5], a recent state-of-the-art novelty detection method based on a CLIP model trained on 400M data, and provide the results here for review:

Our method demonstrates superior effectiveness in outlier detection, outperforming other SOTA techniques in both clean and adversarial environments, with the latter being equally crucial.

Quality-1 :

One limitation of our method is its adversarial performance in fine-grained, pixel-level outlier detection, specifically in datasets like MVTecAD. Although our robust detection outperforms other methods, it still requires further development to achieve promising results. A potential reason for our method's limitation with this dataset is that inliers and outliers have very subtle differences at the texture level, making them susceptible to conversion between inliers and outliers by perturbations from strong attacks like PGD.

Quality-2:

We kindly request that you review our common response.

Clarity :

• The generated outliers must be conceptually distinct from inlier data to prevent misleading our detector during training. Therefore, we have defined two filtering steps using two different thresholds, $\mathcal{T}_{\text{Image}}$ and $\mathcal{T}_{\text{Label}}$, each specified distinctly. After extracting Near-OOD labels with word2vec, we initially filter out labels that exhibit significant similarity to the inlier labels, ensuring we do not generate inlier-like data in subsequent steps. A second threshold is applied after generating images with the CLIP's guidance, targeting the removal of images that are highly similar to inliers, thus excluding those that could be misconstrued as inlier. This strategy directly supports our main claim that a fundamental aspect of OE is its conceptual differentiation from the inlier.

• This was a typo; 'C' must be replaced with $y_{\text{NOOD}}$.

[1]M Cohen Transformaly CVPR 2023

[2]K XU Unsupervised ACL 2021

[3]S Esmaeilpour Zero AAAI 2023

[4]M Felix Contrastive arXiv 2023

[5]P Liznerski Exposing TMLR 2023

Should you find that our revisions have successfully addressed your initial concerns, we would be grateful for your consideration in positively adjusting the paper's score.

Thank you for your review and useful comments. We have made a revision based on the reviewers' concerns, and specific comments are answered below:

W1:

We kindly refer you to our common response, which provides a more detailed theoretical derivation.

W2-1:

We apologize for any inconvenience caused by the lack of clarity in the caption of Figure 1. We have since revised the caption to provide a clearer understanding and would like to offer additional insight into Figure 1 here.

Figure 1 represents the motivation for pursuing adaptive outlier exposure (OE) in our paper. OE refers to a technique used in outlier detection literature, which leverages a fixed auxiliary outlier dataset (e.g., ImageNet) during training to improve detection methods by providing more information about outlier data.

In the cases shown in Figure 1, CIFAR10 is treated as the inlier training set, while CIFAR100 serves as a source of Out-of-Distribution (OOD) data unavailable during training time. The ALOE and ATD aim to robustly discriminate them during test time. Both methods employ Tiny ImageNet as an auxiliary outlier set (i.e., OE) during adversarial training, as integral components of their proposed frameworks. In our experiment, while keeping all other aspects of the original methods constant, we replaced Tiny ImageNet with SVHN, MNIST, and Gaussian noise and repeated the experiments for both ALOE and ATD. This replacement led to a notable decline in OOD detection performance for ALOE and ATD on the CIFAR10 vs. CIFAR100 task, particularly under adversarial attack conditions. We attribute this performance drop to the fact that the SVHN, MNIST, and Gaussian Noise distributions are more distant from CIFAR10 (the inlier distribution in this task) compared to Tiny ImageNet. The results are also presented in detail here:

W2-2:

Previous methods utilized random, fixed auxiliary outlier datasets as OE, which we believe make their methods ineffective when the inlier distribution and the auxiliary outlier set are distant. We support this with our Figure 1, theoretical insight, and extensive ablation studies. For instance, as our Table 4 results indicate, using ImageNet as OE for MNIST as the inlier set is less effective compared to using ImageNet as OE for CIFAR10 as the inlier set. This is because both ImageNet and CIFAR10 include natural images and have less distribution distance compared to ImageNet and MNIST.

W3:

Our ablation study incorporates several alternative OE datasets/techniques for comparison with our proposed adaptive OE method. To address your concern regarding the performance of distant outliers in OOD detection, such as Gaussian Noise referenced in Figure 1, we have conducted experiments using Gaussian Noise as OE. We have updated our tables with these results and provide them here for your review:

In ablation studies tables, we have updated the tables accordingly by adding Gaussian Noise as an additional OE technique. Here we have brought the updated AUROC and FDC tables respectively:

We have strived to address each point you raised comprehensively. If our rebuttal has resolved these issues to your satisfaction, we would greatly appreciate it if you could reassess your scoring of our paper.