An Evidence-Based Post-Hoc Adjustment Framework for Anomaly Detection Under Data Contamination

We thank the reviewer for their constructive feedback. We are happy to note that the reviewer finds our proposed method well motivated, clearly explained and empirically effective in improving AD performance. We address the reviewer's concerns in the points below.

[W1, Q1] How does this approach differ from and compare with other methods which address data contamination in anomaly detection? While we acknowledge prior work on anomaly detection with contaminated data, to the best of our knowledge, no existing framework offers post-hoc adaptation in the same manner as EPHAD. Unlike previous methods, EPHAD does not require access to the pre-trained anomaly detection (AD) model’s weights or training pipeline, making it fundamentally different in its approach. Our primary focus in the experiments is to demonstrate the effectiveness, versatility, and applicability of EPHAD, a simple yet powerful Bayesian inference-based framework, by evaluating its relative performance against the pre-trained model and the evidence function in isolation.

Nonetheless, for completeness, we have compared EPHAD to two existing frameworks: ``Refine'' [1] and Latent Outlier Exposure (LOE) [2], as reported in Table 2 of our paper. While Refine iteratively identifies and removes anomalies during training, LOE incorporates identified anomalies using a contrastive loss.

[1] Yoon et al., "Self-supervise, Refine, Repeat: Improving Unsupervised Anomaly Detection" TMLR, 2022.
[2] Qiu et al., "Latent outlier exposure for anomaly detection with contaminated data." ICML, 2022.

Dear Reviewer qVBz,

Thank you once again for your time and insightful feedback, which has helped us improve our paper. We hope our rebuttal has addressed your concerns regarding the comparison of EPHAD with existing methods that address contamination in anomaly detection.

If you have any further questions or comments, please let us know before the discussion period ends. We are happy to clarify any remaining points.

Best regards,
The Authors

We thank the reviewer for their constructive feedback. We address the reviewer's concerns in the points below.

[W1] It remains unclear how the Evidence Function was designed. In our current draft, we provide additional details on the evidence functions in Appendix B.3. Notably, while EPHAD can incorporate evidence from foundation models like CLIP, it also allows the seamless integration of domain-specific knowledge. To demonstrate this versatility, we have shown CLIP as evidence for image-based experiments and LOF as evidence for tabular datasets. Furthermore, in the appendix, we have shown domain-specific rules as evidence. For this, we examine the AD method, forecastAD, for thermal images introduced by [1] and compare the performance of forecastAD with and without EPHAD under $10\%$ contamination. The results presented in Table 5 of our current draft show that while forecastAD experiences a performance drop of approximately $5\%$, incorporating domain-specific rules R1 and R2, as outlined in [1], provides evidence for EPHAD, which recovers about $2.5\%$ of the lost performance. Moreover, we show this generality beyond image data by applying EPHAD to tabular datasets using traditional AD methods such as LOF and Isolation Forest as the evidence function.

[W2, W3] CLIP inherently supports zero-shot anomaly detection (e.g., WinCLIP [a], MVFA [b]), which is unaffected by contamination in training data. Does using CLIP as the Evidence Function introduce data leakage? While CLIP has been employed in prior work for zero-shot anomaly detection, our approach does not rely on CLIP as a standalone anomaly detector. Instead, we leverage its image-text matching capabilities to provide auxiliary evidence that is integrated within a broader and more flexible framework. It is also worth highlighting that foundation models like CLIP are not applicable in specialised applications such as the detection of anomalous behaviour in solar power plants due to the lack of semantic content in thermal images. This makes zero-shot methods like WinCLIP and AnoCLIP inapplicable. In contrast, while EPHAD can incorporate evidence from foundation models like CLIP, it also allows the seamless integration of domain-specific knowledge as shown in Appendix C.1 of our current draft. This makes our framework more adaptable and broadly applicable across domains.

For the experiments on image-based datasets, we adopt CLIP as used by the authors of WinCLIP, following their zero-shot detection setup (not segmentation). The exact implementation details, including the prompts used, are provided in Appendix B.3.1 of our submission. Thus, in our results, the baseline CLIP performance reflects the standalone performance of WinCLIP. Our method shows the gains achieved when integrating CLIP-based evidence with a base AD model using EPHAD.

Lastly, unlike MVFA, we do not fine-tune the CLIP model. To apply EPHAD in the medical domain, we can use any SOTA AD methods as the base detector with the finetuned CLIP or any other domain-specific evidence function.

[W4] The Introduction claims that existing methods rely on 'the contamination ratio.' However, these papers never mention such a ratio. While it is true that many existing AD methods that operate under contamination do not explicitly assume knowledge of the exact contamination factor, they often rely, either directly or indirectly, on having a reasonable approximation of it for effective performance. For instance, in [2], the contamination factor $\alpha$ is treated as a hyperparameter that "characterizes the assumed fraction of anomalies" in the training data. The authors further provide a sensitivity analysis (Figure 3), which clearly demonstrates that performance degrades significantly, especially for LOE-Hard, when the assumed value deviates from the true contamination level. Similarly, in [3], the threshold $\gamma$ is introduced for refining the contaminated training set. In Section 3.3, the authors note that their method (SRR) remains robust as long as $\gamma$ is selected from a reasonable range, which they define as 1 to 2 times the true anomaly ratio. This again suggests that their method implicitly depends on a rough estimate of the true contamination level for optimal results. Therefore, while these works may not rely on the exact contamination factor, they do depend on having a reasonable approximation.

[1] Patra et al., "Detecting Abnormal Operations in Concentrated Solar Power Plants from Irregular Sequences of Thermal Images." KDD, 2024.
[2] Qiu et al., "Latent outlier exposure for anomaly detection with contaminated data." ICML, 2022.
[3] Yoon et al., "Self-supervise, Refine, Repeat: Improving Unsupervised Anomaly Detection" TMLR, 2022.

Dear Reviewer qm8P,

Thank you once again for your time and insightful feedback, which has helped us improve our paper. We hope our rebuttal has addressed your concerns regarding the design of the evidence functions, including the validity of using CLIP, as well as our discussion of existing methods that rely on prior knowledge of the contamination ratio.

If you have any further questions or comments, please let us know before the discussion period ends. We are happy to clarify any remaining points.

Best regards,
The Authors

We sincerely appreciate the reviewer's thoughtful and constructive feedback. It is encouraging to hear that the reviewer finds our paper to be well written with clear motivation and a strong mathematical background. We address the reviewer's concerns in the points below.

[W1, Q1] Why is it valid to use the test-set as extra information to improve anomaly detection? We would like to clarify a key aspect of our method, EPHAD, which may help address the concern regarding potential test set exposure.

EPHAD is explicitly designed as a test-time adaptation (TTA) method in which anomaly scores produced by a detector trained on a contaminated dataset are updated during inference using an auxiliary evidence function. Importantly, during adaptation, we do not have access to ground-truth labels for test samples. This ensures that our approach strictly avoids data leakage and remains consistent with standard practices in the TTA literature [1,2].

We would also like to highlight a critical difference between Source-Free Unsupervised Domain Adaptation (SF-UDA) and TTA. While both operate without test labels, SF-UDA typically involves a separate, offline adaptation phase before inference, often requiring iterative fine-tuning on the unlabelled validation samples from the test distribution [3]. In contrast, TTA methods, under which EPHAD falls, do not have a separate model adaptation step. Here, the model is adapted during inference. The benefits of TTA approaches are twofold: (i) they avoid iterative training, improving the computational efficiency, and (ii) they do not require additional validation samples, which is beneficial for data-starved settings such as AD.

Moreover, EPHAD takes a step forward by being model-agnostic. It operates in a black-box setting, requiring only the output scores from the base detector. This flexibility broadens its applicability, particularly in settings where access to model internals is restricted.

We hope this clarifies that the effectiveness of EPHAD does not arise from any exposure to the test set, but rather stems from our proposed adaptation strategy, aligned with established TTA methodologies.

[W2] It'd be great to expand a bit more in Lines L183-L185. We have expanded lines 183-185 in our updated draft describing how EPHAD is a simple yet effective method for real-world applications. The revised text reads as follows:

EPHAD is a simple yet effective inference-time adaptation framework aimed at mitigating the effects of training data contamination without requiring access to the pre-trained anomaly detection model’s weights or original training pipeline. This makes EPHAD uniquely suited for integration with existing systems, especially when retraining is not feasible due to resource, access, or time constraints. EPHAD can seamlessly incorporate evidence from foundation models such as CLIP and also domain-specific knowledge in specialised applications.

[W3] L173: Should be Eq. (3), not (3). In the current draft, we have referred to equations using their corresponding number within the first bracket, e.g. (3). We have updated the format in our updated draft.

[W4] L195-196: How do you choose the threshold? Threshold selection plays a crucial role in deciding the final label given an anomaly score. Thus, several existing studies are dedicated solely to this aspect [4-6]. However, the primary focus of our work lies in refining the anomaly scoring mechanism itself, not in optimizing the decision threshold. As such, determining a fixed threshold falls outside the scope of this study. To ensure a fair and threshold-independent evaluation aligned with our objectives, we report performance using AUROC.

[W5, W6] Eq (13): I think you have an extra "exp". L291: I think the $\epsilon\%$ should only be $\epsilon$. We have fixed the typos in our updated draft.

[W7] I would expand a bit more in how did you use the evidence function. Following the reviewer's suggestion and to help our readers, we added further details on how CLIP is used as an evidence function, which was previously in our appendix.

[Q2] How would you use this technique in a real-world scenario? The approach presented in our current draft requires access to the full unlabelled test set to compute the outlier probability. Thus, it cannot be applied in online settings. However, we provide an alternative approach to applying EPHAD, which does not require converting the anomaly score to outlier probability for non-density-based methods. Consequently, EPHAD can be applied in an online setting, provided the evidence function can be computed sample-wise, such as when using CLIP or rule-based evidence functions. The alternative version of EPHAD is summarised below:

Given an AD model trained on the contaminated dataset, we denote the anomaly score for a data point $x$ as $s^-_u(x)$. Then, we can compute the inlier score as $s^+_u(x) = - s^-_u(x)$. Considering $s^+_u(x)$ as prior and an auxiliary evidence function $T(x)$, EPHAD computes the revised score $s^+_c(x)$ using exponential tilting. It is a technique used to adjust a PDF by ``tilting'' it toward a specific outcome. Recall that the inlier score is an order-preserving transformation of the inlier PDF, i.e., $s^+(x) = \phi(f^+_X(x))$ where $\phi$ is a transformation function, e.g. log-likelihood. Thus, given a score-based AD model that learns $s^-_u(x)$, tilting increases the relative scores of the normal samples over the anomalous samples, steering the model toward an outcome supported by the evidence function. We first exponentiate the inlier score $s^+_u(x) = - s^-_u(x)$ to ensure non-negativity. Then, we rescale $T(x)$ with a temperature parameter $\beta$ and exponentiate it. Finally, the revised inlier score is computed as:

$s^+_c(x) = \exp(s^+_u(x))\exp(T(x)/\beta).$

[1] Wang et al., "Tent: Fully test-time adaptation by entropy minimization." ICLR 2021.
[2] Xiao et al., "Beyond model adaptation at test time: A survey." ArXiv 2024.
[3] Fang et al., "Source-free unsupervised domain adaptation: A survey." Neural Networks 2024.
[4] Perini et al., "Estimating the contamination factor’s distribution in unsupervised anomaly detection." ICML, 2023.
[5] Perini et al., "Transferring the contamination factor between anomaly detection domains by shape similarity.", AAAI, 2022.
[6] Hendrickx et al., "Machine learning with a reject option: a survey." Machine Learning, 2024.

Dear Reviewer 68bH,

Thank you once again for your time and insightful feedback, which has helped us improve our paper. We hope our rebuttal has addressed your concerns regarding the validity of our approach and the application of EPHAD in real-world scenarios.

If you have any further questions or comments, please let us know before the discussion period ends. We are happy to clarify any remaining points.

Best regards,
The Authors

Dear authors,

Thanks for the detailed rebuttal. You response does answer my main concerns. I appreciate your more detailed explanation about the settings of your method and I believe is a moderate to good addition to the anomaly detection literature. Therefore, I'll update my score from Reject to Accept, in light to not fall into any Borderline option. I think once concern I still have is the one raised by reviewer qm8P about CLIP data leakage, so I'd recommend to go deeper in the discussion in the final paper. In any case, it is a good rebuttal, well done!

We sincerely appreciate the reviewer’s insightful and valuable feedback. We are happy to note that the reviewer finds the problem addressed in our work to be an important but under-explored real-world challenge. We address the reviewer's concerns in the points below.

[W1, Q4] Thresholding Left Unspecified. Selecting a threshold is an essential consideration in deciding the final label given an anomaly score. Thus, several existing studies are dedicated solely to this aspect [1-3]. However, the primary focus of our work lies in refining the anomaly scoring mechanism itself, not in optimising the decision threshold. As such, determining a fixed threshold falls outside the scope of this study. To ensure a fair and threshold-independent evaluation aligned with our objectives, we report performance using AUROC.

[W2, Q2] No reporting of standard deviations in result table. We have now repeated all the experiments with three seeds in our updated draft.

[W3, Q3, L3] Lack of resource efficiency reporting or runtime metrics. We would like to emphasise that EPHAD operates without modifying the base AD model and does not introduce any additional fine-tuning steps. As a result, the memory and computational requirements remain unchanged relative to the original model. The only additional cost stems from the auxiliary evidence function, whose overhead depends on the specific implementation chosen.

For example, in our real-world use case involving solar power plants, a simple rule-based heuristic serves as the evidence function, resulting in negligible memory and compute overhead. Similarly, in the tabular AD setting where we employ Local Outlier Factor (LOF), the added cost is minimal. In scenarios involving foundation models such as CLIP, while there is an increase in memory usage due to model size, the latency remains low, as adaptation involves only a single forward pass followed by a lightweight similarity computation with predefined templates.

To better illustrate these trade-offs, we will include a dedicated section in the updated draft of our paper that reports runtime and memory overheads across the different use cases under consideration to provide clarity about the practical applicability of EPHAD.

[W4] Missing discussion of broader societal impacts. Thank you for your suggestion. We will incorporate a section discussing the broader impact of our work in the updated draft. EPHAD provides an effective way to adapt pre-trained AD models during inference, removing the need for specialised architectures to handle data contamination. This is crucial for high-stakes applications such as industrial automation and medical diagnostics.

[Q1, L1] Can EPHAD be adapted for online settings? For the approach discussed in our current draft, we need to have the unlabelled test set together to compute the outlier probability. Thus, it cannot be applied in online settings. However, we provide an alternative approach to applying EPHAD, which does not require converting the anomaly score to outlier probability for non-density-based methods. Consequently, EPHAD can be applied in an online setting, provided the evidence function can be computed sample-wise, such as when using CLIP or rule-based evidence functions. The alternative version of EPHAD is summarised below:

Given an AD model trained on the contaminated dataset, we denote the anomaly score for a data point $x$ as $s^-_u(x)$. Then, we can compute the inlier score as $s^+_u(x) = - s^-_u(x)$. Considering $s^+_u(x)$ as prior and an auxiliary evidence function $T(x)$, EPHAD computes the revised score $s^+_c(x)$ using exponential tilting. It is a technique used to adjust a PDF by ``tilting'' it toward a specific outcome. Recall that the inlier score is an order-preserving transformation of the inlier PDF, i.e., $s^+(x) = \phi(f^+_X(x))$ where $\phi$ is a transformation function, e.g. log-likelihood. Thus, given a score-based AD model that learns $s^-_u(x)$, tilting increases the relative scores of the normal samples over the anomalous samples, steering the model toward an outcome supported by the evidence function. We first exponentiate the inlier score $s^+_u(x) = - s^-_u(x)$ to ensure non-negativity. Then, we rescale $T(x)$ with a temperature parameter $\beta$ and exponentiate it. Finally, the revised inlier score is computed as:

$s^+_c(x) = \exp(s^+_u(x))\exp(T(x)/\beta).$

[Q5, L2] How does EPHAD handle cases where the evidence function $T(x)$ is noisy? How robust is EPHAD if the evidence is misleading? In our experiments on the image-based AD datasets presented in Table 1, we employed CLIP as the evidence function. In certain datasets, such as SVHN, the evidence function can be noisy. For instance, when used in isolation (i.e., without EPHAD), CLIP achieves an AUROC of only 58.46% on SVHN, indicating limited reliability in this context. In such cases, applying EPHAD with the default setting of $\beta=0.5$ does not lead to improvements over the base detector. However, we emphasise that $\beta$ is a tunable parameter. As shown in Figure 4 in our current draft, EPHAD offers a smooth transition between the two models, illustrating not only reliability but also adaptability, offering a principled way to balance the contributions of the base detector and the evidence function. To further support this adaptability, we provide an unsupervised strategy for selecting an appropriate $\beta$ in Appendix C.6.

[1] Perini et al., "Estimating the contamination factor’s distribution in unsupervised anomaly detection." ICML, 2023.
[2] Perini et al., "Transferring the contamination factor between anomaly detection domains by shape similarity.", AAAI, 2022.
[3] Hendrickx et al., "Machine learning with a reject option: a survey." Machine Learning, 2024.