Fine-grained Abnormality Prompt Learning for Zero-shot Anomaly Detection

We sincerely thank your valuable and constructive feedback and suggestions. Below, we provide a detailed, point-by-point response to your questions, and hope these could help address your concerns.

Weakness #1. "The design of abnormality prompts is not entirely consistent with the concept of "fine-grained." Although CAP leverages a set of abnormality prompts with an orthogonal constraint loss, the final abnormality embedding is derived as the average of all abnormality prompt embeddings, resulting in a vector-based abnormality prompt prototype that essentially reduces the diversity of abnormalities. If there are multiple types of anomalies within an image, is a single abnormality prompt prototype sufficient?"

Please refer to our response to Shared Concern #5 in the General Response section above for detailed clarification of why the averaging operation is used in CAP.

To assess the performance on samples containing multiple anomalous types within a single image, we also provide visualization of pixel-level detection results on such samples from three MVTecAD categories (zipper, pill and wood) and AITEX. The results shown in Figure 9, page 24 (See $`Appendix`$ C.6) in the revised manuscript demonstrate that despite using a single abnormality prompt prototype, $`FAPrompt`$ can still effectively detect multiple anomaly types in a single image.

Weakness #2. "The number of abnormality prompts, K, shows very limited difference to pixel-level detection results (see Figure 7), making it difficult to evaluate the necessity of multiple abnormality prompts. Additionally, Figures 4 and 7 are missing the results when K=1."

Please refer to our response to Shared Concern #4 in the General Response section above for results of $`FAPrompt`$ with $K=1$. We also add the results with $K=1$ into Figures 4 and 7 in our revised manuscript.

As illustrated by the updated Figures 4 and 7, we observe noticeable differences in pixel-level metrics, particularly in AUPRO, with varying $K$ values. This supports the importance of using multiple prompts for fine-grained abnormality detection, since increasing $K$ improves the model's ability for more diverse abnormality learning and better detection performance at the pixel level.

Weakness #3. "The proposed DAP is technically similar to CoCoOp, and it uses the features of selected image patches instead of the global image feature. Moreover, the parameter M appears to be quite sensitive for pixel-level ZSAD."

Please refer to our response to Shared Concern #1 in the General Response section above for clarification of difference between DAP and CoCoOp, and the sensitivity analysis of $M$.

Dear Reviewer RtD8,

We would like to gently remind you of our recent response. If we have sufficiently addressed your questions, we would be grateful if you could consider updating your rating in your review.

If you have any additional questions or need further clarification, please feel free to let us know. We are more than happy to provide any additional information you might need. Thank you again for your time and valuable feedback.

Authors

We sincerely thank your valuable and constructive feedback and suggestions. Below, we provide a detailed, point-by-point response to your questions, and hope these could help address your concerns.

Weaknesses #1. "The novelty of $`FAPrompt`$ is limited. The design of CAP seems a simple combination of prompt tuning and prototype learning."

Our work introduces a new ZSAD framework that devises two novel modules -- the CAP and DAP modules -- to learn fine-grained abnormalities. As far as we know, there is no similar work done on this problem. The CAP module differs significantly from existing prompt tuning methods in terms of both intuition and design:

• Intuition. The compound abnormality design in CAP is based on the insight that abnormal samples exhibit varying magnitudes of deviation from their normal counterparts (see Lines 221-224). Thus, it is essential for having prompts that are able to capture the varying abnormalities for effective ZSAD. CAP is designed specifically for this purpose. In contrast, previous prompt tuning-based AD methods are focused on single abnormality prompt learning, and thus, do not account for this characteristic of abnormalities.
• Design. To learn diverse, complementary abnormalities, we introduce a set of compound abnormality prompts in CAP, guided by an orthogonal constraint on the prompts. This helps effectively encourage the learning complementary abnormality prompts, enforcing that they capture distinct aspects of abnormalities. This effect is illustrated by visualization results in Figure 3 and Figure 6 in the revised manuscript. Please also refer to our response to Shared Concern #2 in the General Response section for a clarification on the significance of the compound prompt design). As far as we know, this design is unique and novel in the AD and other topics.

In addition to CAP, the DAP module also has major novelty. Please refer to our response to Shared Concern #1 in the General Response section above for the innovation of DAP.

Weakness #2. "Besides CoOp and CoCoOp, other state-of-the-art prompt tuning approaches are expected for comparisons (e.g. PromptSRC[1] and TCP[2])."

Both PromptSRC and TCP are not originally designed for Anomaly Detection, and their contextual information relies heavily on handcrafted text prompts. Due to time constraints during the discussion period, we focused on conducting additional experiments with the more recent TCP approach.

For a fair comparison, we maintained the original design and settings of TCP throughout the experiments. Specifically, we adapted TCP for the ZSAD by testing two types of AD-oriented text prompts, resulting in two variants of TCP for ZSAD: TCP_V1 and TCP_V2:

• TCP_V1, where we used a straightforward prompt design:

  Normal prompt: "This is a photo of [cls]."; Abnormal prompt: "This is a photo of damaged [cls].

• TCP_V2, where we adopted the complete set of the prompt templates from WinCLIP.

The results are shown in the table below:

As shown, both TCP variants largely underperform AnomalyCLIP and $`FAPrompt`$ in the ZSAD task. This is primarily due to the fact that TCP is not designed for ZSAD and also has strong reliance on handcrafted text prompts.

In contrast, $`FAPrompt`$ is specifically designed for the ZSAD task, leveraging data-dependent abnormality prior of the query images to learn complementary abnormality prompts. This adaptive approach enables $`FAPrompt`$ to more effectively capture a wide variety of anomalies, resulting in promising performance in both image-level and pixel-level ZSAD tasks. We have included these results in $`Appendix`$ C.2 to address this concern.

References

[1] Khattak, Muhammad Uzair, et al. "Self-regulating prompts: Foundational model adaptation without forgetting." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[2] Yao, Hantao, Rui Zhang, and Changsheng Xu. "TCP: Textual-based Class-aware Prompt tuning for Visual-Language Model." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

Question #1. "In table 4, image-level results on medical datasets show that CAP achieves the best result compared with $`FAPrompt`$, is there any further explanation?"

The AP metric is sensitive to detection precision and recall. In smaller datasets, a single incorrect or correct detection can significantly influence these two metrics, causing more variability in AP. Our breakdown ablation study results (Table 17) in $`Appendix`$ D.2 provides a comprehensive analysis of image-level results on four different medical datasets. Notably, the difference in AP between CAP and $`FAPrompt`$ is observed primarily in the HeadCT dataset, which is small with only 200 samples (further details can be found in the $`Appendix`$ A.1). In this dataset, '+ CAP' achieves an AP of 94.6, while $`FAPrompt`$ achieves an AP of 93.5. However, in the other three datasets, including BrainMRI (253 samples), LAG (2,497 samples) and Br35H (3,000 samples), $`FAPrompt`$ consistently outperforms '+ CAP'.

Question #2. "How to demonstrate that CAP's ability of learning fine-grained abnormality semantics? In fig 3, fine-grained prompts (with different colors) seems not to be seperable."

Please refer to our response to Shared Concern #3 in the General Response section above for clarification and empirical evidence on the ability of learning fine-grained abnormalities.

Dear Reviewer agpf,

The author-reviewer discussion will end soon. Please kindly check whether our response helps address your concerns or not. We're more than happy to provide more details or empirical justification if you have any follow-up questions. Thank you!

Best regards,

Authors of Paper 9942

We sincerely thank your valuable and constructive feedback and suggestions. Below, we provide a detailed, point-by-point response to your questions, and hope these could help address your concerns.

Weaknesses #1. "The description of the learnable layers in the DAP module is missing."

The original manuscript has included the description of the learnable layers in DAP, which is also the abnormality prior network $\psi(\cdot)$ (See Line 274-276). We also provide the network details of $\psi(\cdot)$ in Appendix B.1.

Weaknesses #2. "The visualization in Figure 1 of the proposed methodology conflicts with the actual experimental results shown."

Thank you for your insightful feedback. We apologize for any confusion caused by Figure 1. The diversity of fine-grained abnormality prompts in our method is learned through the compound abnormality prompting and an orthogonal constraint loss, rather than by relying on additional human annotations for specific types of anomalies. Our intention is not to claim that each individual prompt represents exactly a particular anomaly type. Instead, our goal is to encourage diversity and complementarity among the learned fine-grained abnormality prompts. To address this issue, we have revised the data visualization in Figure 1 and clarify the explanation in the revised manuscript to prevent the potential confusions. This visualization is now consistent with our observations from other qualitative results, such as those in Figure 3.

Questions #1. "Please provide specific examples or criteria that distinguish 'coarse-grained' prompts from the proposed fine-grained prompts. Clarifying this would help determine if the label is used due to a single prompt approach or if there is a deeper rationale."

Thank you very much for the suggestion. Please refer to our response to Shared Concern #3 in the General Response section above for clarification and empirical evidence on the ability of $`FAPrompt`$ in learning fine-grained abnormalities.

Questions #2. "Consider alternatives to averaging the abnormality prompts in the CAP module, such as selecting the most similar prompt for each detected abnormality. If other approaches were tested, sharing the rationale or results behind choosing the current method would add clarity."

and Questions #5. "Could the authors clarify how this approach differs from existing prompt ensemble methods? Specifically, what distinguishes the process of averaging the abnormality prompts from a traditional ensemble approach, and in what ways does this differ from selecting the highest anomaly score across individual abnormality prompts."

Thank you very much for the suggestion. Please refer to our response to Shared Concern #5 in the General Response section above for detailed clarification and empirical evidence on alternative methods comparing to the averaging method in CAP.

Questions #3. "Could the authors provide additional evidence or clarification on how the fine-grained distributions in Figures 1 and 3 align with each other?Additional context or visual consistency would strengthen the claim that each abnormality prompt represents a specific abnormal state."

We apologize for the misleading that each abnormality prompt represents a specific abnormal state' implied in Figure 1. We aim to highlight throughout the paper except Figure 1 that $FAPrompt$ learns a set of complementary, fine-grained abnormality prompts. Indeed, each abnormality prompt learned in $FAPrompt$ does not correspond to a specific abnormal state/type. This is very challenging to achieve without detailed human annotations of abnormality on specific abnormal states/types (recall that $FAPrompt`$ and its competing methods are trained with coarse binary anomaly labels only). Please refer to our response to Weakness #2 for the consistency between Figures 1 and 3.

Questions #4. "The explanation regarding the effectiveness of the compound prompt seems insufficient. Could more information be provided on the outcomes when combining the normal learnable token with the abnormality token? Specifically, what results were achieved if CAP and DAP were applied independently, without this combination?"

The original manuscript has included ablation studies where CAP and DAP were applied independently (see Table 3). The results demonstrate that both CAP and DAP independently contribute to improved performance over the base model, and the full model $`FAPrompt`$ achieves its best results.

This conclusion also holds when the number of abnormality prompts changes. Please refer to our response to Shared Concern #4 in the General Response section above for detailed clarification.

The effectiveness of our compound abnormality prompts is also verified by comparing $`FAPrompt`$ to the prompt ensemble methods based on AnomalyCLIP. Please our response to Shared Concern #2 in the General Response section above for detailed clarification on this.

We sincerely thank your valuable and constructive feedback and suggestions. Below, we provide a detailed, point-by-point response to your questions, and hope these could help address your concerns.

Weakness #1. "There is marginal improvement in pixel-level ZSAD in Table 2. In contrast, the simple AnomalyCLIP achieves comparable, and even superior, results on industrial datasets."

Thank you very much for the comment, but it misses various important empirical results.

It is true that the pixel-level ZSAD improvement of $`FAPrompt`$ over AnomalyCLIP is marginal on part of the industrial datasets due to relatively small sample size and/or simpler pixel-level anomaly patterns in these datasets, but the improvement is large on the other industrial datasets that have a larger sample size or more complex anomaly patterns like DAGM. The latter case is particularly true on the medical image datasets, where the pixel-level anomaly patterns are more difficult to detect under the ZSAD setting, leading to AUROC performance typically below 0.90. On such challenging datasets like DAGM and medical image datasets, $`FAPrompt`$ achieves large pixel-level ZSAD performance over all competing methods, including the best competing method AnomalyCLIP.

Moreover, we'd like to highlight the significant image-level ZSAD improvement of $`FAPrompt`$ over all competing methods across both industrial and medical image datasets, as shown in Table 1 in the main text.

Therefore, $`FAPrompt`$ offers a substantially better ZSAD solution than existing methods in both pixel- and image-level anomaly detection.

Weakness #2. "The design of DAP is very similar to CoCoOp, and using a fixed M across images with varying scales of anomalous regions is unreasonable."

Please refer to our response to Shared Concern #1 in the General Response section above for detailed clarification.

Weakness #3. "Missing necessary baseline: AnomalyCLIP with an ensemble of multiple abnormality prompts with orthogonal constraint loss; otherwise, it is difficult to justify the fine-grained prompts."

Thank you for your insightful feedback regarding the baselines. Please refer to our response to Shared Concern #2 in the General Response section above for detailed clarification and empirical evidence of the significance of compound design in fine-grained prompts.

Questions #1. "Could you provide additional evidence to substantiate the claim that the prompt-wise anomaly scores visualized in Figure 3 demonstrate the discriminability of $`FAPrompt`$? A comparison with visualizations from baseline models would effectively highlight the advantages."

Thanks for the valuable suggestion. Please refer to our response to Shared Concern #3 in the General Response section above for clarification and empirical evidence on the ability of learning fine-grained abnormalities.

Questions #2. "To better verify the necessity of using multiple prompts, it is suggested to include more detailed ablation experiments, such as k=1 with and without CAP/DAP, similar to those in Table 4."

Please refer to our response to Shared Concern #4 in the General Response section above for detailed discussion.

Questions #3. "Ablation studies on the number of layers with learnable tokens and the length of the tokens should be included."

We would like to clarify that the learnable token design follows the settings used in AnomalyCLIP and is not part of our main contributions. We use exactly the same design in this part to ensure that the performance gain/loss of $`FAPrompt`$ compared to AnomalyCLIP is not affected by this part. For clarity, a detailed description of the learnable tokens is included in the original/revised manuscript ($`Appendix`$ B.1).

To evaluate the sensitivity, we also conduct additional ablation studies on the learnable tokens. The results are shown in the tables below:

# Layers having learnable tokens：

# Length of learnable token：

As shown, the performance generally gets improved with an increasing number of layers, reaching optimal performance at 9 layers. Beyond 9 layers, it tends to over-generalization, leading to a decrease in the detection performance. A similar pattern was observed with the token length, where $`FAPrompt`$ achieves the best overall performance with a token length of 4 and 6. We have included related results in $`Appendix`$ C.5 of the revised manuscript.

Questions #4. "Minor: I recommend merging Table 3 and Table 4 for improved method comparison."

Thank you for the suggestion. We have combined Table 3 and Table 4 into a single table in the revised manuscript.

Shared Concern #5. Significance to Averaging Strategy in CAP [nnbo, RtD8].

Despite the simplicity, the use of the averaging operation is due to its general effectiveness in aggregating multiple patterns. This strategy is also widely used in existing ZSAD and FSAD methods, such as WinCLIP and AnoVL, to deal with diverse and complementary abnormality text information.

To validate its advantage over the alternatives, we conduct additional experiments to evaluate two variants of $`FAPrompt`$, with the results presented in table below:

$`FAPrompt`_{0.1}:$ Selecting the Most Similar Prompt for Each Detected Abnormality.

In this variant of $`FAPrompt`$, we calculate the cosine similarity between the individual abnormality prompts and each test image to select the similarity to the most similar prompt as the anomaly score during inference. While this approach shows comparable performance on image-level ZSAD results, it can largely underperform the primary $`FAPrompt`$ in pixel-level ZSAD. This is mainly because selecting only a single prompt can lead to the loss of complementary information from other abnormality prompts, limiting the model's ability to detect the full spectrum of abnormalities.

$`FAPrompt`_{0.2}:$ Using Weighted Abnormality Prompts.

In this variant, we use a prompt importance learning network to learn a set of weights for each abnormality prompt based on the selected most abnormal patch tokens of the query images. These weights are then used to combine multiple abnormality prompts into a single weighted abnormality prompt (a weighted abnormality prototype) for ZSAD.

Although this variant outperforms $`FAPrompt`_{0.1}$ by retaining some complementary abnormality information, it does not match the performance of the simple averaging. This may be due to the greater power of the model in fitting the query images, which can lead to overfitting of the tuning auxiliary dataset in the zero-shot setting, i.e., the learned weights may well reflect the significance of each prompt in the tuning dataset but not in the target datasets.

Given these results, we chose to average the abnormality prompts in $`FAPrompt`$, as it offers a straightforward yet effective way to integrate diverse abnormalities while preserving their complementary information. We have included related results and analysis in $`Appendix`$ C.4 in the revised manuscript.

Shared Concern #2. Significance of Compound design in Abnormality Prompts [TTqm, nnbo, agpf].

We conduct additional experiments for AnomalyCLIP with an ensemble of multiple abnormality prompts and an orthogonal constraint loss, denoted by AnomalyCLIP Ensemble*. The results are shown in the table below, which is included in Table 3 in the revised manuscript:

As shown, AnomalyCLIP Ensemble* shows only some modest improvement over the baseline AnomalyCLIP, and it is not always more effective than the simple AnomalyCLIP ensemble approach, AnomalyCLIP Ensemble (already included in original manuscript).

However, $`FAPrompt`$ consistently outperforms this ensemble method across both image-level and pixel-level ZSAD results. This better performance of $`FAPrompt`$ can be attributed to the effectiveness of compounded prompt design in multiple abnormality prompts and abnormality prior from the DAP module. This helps justify that the fine-grained abnormalities learned in $`FAPrompt`$ cannot be learned in simple prompt ensemble approaches.

Shared Concern #3. Additional Evidence for Verifying Significance of Fine-grained Abnormality Prompts in Figure 3 [TTqm, nnbo, agpf].

We provide t-SNE visualization and quantitative results of AnomalyCLIP and the prompt ensemble method AnomalyCLIP Ensemble* for their comparison with $`FAPrompt`$ on the datasets. The results are included in Figure 6, page 20 of the revised manuscript ($`Appendix`$ C.3). Note that the difference between AnomalyCLIP and $`FAPrompt`$/AnomalyCLIP Ensemble* in the figure is because AnomalyCLIP learns one single abnormality prompt only while the $`FAPrompt`$/AnomalyCLIP Ensemble* learns 10 abnormality prompts.

• $`FAPrompt`$ vs. AnomalyCLIP. It is clear that compared to AnomalyCLIP, $`FAPrompt`$ learns a set of effective complementary abnormal patterns captured by the 10 abnormality prompts, resulting in better detection performance on datasets with complex anomaly cases. For example, on the datasets BTAD(01) and VisA (pcb4), several anomalies distributed very closely to, or overlapped with part of the normal images, are difficult to detect using single abnormality prompt in AnomalyCLIP, indicating that this abnormality prompt is not discriminative w.r.t. these anomalies. In contrast, $`FAPrompt`$ alleviates this situation with the abnormality prompts that show visually different, discriminative power.

  For datasets with simpler patterns like VisA (chewinggum), single abnormality prompt is sufficient, while having multiple abnormality prompts in $`FAPrompt`$ do not have adverse effects. This demonstrates the capability of $`FAPrompt`$ in achieving stable, effective detection across simple and complex datasets.

• $`FAPrompt`$ vs. the prompt ensemble method AnomalyCLIP Ensemble$*$. Despite also learning multiple abnormality prompts, it is clear from the visualization that the abnormality prompts in AnomalyCLIP Ensemble* tend to be clustered closely, while that in $`FAPrompt`$ is much more disperse, e.g., two clustered patterns on BTAD(01) and one clustered pattern on VisA (pcb4) learned by AnomalyCLIP Ensemble* vs. four disperse patterns on both datasets learned by $`FAPrompt`$. Importantly, the more disperse abnormal patterns from $`FAPrompt`$ provides complementary discriminative power to each other, substantiated by the enhanced AUROC/AP performance compared to AnomalyCLIP Ensemble*.

Shared Concern #4. Additional Results for $`FAPrompt`$ with $K=1$ [TTqm, nnbo, RtD8].

We conduct additional experiments for $`FAPrompt`$ with $K = 1$. The results are shown in table below:

Even without applying DAP, the $`FAPrompt`$ variant using a single compound abnormality prompt '+ CAP($K = 1)$)' also gains improved performance over the base model AnomalyCLIP. This improvement becomes more pronounced as $K$ increases to 10, which denoted as '+ CAP($K = 10$)'. This improvement indicates that multiple prompts are effective in capturing a broader spectrum of abnormalities.

The combination of '+ CAP(K = 1) + DAP' underperforms compared to using DAP alone. This is because '+ CAP($K = 1$) + DAP' relies on just a single abnormality prompt with a limited set of abnormal tokens, restricting its ability to capture the full diversity of abnormalities and leverage the abnormality prior provided by DAP effectively. However, when the number of abnormality prompts increases to 10, the ability of $`FAPrompt`$ to learn diverse abnormal patterns improves substantially. As a result, '+ CAP($K = 10$) + DAP', which is also our full $`FAPrompt`$, achieves the best performance across various datasets.

These results demonstrate that using multiple prompts enables $`FAPrompt`$ to better capture diverse, complementary abnormalities, maximizing the benefit of both CAP and DAP components for the overall superior performance. We have included related results in Figure 4 and Figure 7, as well as associated analysis in$`Appendix`$ C.5 in the revised manuscript.

Many thanks for your follow-up suggestion. We have conducted additional experiments with another alternative variant of $`FAPrompt`$, $`FAPrompt`_{0.3}$.

In this variant, we use the cosine similarity of each image patch token to its most similar abnormality prompt to obtain the pixel-level anomaly scores and generate the pixel-level anomaly detection map. In the same way as $`FAPrompt`$, the image-level anomaly score is then derived from the cosine similarity of the image token to its most similar abnormality prompt, combined with the maximum score from the pixel-level anomaly detection map. The image- and pixel-level AD results of $`FAPrompt`_{0.3}$ are reported in the table below.

The results indicate that while $`FAPrompt`_{0.3}$ achieves comparably good performance in pixel-level AD, it substantially underperforms the simply averaging method ($`FAPrompt`$) in image-level AD. This further supports our observation that relying on a single prompt can result in the loss of complementary information provided by the other abnormality prompts, thereby limiting the model's ability to effectively capture the full spectrum of abnormalities; and averaging the abnormality prompts serves as a simple yet effective way to aggregate the different abnormalities from the multiple abnormality prompts learned by $`FAPrompt`$.

Additionally, the strategy used in $`FAPrompt`_{0.3}$ can incur significant computational overhead, since it requires repeated computation to select the most similar prompt for each patch tokens, i.e., given an image having 1,369 (37x37) patch tokens in our experiments, there would be a computational overhead of $(1,370 \times 10 - 1,370 \times 1)$ similarity calculation per test image.

Therefore, in terms of either effectiveness or efficiency, averaging is a straightforward but more cost-effective way to integrate diverse abnormalities while preserving their complementary information.

Thank you very much for your comment. Since there are various ways to calculate the anomaly scores during training and inference of $`FAPrompt`$, to avoid confusion, we re-organize the results of $`FAPrompt`$, together with some additional results, to address your concern on how the fine-grained abnormality prompts should be used.

Notation. We use the below notations for better clarity.

• Training. We use ^{\ast}$$`FAPrompt` and $`FAPrompt`$ to represent two different ways of using the abnormality prompts during training. In particular, ^{\ast}$$`FAPrompt` denotes the variant of $`FAPrompt`$ that uses the cosine similarity of each patch token to its most similar abnormality prompt to obtain the pixel-level anomaly scores and generate the pixel-level anomaly detection map. Similarly, ^{\ast}$$`FAPrompt` uses the similarity between the image embedding and its most similar abnormality prompt to define the image-level similarity score, i.e., $s_a(x)$. $`FAPrompt`$ represents the default $`FAPrompt`$ in our paper that uses the abnormality prompt prototype generated from the $K$ multiple complementary abnormality prompts in the above two similarity calculations, rather than the best-matched individual abnormality prompt.
• Testing. At the testing stage, both models have three ways to calculate the image-level anomaly scores, including the use of the maximum pixel-level anomaly score solely, i.e., $s^{\prime}_a(x)$ in our paper (we denote it by NAME$-\text{P}$ here), the use of image-level similarity score solely, i.e., $s_a(x)$ in our paper (denoted by NAME$-\text{I}$), and their averaged combination as defined in Eq. 9 in our paper (denoted by NAME$-\text{PI}$), where NAME can be replaced by either $`FAPrompt`$ or ^{\ast}$$`FAPrompt`. Please be noted that the similarity calculation in both ^{\ast}$$`FAPrompt` and $`FAPrompt`$ during testing is consistent with those used in the training, i.e., the best-matched abnormality prompt is used in ^{\ast}$$`FAPrompt` and the abnormality prompt prototype is used in $`FAPrompt`$.

If we'd like to align these new notations with the ones used in the previous responses, $^{\ast}`FAPrompt`-\text{PI}$ is $`FAPrompt`_{0.3}$.

$`FAPrompt`-\text{PI}$ is the default $`FAPrompt`$ model used in the experiments in our paper. If we understand correctly, you requested the results of $`FAPrompt`-\text{P}$ in the latest comment.

Comparison of Image-level Results. The image-level results (AUROC, AP) are presented in the table below. We analyze each of the three image-level anomaly scoring methods for both $^{\ast}`FAPrompt`$ and $`FAPrompt`$ in detail in the following.

• Using maximum pixel-level anomaly score as image-level anomaly score ($s_a^\prime(x)$)

  Both $^{\ast}`FAPrompt`-\text{P}$ and $`FAPrompt`-\text{P}$ achieve strong image-level ZSAD performance in this case because the maximum pixel-level anomaly score effectively captures local abnormality information, which can be used to distinguish between normal and abnormal samples. This indicates that both the best-matched abnormality prompt in $^{\ast}`FAPrompt`-\text{P}$ and the averaged abnormality prompt in $`FAPrompt`-\text{P}$ are highly effective in representing pixel-level information within the patch token embeddings.

• Using image-level similarity score as image-level anomaly score ($s_a(x)$)

  $`FAPrompt`-\text{I}$ significantly outperforms $^{\ast}`FAPrompt`-\text{I}$, with the latter yielding poor performance. This large performance gap is due to the fact that the abnormality prompt learning in $^{\ast}`FAPrompt`-\text{I}$ being strongly influenced by their alignment with the 1,369 patch token embeddings, leading to severe bias toward patch-level anomaly scores when matching these prompts to the image token embedding.

  This argument is also supported by the comparably strong image-level performance of $`FAPrompt`_{0.1}$, as discussed in our previous response.

  The key difference between them is that $`FAPrompt`_{0.1}$ uses the same abnormality prompt (i.e., the one best-matched to the image token embedding) for both patch and image token embeddings, avoiding the bias caused by the best-matching to a large set of patch tokens, whereas $^{\ast}`FAPrompt`-\text{I}$ selects separate best-matched prompts for patch and image token embeddings.

  These results highlight the stronger power of the averaged abnormality prompt in $^{\ast}`FAPrompt`-\text{I}$ in capturing global abnormality information within the image token embedding.

• Using the average of $s_a^\prime(x)$ and $s_a(x)$ as image-level anomaly score

  When using both local and global anomaly scores, $`FAPrompt`-\text{PI}$ consistently outperforms $^{\ast}`FAPrompt`-\text{PI}$ by a significant margin. This demonstrates that the averaged fine-grained abnormality prompts are more effective than the single best-matched prompt in collaboratively capturing local and glocal abnormality, as the averaging operation ensures better compatibility with both global (from image token embedding) and local (from patch token embeddings) information.

We would like to kindly point out that using the average of the global (image-level $s_a(x)$) and local (patch-level $s_a(x)$) anomaly scores to define holistic anomaly scores for images is a commonly-used, effective strategy, which has been demonstrated in a number of recent zero- and few-shot anomaly detection methods (e.g., WinCLIP [1], AnomalyCLIP [2], April-GAN [3], InCTRL [4], and PromptAD [5]). Our image-level anomaly scoring follows this approach and verifies its effectiveness in our $`FAPrompt`$.

It is also very clear that $`FAPrompt`-\text{PI}$ is consistently the best performer compared to all the variants of $^{\ast}`FAPrompt`$. These comprehensive ablation study results well justify the effectiveness of $`FAPrompt`$ using the averaged abnormality prompt (i.e., the prototype) in capturing the diverse complementary abnormalities