Language-Assisted Feature Transformation for Anomaly Detection

1. The criteria for selecting text prompts are ambiguous. Some datasets utilize the category names of the samples, while others employ diverse descriptions. These approaches rest on the critical assumption that anomalies are distinctly defined, exemplified by MNIST, where anomalies arise from differences in numerals rather than variations in handwriting styles or colors. Should the actual anomalies diverge from these presuppositions, might the proposed model's performance diminish relative to methods devoid of textual guidance? In other words, could the model forfeit its capacity to detect all possible anomalies?

2. In the MVTec dataset experiment, the author opted not to employ the concise anomaly descriptions provided by the dataset itself for text prompts, instead relying solely on item categories, mirroring the approach of WinCLIP. What rationale informed this decision?

3. The proposed model is an extension of WinCLIP, yet it appears to forgo the anomaly segmentation functionality inherent to WinCLIP. Is this omission attributable to certain design elements that potentially diminish the model's anomaly localization capabilities?

4. Experiments have been conducted on synthetic datasets like MNIST and CelebA by altering the original datasets. While I acknowledge the challenge of selecting appropriate text prompts for real-world datasets such as MVTec, the author should endeavor to incorporate more authentic datasets into their study, such as the VisA dataset utilized in WinCLIP or the medical AD benchmark employed in MVFA [a].

[a] Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images. CVPR 2024.

1. Figure 1 is not particularly intuitive or clear, and it is not explained in the text.

2. As the exact formulation of prompts is absolutely critical for this methodology, it should have more dedicated explanation in the main text of the paper, not relegated almost entirely to the appendix.

3. There are not many baselines, and it would have been more convincing if you compare more baselines with and without LAFT transformations.

4. The range of experiments presented are quite restricted. For example with Coloured MNIST, it appears that only one number-colour combination as the normal set was tried. It would be more proper to conduct multiple experiments with different combinations of attributes and show the average result. The same can be said for the other datasets.

1. While LAFT demonstrates significant improvements in controlled environments, such as the Colored MNIST dataset, its performance gains appear less pronounced when applied to complex real-world datasets. This discrepancy suggests that the model may struggle to maintain robustness across multiple intricate attributes, highlighting the need for further refinement in handling multi-attribute scenarios.
2. The experimental setup lacks comprehensive comparisons, particularly between language-assisted and vision-assisted approaches. For instance, incorporating image guidance by utilizing related reference normal images (e.g., normal digits in various colors) or color-augmentation for kNN baseline could provide valuable insights. A thorough examination of both language-based and vision-based assistance would strengthen the evaluation of LAFT's efficacy.
3. The impact of the number of PCA components, which is the sole hyperparameter in LAFT, is not adequately investigated. Given that this parameter influences the model's performance, it is crucial to explore its effect across different datasets. Specifically, an analysis of whether a larger number of components may be beneficial for more complex datasets would provide valuable insights into optimizing the model’s performance.

• The paper uses the vector difference between two textual descriptions to represent a single attribute and maps this attribute directly to image feature transformation. However, this simplification raises at least three issues:

  • The properties of objects cannot be adequately represented by the difference between two concepts.
  • Real-world attributes are often complex and may involve different colors or textures across various parts of an object.
  • The text embedding space and the image embedding space in CLIP are not perfectly aligned; therefore, vectors derived from the text space may not be directly applicable to the image space.

• To validate the effectiveness of feature transformation, using a CLIP-based classification task would be more suitable than anomaly detection.

• The paper lacks results on anomaly localization, which is crucial for industrial applications.

• The language throughout the paper could be clearer. It is recommended to refer to previous works using proper method names and provide concise descriptions of these methods.

• The axis labels in Figure 3 are inconsistent. How were the attributes 'Number' and 'Color' derived?

• The dataset chosen for experiments, SEMANTIC ANOMALY DETECTION, focuses on distinguishing simple concepts. Why not test the method on widely recognized OOD datasets such as ImageNet-1k and OpenOOD? Industrial anomaly detection would benefit from validation on datasets like VisA and Real-IAD as well.

• The comparison methods included are relatively weak. Why not compare with more recent OOD detection approaches such as NegLabel [1] and ClipN [2]?

• [1] X. Jiang, F. Liu, Z. Fang, H. Chen, T. Liu, F. Zheng, and B. Han, “Negative label guided OOD detection with pretrained vision-language models,” in The Twelfth International Conference on Learning Representations, 2024.
• [2] Hualiang Wang, Yi Li, Huifeng Yao, and Xiaomeng Li. ClipN for zero-shot OOD detection: Teaching CLIP to say no. ICCV, 2023.

If the author can address my concerns, I will consider increasing the score.