Set Features for Anomaly Detection

• The method is applicable to a narrow case, specifically when the anomalies are represented by an unusual distribution of normal elements. This setting does not apply to detecting anomalies in general. The proposed method should be integrated within a method able to detect other kinds of anomalies.
• There are several methods reporting much better results on MVTec-LOCO (see [1]). The authors claim to achieve state-of-the-art results, but according to [1], this is clearly not the case. Since the method does not surpasses competing methods, its benefits are not well justified from a practical point of view.
• The method is demonstrated for only one bachbone: ResNet. It is not clear how the method generalizes to other backbones.
• It is not clear if the method applies to large scale datasets, since it relies on kNN.
• The inference time of the presented method is not discussed.
• With three lines, the conclusion is too short. The authors did not seem to manage the space too well. The conclusion should be more consistent.
• There are a few typos here and there that should be corrected:
  • "Fig.2. The average of a set of" => "Fig. 2, the average of a set of".

[1] https://paperswithcode.com/sota/anomaly-detection-on-mvtec-loco-ad

(1) The method relies on a strong assumption that individual elements within a sample are normal, but their combination leads to anomalies (i.e., logical anomalies). However, there is no foolproof way to validate this assumption when a query sample is introduced, casting doubt on the practicality of the algorithm.

(2) The evaluation lacks comprehensiveness as it does not include state-of-the-art (SOTA) anomaly detection algorithms like CutPaste, RD4AD, SimpleNet, PaDim, CS-Flow, etc., in its experiment results (e.g. Figure 1). Over the past year, many anomaly detection algorithms have demonstrated promising results on the MVTec-LOCO benchmark. A comparison against these SOTA algorithms would better establish the effectiveness of the proposed method.

(3) Contemporary anomaly detection algorithms typically generate anomaly score maps, pinpointing the location or segment of abnormality within the query data. It remains unclear whether this method is capable of producing such anomaly localization results.

(4) In implementation details, the authors mention that combining multiple crops of a query sample enhances the proposed method's performance. However, there is no mention of the computational overhead introduced by this approach or whether there is a tradeoff between computational cost and algorithm accuracy. Moreover, it would be valuable to know if applying the strategy of using multiple crops to other methods, such as patchcore, yields similar or better results than the proposed method.

(5) The authors claims in the first page that the usually anomaly detection procedure follows the paradigm of detection-by-segmentation. This claim is too strong and not true. The majority of anomaly detection algorithms are based on data reconstruction and embedding similarity quantification.

1. The method is not clearly presented. The notations are unclear; What is f_i[j] in Sec. 3.3? How is f in Eq. (1) exactly obtained?
2. One of the papers main claim that it is the sota on MVTec-Loco, is not valid; the proposed method is outperformed by EfficientAD [2], which is 2 years already outdated. Particularly, the proposed method significantly underperforms in the structural anomaly detection.
3. The method has not been tested on conventional benchmarks MVTec-AD [3] and VisA [4].
4. There is no computation cost analysis of the method in the paper. Is this efficient compared to EfficientAD and PatchCore?
5. Although the usage of histograms can be regarded as new in the anomaly detection task, the method itself is quite a classical one.

[2] Batzner, Kilian, Lars Heckler, and Rebecca König. "Efficientad: Accurate visual anomaly detection at millisecond-level latencies." arXiv preprint arXiv:2303.14535 (2023). [3] Bergmann, Paul, et al. "MVTec AD--A comprehensive real-world dataset for unsupervised anomaly detection." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019. [4] Zou, Yang, et al. "Spot-the-difference self-supervised pre-training for anomaly detection and segmentation." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.

1. The paper should present the algorithm better. Currently, some vital information is only in the Introduction (last para on page 1). That should be moved to Section 3 for clarity.

2. Section 3.1 second paragraph "The typical way ...": While the discussion here refers to only the pooling aspect of deep networks, it overlooks their property of detecting higher-level abstractions. There are multiple feature maps in the layers of deep nets and each feature map detects one type of abstraction. Across multiple feature maps, it can be argued that they learn multiple 'sets' of features and the final task (classification/sequence prediction/anomaly detection) will be a function of the sets of features. Moreover, deepnets can learn the feature map finetuned to the task. In that respect, the proposed algorithm (SINBAD) is limited by the fixed set of histogram projections which cannot be finetuned. For example, such an ability to learn would automatically avoid projections along original axes if they are not discriminative. The point here is that, in theory, there is nothing fundamental about the proposed 'set' based design that a general purpose deepnet cannot do (e.g., for images, ResNet).

3. The procedure for generating the histogram descriptors should be illustrated either with a simple figure or in algorithm format.