POUTA - Produce once, utilize twice for anomaly detection

1. This paper is lacking important references from the anomaly detection literature. The primary claim made by the authors is that they approach anomaly detection at the feature level rather than the conventional image-level distinction to locate anomalies. However, feature-level reconstruction has already been explored extensively in the literature, including UniAD [a] and its subsequent studies. In these works, the approach of abstaining from reconstructing data at the image level and not using a discriminative network has already been applied. This discrepancy conflicts with the authors' description and literature review.
2. The ablation studies are carried out on the relatively straightforward MVTec dataset, where the baseline method already achieves an AUC of 98.4%, rendering the comparisons somewhat inconclusive. A more robust evaluation should be conducted on a more challenging dataset like VisA to assess the contributions of each module effectively. Furthermore, the designs of both FCM and HSG appear to be incremental and lack novelty.
3. This study appears to be outdated, given that there are existing anomaly detection methods that achieve exceptionally high performance on isolated tasks (e.g., 100% AUC on half of the classes in the MVTec dataset). Many recent studies focus on examining how methods perform under a universal model (using a single model for all classes) [a] or in zero-/few-shot settings, such as [b]. It is important to assess how the proposed method performs in these settings as well.
4. The paper suffers from issues in its writing. It proves challenging to grasp the content, even after multiple readings.

[a] You et al. "A Unified Model for Multi-class Anomaly Detection." NeurIPS 2022.

[b] Jeong et al. "WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation." CVPR 2022.

The major concern with this paper lies in the limited novelty. Despite the feature-level inputs, using discriminative network to identify synthetic anomaly is not new to me. Some similar works exist. For example, the overall framework can be considered as an incremental improvement of DRÆM (Zavrtanik et al., 2021a).

Some claims may not be supported well, e.g., "the discriminative network cannot obtain the difference between the original and reconstructed image with image-level information. And the feature-level information contains more accurate information about the anomaly." Considering the information flow in the reconstruction process, abnormal image contains more anomaly information than the features produced by following encoder, and the normal-like decoded image should also discard more anomaly information. That is, ideally, image-level inputs should be more discriminative.

As mentioned above, I wonder if there is any intuitive explanation as to why DRÆM with more discriminative information cannot be optimized as well as POUTA.

1. The writing is a bit unbalanced: some exposition (especially in Sec. 1, Introduction) can be condensed considerably. On the other hand, some portions lack sufficient technical details. For instance,

   a. What is the architecture of the MSS module? I understand the loss functions used to train it, but there seems to be no description of its architecture.

   b. Further, can the authors please provide additional details about the training regimen? I'm assuming that for each MVTec/DAGM category, a separate set of weights is learnt? Is the whole pipeline (reconstructive network + FCM + HSG + MSS) trained jointly? Or is the reconstructive network trained first, and then the remaining modules (FCM + HSG + MSS) trained from the features of the pre-trained autoencoder?

2. Finally, comparison to non-reconstruction-based approaches is missing. These constitute an important class of methods in the visual anomaly detection domain, and there are several such methods that give state-of-the-art results, a couple of which are listed below:

   • Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T. and Gehler, P., 2022. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.
   • Defard, T., Setkov, A., Loesch, A. and Audigier, R., 2021, January. Padim: a patch distribution modeling framework for anomaly detection and localization. In International Conference on Pattern Recognition.