Hierarchical Gaussian Mixture Normalizing Flows Modeling for Multi-Class Anomaly Detection

[To Q1]. $p_\theta(x) = {\rm e}^{{\rm log}p_\theta(x)}$ is coming from the log-likelihood ${\rm log}p_\theta(x)$ by exponential function. We call it probability as the value of $p_\theta(x)$ is in $(0,1)$. Our statement is: $s(x) = 1 - p_\theta(x)$, $p_\theta(x)$ is a value for representing normality, $s(x)$ is thus a value for representing abnormality.

[To Q2]. It's necessary. With positional embeddings, we can achieve better results. We don't conduct an ablation study for this design, as the effectiveness of positional embeddings is already validated in the previous work CFLOW [1]. In our paper, we also state that the normalizing flow used in our model is the same as the one in CFLOW (please see sec. 4.2, Quantitative Results).

[1] Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In IEEE Winter Conference on Application of Computer Vision, 2022.

[To Q3]. Because we think that parameterization $p(Y)$ can enable the network to adaptively learn the class weights, and parameterization $p(Y)$ only introduces a small number of parameters. The parameter to control $p(Y)$ is $\psi$, and $\psi$ is learned by optimizing the E.q(6). Then $p(y)$ is estimated by ${\rm softmax}_y(\psi)$. No module in Figure 3 is dedicated to estimating $p(Y)$, the parameter $\psi$ belongs to the normalizing flow model.

[To Q4]. We respectfully disagree with this comment. $\mu_{y^\prime}$ means all other class centers except the class center $\mu_y$ corresponding to $y$. Because our method is used for multi-class anomaly detection, there will be many class centers $\{\mu_y\}, y \in \{1,\dots,N\}$. In Eq.(8) and (9), when we employ the softmax function to calculate the value for a special class $y$, all the other classes are naturally represented as $y^\prime$. We think that such notations are commonly used in the softmax function. So, we don't specifically explain $y^\prime$. We will add corresponding explanations to make E.q (8) and (9) easier to understand in the revision.

Loss (9) is necessary. In Tab 3(c), we show that using Entropy is beneficial to achieve better results. And loss (9) is used to optimize entropy during training. The total loss is $\mathcal{L} = \lambda _1\mathcal{L} _g + \lambda _2\mathcal{L} _{mi} + \mathcal{L} _e + \mathcal{L} _{in}$. Our design is to use $\mathcal{L} _e$ and $\mathcal{L} _{in}$ as auxiliary losses. As there are four optimization objectives, adding a weighting factor and conducting ablation studies for each loss item will result in a large number of combinational experiments, which will bring us too much burden.

[To Q5]. In Appendix A, we have clearly explained this statement. The existing AD datasets are collected for one-for-one anomaly detection (i.e., we need to train a model for each class). Thus, the existing AD datasets need to be separated according to classes, with each class as a subdataset. Therefore, one-for-one AD methods also need class labels, as they require normal samples from the same class to train, but they don't explicitly use the class label. Our method can achieve only training a unified model for all classes, this is a significant innovation compared to the one-for-one AD methods. Moreover, our method still follows the same data organization format as the one-for-one AD methods, but we need to explicitly assign a label for each class. This actually doesn't introduce any extra data collection cost. Please see Appendix A.1 for more discussions.

[To Q6]. As we mentioned above, the label is used to distinguish which class each sample belongs to. In a stricter condition, we may hope the multi-class AD models even don't need labels, which means that we have many normal samples from different classes but don't know which class they belong to. Because the AD datasets themselves separate samples according to classes, we can easily get the class label from these datasets. Therefore, we claim that using class labels should not be a strict condition (or constraint) when designing multi-class AD methods. We think that we explain this clearly, this conclusion seems not to require experimental validation.

[To W1]. The red means abnormal and the green means normal. In anomaly detection papers, authors usually use red to represent abnormal and green for normal, we are a bit careless in not clearly indicating the corresponding classes of the two colors in Figure 2. We will add a legend to the figure to indicate the meaning of two colors in the revision.

[To W2]. We are sorry about this confusion. Due to page limitation, we move the overall loss function to the Appendix. This results in the absence of definitions of hyperparameters $\lambda_1$ and $\lambda_2$. We really appreciate that you point out this problem. We will revise the layout of our paper to ensure that such a problem won't appear in our paper.

[To W3]. The Feature Extractor will extract multi-scale feature maps, level-k means the kth feature map level. Using multi-scale features is common in anomaly detection, and our method also builds a normalizing flow model at each feature level. We appreciate your suggestion and will add explanations for level-k in the revision.

Firstly, we will explain the difference between multi-class anomaly detection (AD) and multi-class classification, you may misunderstand our multi-class AD task. Multi-class classification focuses on classifying input samples, while anomaly detection focuses on detecting abnormal regions in input samples. Previous AD methods follow the one-for-one paradigm (i.e., we train one model for each class). In multi-class AD, we only train one unified model for all classes. Thus, $y$ doesn't indicate whether the input is normal or abnormal but indicates what class the input belongs to. Unlike multi-class classification, we do not know whether the input is normal or abnormal, but we can know which class it belongs to. Please also see our responses to W5 and Q5.

[To W4]. In anomaly detection, we only use normal samples for training as our goal is to detect anomalies. In our paper, we have clearly explained that normalizing flow (NF) may have a ”homogeneous mapping'' problem when used for multi-class anomaly detection, where NF may take a bias to map different input features to similar latent features (i.e., this means anomalies will have similar log-likelihoods to normal). Moreover, the goal of training is to maximize the log-likelihoods of normal features. Thus, normalizing flows will represent large likelihoods for both normal and abnormal features. Please see the sec. 3.2 in our paper, we have provided detailed explanations for this.

[To W5]. Firstly, we will indicate the difference between industrial anomaly detection and semantic anomaly detection. Our paper focuses on industrial anomaly detection rather than semantic anomaly detection. In industrial anomaly detection, all classes are normal, the anomalies are defective areas that exist in the image and don't have classes. Only in semantic anomaly detection (e.g., CIFAR10, CIFAR100, and ImageNet-30 are usually the datasets), we will select some classes as normal and the other classes are used as abnormal.

The partition follows the standard way of these industrial AD datasets, MVTecAD, BTAD, MVTec3D-RGB, and VisA. Generally speaking, most industrial AD papers will not specifically elaborate on the data partition, as the training and test sets are fixed.

We have clearly defined the multi-class AD in the second paragraph of Introduction: one unified model is trained with normal samples from multiple classes, and the objective is to detect anomalies in these classes. In training, we only use normal samples from multiple product classes without any anomaly, the label information is used to indicate one normal sample belongs to which class. For the last question, please see our response to W3.

[To W6]. Thanks for your suggestion, we will add the pseudocodes of our algorithm in the revision.

[To W7]. We validate the effectiveness of the intra-class centers in ablation studies (see Tab. 3(a)). The number of intra-class centers is the same for each class (we state this in our paper, please see sec. 4.2, Setup). Figure 3 is just a schematic diagram of our model. Moreover, we can see that the number of intra-class centers in the two classes is the same, but the intra-class distributions are different (this is our purpose of introducing intra-class centers).

[To W1]. We are very grateful for this suggestion, we will polish the abstract to make it more concise in the reavision.

[To W2]. Thanks for your suggestion. We will attempt to use a larger font size to plot the figure for making the coordinate/legend large enough to directly read.

[To W3]. Thanks for your suggestion. E.q(6) is a derived loss (the detailed derivation is in Appendix E), we will include some key intermediate steps for assisting understanding. E.q(9) is actually the entropy formula, where we use $-||\varphi_\theta(x)-\mu_y||^2_2/2$ as class logits. $\mu_{y^\prime}$ means all the other class centers except for $\mu_y$. We don't specifically explain $\mu_{y^\prime}$ as we think this notation is commonly used in the softmax function. We will add more explanations to make E.q(9) easier to understand. For other equations, we will also carefully check and explain these equations more clearly in the revision.

[To Q1]. Thank you for the comment, but we cannot fully agree with this comment. The identical shortcut is essentially caused by the leakage of abnormal information. The process of reconstruction is to remove abnormal information in the input, resulting in the failure of reconstruction in abnormal regions. But if the reconstruction network is overfitted, the abnormal features may be leaked into the output, resulting in the reconstruction network directly returning a copy of the input as output. So, the reconstruction errors in abnormal regions will be small, leading to abnormal missing detection. This issue usually can be addressed by masking, such as the neighbor masking mechanism in UniAD.

Homogeneous mapping is a specific issue in normalizing flow (NF) based AD methods. In previous NF-based AD methods, the latent feature space has a single center. When used for multi-class AD, we need to map different class features to the single latent center, this may cause the model more prone to take a bias to map different input features to similar latent features. Thus, with the bias, the log-likelihoods of abnormal features will become closer to the log-likelihoods of normal features, causing normal misdetection or abnormal missing detection. To address this issue, we propose hierarchical Gaussian Mixture normalizing flow modeling. Because there are significant differences in the causes and solutions of the two issues, we think that the two issues are not intrinsically equal. In Appendix A.2, we have provided a thorough discussion for this question.

[To Q2]. We are very grateful for this comment, it's really helpful for us to improve our paper. We will revise this issue in the revision.

[To Q3]. HGAD is taken from our paper title: Hierarchical Gaussian Mixture Normalizing Flows Modeling for Multi-class Anomaly Detection. As this is too long, we didn't list the full name in the main text (only list “Hierarchical Gaussian mixture” in Abstract). In the revision, we will explain the HGAD naming in the main text.

[To Q4]. In our paper, multi-class and unified actually have the same meaning. In the caption of Tab 1, we use “unified/multi-class” to express that multi-class is an alias for unified. The meaning of results is explained by the following sentence: “$\cdot$/$\cdot$ means the image-level and pixel-level AUROCs''. We are sorry for this misunderstanding. We will revise this misleading caption in the revision.

[To Q5]. In Tab 3(a), the hyperparameters $\lambda_1$ and $\lambda_2$ are set as 1 and 10. However, we later find that setting $\lambda_1$ and $\lambda_2$ to 1 and 100 can get better results (see Tab 3(d)). In Tab 1, we update the better results, but most results in Tab 3 are obtained with the $\lambda_1$ and $\lambda_2$ set as 1 and 10.

[To W1]. In the anomaly detection field, we think that our method should not be seen as supervised. Because anomaly detection focuses on distinguishing anomalies from normals, methods that use abnormal samples during training should be seen as supervised. For example, BGAD is supervised as it uses few-shot anomalies during training. However, our method only uses normal samples during training.

As we have explained in Appendix A.1, our method has the same data organization format as the one-for-one AD methods (e.g., PaDiM, MKD, DRAEM, FastFlow, and CFLOW in our paper) and doesn't introduce any extra data collection cost. The existing AD datasets are collected for one-for-one anomaly detection (i.e., we need to train a model for each class). Thus, the existing AD datasets need to be separated according to classes, with each class as a subdataset. Therefore, one-for-one AD methods also need class labels, as they require normal samples from the same class to train. If the classes are not separated (in other words, without class labels), these one-for-one AD methods cannot be used either. Our method actually has the same supervision as these methods. The difference is that we explicitly use the class labels but they don't explicitly use the class labels. However, our advantage is that we can achieve only training a unified model for all classes, this is a significant innovation compared to the one-for-one AD methods. The unified AD method, PMAD, also explicitly uses class labels when accomplishing multi-class anomaly detection. The UniAD and OmniAL don't use class labels, but as we mentioned above, using class labels doesn't introduce any extra data collection cost. However, our method can achieve better multi-class detection results than UniAD and OmniAL.

In a stricter condition, we may hope the multi-class AD models even don’t need labels, which means that we have many normal samples from different classes but don’t know which class they belong to. Because the AD datasets themselves separate samples according to classes, we can easily get the class labels from these datasets. Therefore, we think that using class labels should not be a strict condition (or constraint) when designing multi-class AD methods.

[To W2]. Thanks for your suggestion. In the revision, we will provide an explicit pseudo-code of our algorithm to more clearly describe the whole algorithm process and the overall optimization object to make readers more easier to follow our paper.

[To W3]. Thanks for this comment, it’s really helpful for us to improve our paper. We will take your suggestions and other reviewers' suggestions seriously to improve the presentation and organization order of our paper. We would be very grateful if you could provide us with more suggestions on further improving our paper.

[To Q1]. As we discussed in W1, BGAD uses few-shot anomalies during training while our method doesn't. The data requirements of our method and BGAD are quite different, so it should be reasonable that we don't compare with BGAD. Moreover, we have compared with the SOTA NF-based AD methods, FastFlow and CFLOW, and this has demonstrated the effectiveness of our method. Our method has the same data organization format as the one-for-one AD methods, such as PaDiM, MKD, DRAEM, FastFlow, and CFLOW. The unified AD method, PMAD, also explicitly uses class labels. From the data requirement perspective, these methods are like our method, but we can achieve multi-class anomaly detection or better multi-class AD results.