NECO: NEural Collapse Based Out-of-distribution detection

We appreciate your thoughtful review and constructive feedback. We have organized your concerns into seven categories and will address each of them:

Clarity in Writing: Thank you for your feedback. We employed the official notations as per the ICLR template. However, to enhance clarity, we will introduce these notations in the supplementary material. If there are other clarity issues please warn us. We have replaced the word operators by operation in section D.

Operator meaning: It is true that the operator $Avg_{c,c'}$ can be ambiguous. In our case, this operator computes the equation inside the $\left|...\right|$ over all possible combinations of the couple $(c , c’)$, where $c,c’ \\in \\{ 1 \\ldots C\\}$, and then averages the result on these cases.

Additional Experiments on ResNet-50: Thank you for your suggestion. We have conducted the necessary experiments using ResNet-50 and present the results in the table below. However, it's worth noting that the effectiveness of certain techniques may vary when applied to transformers.

Isolating Equation (8) Performance: We conducted experiments specifically using only Equation 8 across various architectures and datasets. Here are the results obtained when the MaxLogit is not utilized:

Understanding Equation 6:
I confirm to you that the denominator is bounded. According the NC1 property that guarantees that all ID class clusters must be far away from each other i.e maximal inter-class variance guarantees that $||\mu_c||_2>\epsilon$. As for the numerator, if the denominator is a finite value, the numerator converges to zero.

Dear Reviewer,

Thank you for responding to our comments, which have significantly contributed to the enhancement of our manuscript. Kindly find our responses to your questions below:

1.A We empirically observed that the performances are stable across different values of $d$, by performing an exhaustive evaluation for all d \in \]1, D\]. In the case of the evaluation phase, we fixed $d$ by selecting the first $d$ principal components that explain at least 90% of the ID variance from the train dataset. In the paper, we updated Figure C4 and Figure C5 with the explained variance curve. 1.B Thank you for noticing the issue on Figure C5.b and Figure C6.b, we have updated the figures in the final manuscript. Now, the behavior is coherent with respect to other architectures.

2. Please note that we updated the claim, indicating that NECO demonstrates superior performance compared to SOTA methods. Specifically, NECO outperforms existing methods on ResNet-18 and ViT with CIFAR-10/CIFAR-100 as in-distribution datasets, and on ViT and SwinV2 with ImageNet as the in-distribution dataset. Furthermore, our performance on the openOOD benchmark is at state-of-the-art with ViT and SwinV2, achieving top performances in AUPR and FPR, while for DeiT, we exhibit SOTA performances in FPR. The only exception is ResNet-50 with ImageNet, where we rank second or third in performance. We explain our poor performance on ResNet-50 by the fact that ViT is more overparameterized than ResNet-50, a property that favors Neural collapse. This overparameterization gap is further widened by the fact that ResNet models have a sparse latent space, i.e. many dead neurons [1] which can be tolerated for ID classification whilst impacting neural collapse. Empirically ResNets can be pruned up to 60% while preserving its classification performance. An additional reason that can be explored is the training procedure of both models.

[1] Ding, X., Hao, T., Liu, J., Han, J., Guo, Y., & Ding, G. (2020). Lossless cnn channel pruning via gradient resetting and convolutional re-parameterization. arXiv preprint arXiv:2007.03260, 3.

Dear Reviewer Qb8y,

As the discussion period concludes today, we would like to inquire about your satisfaction with our responses. Given that we addressed all the weaknesses and questions, would you be open to reconsidering and possibly adjusting the score?

Thank you for your time and consideration.

We thank you for taking your time to review our work and give us helpful comments. We have summarize you concerns into 3 categories and will answer to all of them.

NECO vs. ViM Score:

Our primary distinction from ViM lies in the utilization of the principal space, specifically the space defining the Simplex ETF structure, rather than the null space. Additionally, our scoring function is simpler, eliminating the need to compute training statistics for rescaling. It's worth noting that the null space is unaffected by the Neural Collapse (NC) phenomenon, and there's no assurance that more complex models, ones demonstrating further collapse, will yield higher values for Out-of-Distribution (OOD) samples in the null space to enhance separation. However, in our case, both experimental and theoretical justifications support the claim that using more collapsed models guarantees improved performance.

Lack of Baseline Comparisons:

Thank you for your suggestion. We will incorporate [1] into the manuscript in the coming days.

Absence of NuSA in Baseline: In response to your recommendation and the insights from Review of Ek3F, we have now implemented NuSA using the same dimension as in ViM. The corresponding results are available in the table below, and we will incorporate these results into the paper.

Dear Reviewer Ek3F,

As the discussion period concludes today, we would like to inquire about your satisfaction with our responses. Given that you have not identified any specific weaknesses, would you be open to reconsidering and possibly adjusting the score?

Thank you for your time and consideration.

Thank you for dedicating time to review our work and providing valuable comments. We have categorized your concerns into nine points and will address each of them.

Lack of Discussion on Neural Network Collapse Conditions:

We acknowledge this concern and plan to include a discussion on the conditions that lead to Neural Collapse (NC) in deep neural networks (DNNs). We recognize the need for a comprehensive exploration of NC conditions to enhance the understanding of our approach.

Limited Analysis of Anomaly Detection Metrics:

We applied classic anomaly detection and OOD detection metrics. In response to the reviewer's suggestion, we will expand the discussion in Section 5 and include additional details in the supplementary section to describe these metrics more comprehensively. Regarding sensitivity analysis, we have already investigated the metric's behavior with changes in PCA dimensionality, as outlined in Section C.5.

If you have particular experiments in mind for further evaluation, please provide details. Note that we currently have a single parameter, the PCA dimension, and we have already examined its sensitivity.

Absence of Interpretability in Anomaly Detection Metrics:

We're uncertain about the question's intent. If you're inquiring why we haven't delved into the interpretability of anomaly detection, it's important to note that it's not the primary focus of our method. Our objective is to present an OOD detection algorithm and elucidate its effectiveness. Our approach calculates the relative norm of the data within the Simplex ETF structure to derive the OOD-ness score. Therefore, a misclassification of OOD sample indicates that the data sample shares some similarity with some ID classes.

Request for Visual Explanations and pseudo code:

We are uncertain about the specific requirements, as we have already included figures 1, 2, D7, D9, D10, D11, D12, D13. Therefore, it's not entirely clear what the reviewer is requesting. If the suggestion is to provide visual explanations for the definitions in Section 3, we will incorporate them accordingly. We will also add the pseudo code in the supplementary section C.2.

Clarity on Out-of-Distribution Neural Collapse: Our primary contribution lies in demonstrating experimental results, specifically illustrating NC1 to NC4, as shown in section 4 and section D. We have conducted a qualitative evaluation similar to [1], presenting the results in figures 1, 2, D7, D9, D10, D11, D12, D13. If additional evaluations are needed, please feel free to request them.

Rationale Behind NECO(x) Definition:

The motivation behind NECO is interesting. We observed that performing PCA on the latent space of the prelogit of pretrained DNN reveals the emergence of clusters in both the ID and OOD data when the dimension is reduced. Remarkably, the OOD cluster consistently appears centered around the null vector, as depicted in figures 2 and D13. Consequently, NECO(x) is small for OOD data. This phenomenon is consistent across different architectures. While we recognized a potential link to NC, we acknowledge that current literature in NC does not explain OOD, prompting us to propose a new NC property on OOD scenarios (NC5).

Dear Reviewer 1ZaR,

As the discussion period concludes today, we would like to inquire about your satisfaction with our responses. Given that you have not identified any specific weaknesses, would you be open to reconsidering and possibly adjusting the score?

Thank you for your time and consideration.

We appreciate your thorough review and the valuable feedback provided. We've categorized your concerns into four areas and addressed each of them:

Clarity Concerns:

We acknowledge the need for clarity improvements in the manuscript. A proofread is scheduled to rectify misleading formulations and correct any typos by next Friday.

Performance Variability:

The performance of out-of-distribution (OOD) detection is intricately tied to both the specific OOD dataset used and the underlying architecture, a characteristic common to all techniques.

Take, for instance, the ImageNet-O dataset, which consists of adversarially filtered images specifically designed to challenge OOD detectors.

The adversarial selection for this dataset was based on ResNet-50, explaining why ViT performs significantly better in this case compared to DeiT, which employs ResNet as a teacher, and Swin, which emulates ResNets in its architecture.

In essence, ViT demonstrates superior performance on average, and this can be attributed to its tendency to exhibit more collapse due to its classical style of training. This is in contrast to other transformer models that modify their training procedures in diverse way: DeIT utilizes a distillation strategy while SwinV2 utilizes a shifted window mechanism.

Exploring the impact of training and pre-training on the neural collapse phenomenon could provide valuable insights and serve to validate our assumptions. This requires a specific experimental setting, hence we suggest exploring this topic in future works.

Activation Point of NECO:

All models utilized in our experiments were trained until convergence, without any over-training, to ensure a fair comparison across all methods, including ViM and NuSA. While over-training has the potential to enhance neural collapse (NC), the observed improvement is marginal, as evident in figures 1, D.7-D.12. It's important to note that the models depicted in the CIFAR10 figures are not the ones used for testing; additional steps were added for figure generation to illustrate clear convergence.

Enhancing NC metrics is likely to benefit out-of-distribution (OOD) detection, as it simplifies the model representation. This improvement would consequently enhance all methods leveraging NC properties, even indirectly. Self-supervised methods, such as those used in Foundation models or transformers, present promising avenues to increase the NC, as it inherently aims at clustering the data in the latent space.

Regarding ViM/NuSa, while increased collapse might enhance performance, there are no guarantees, as components outside the ETF structure are not governed by the NC phenomenon. These components may or may not exhibit larger values for OOD.

Linearity Assumption:

Despite the inherent non-linearity of DNNs, the occurrence of the NC phenomenon simplifies this property, particularly at the penultimate layer/DNN-classifier level, where the ETF structure becomes linear by definition [see Introduction section: Convergence to Simplex ETF (NC2), and the figure D.7.

This structure defines the feature representation and geometry. It's worth noting that alternative techniques like kernel PCA and manifold learning were tested, but no significant improvement in results was observed. If needed we can add a discussion on that.

Dear Reviewer r6r7,

As the discussion period concludes today, we would like to inquire about your satisfaction with our responses. Given that you have not identified any specific weaknesses, would you be open to reconsidering and possibly adjusting the score?

Thank you for your time and consideration.