Advancing Out-of-Distribution Detection via Local Neuroplasticity

We thank the reviewer for their detailed feedback and constructive criticism. We appreciate the acknowledgment of the originality, performance, and exhaustive experimentation of our method. Below, we address the concerns raised in the review:

Q1: No experiments demonstrate the method's scalability to larger images or real-world problems.
A1: We acknowledge the absence of large-scale experiments in our initial submission. To address this, we have conducted additional experiments on the ImageNet-200 dataset, which is part of the OpenOOD benchmarks and contains five times more images that are seven times larger compared to the CIFAR benchmark. We specifically considered the full-spectrum version of the benchmark as it makes the detection problem more challenging and closer to real-world situations by adding various covariate-shifted samples to the InD test set. As shown in Table 2 of Section 3.2 of our revised manuscript, the KAN detector ranks first, surpassing the previous best method by approximately 4%.

Q2: The partitioning problem of KANs is very severe.
A2: As suggested by Reviewer Asxo, we incorporated additional experiments on regression datasets where no classes exist. We show that even when classes are not available, the partitioning method still performs well (see Appendix A.2). We hope that this experiment, along with the tests on the large-scale ImageNet-200, demonstrates that the partitioning method is an essential component of our detector and that it effectively works in various scenarios.

Q3: The influence of model capacity is unclear.
A3: The model size in our case is controlled by three factors: the input size, the output size, and the grid size. The first two are generally dictated by the problem itself. Note, however, that the input to our detector is not the raw image but the latent features space of the backbone. Typically, the latent space has a size drastically smaller compared to the input image, ensuring that larger images do not compromise the scalability of our detector. The effect on performance given by the grid size, and thus the model size, is shown in Table 8 of our manuscript. We also added this explanation to the revised manuscript in Appendix A.11.

Q4: Line 43 has an incorrect citation.
A4: Thank you for pointing this out. We have corrected the citation in Line 43.

Q5: Clarification on how the hyperparameter search is conducted.
A5: We tried to keep the search space quite large to ensure that we capture the optimal values. For the parameters related to the KAN (i.e. grid size, learning rate, epochs) we used similar ranges to what described in the examples of the original KAN paper. We have added the ranges considered for each parameter in Appendix A.12.

Thank you once again for your valuable feedback. We have carefully addressed all the comments and made revisions based on your suggestions, which we believe have greatly enhanced the quality and clarity of our paper.

Thank you for your thorough review and constructive feedback on our submission. We appreciate your positive remarks regarding the relevance, novelty, and motivation of our work, as well as your recognition of the extensive set of experiments we conducted. Below, we address the specific questions and concerns you raised:

Q1: Univariate Nature of KANs
A1: We acknowledge that KANs, being inherently univariate, might seem to limit their applicability. However, our dataset partitioning strategy—whether by class labels or clustering methods such as k-means—enables us to divide complex and correlated feature distributions into smaller ones that can be well-approximated using only the marginal feature distribution. Consequently, the KAN detector can effectively process these partitions. We have clarified this point in Appendix A.7 together with new experiments and discussions on alternative clustering techniques. As suggested by another reviewer, we also tested the partitioning method of our detector in regression-based datasets where no classes exist and showed that the partitioning method still performs well (see Appendix A.2).

Q2: Scalability to Large Datasets
A2: We demonstrate that our method effectively handles large-scale datasets with a new experiment on the ImageNet-200 benchmark which contains five times more images that are seven times larger compared to the CIFAR benchmark. Here our method outperforms all previous baselines (see Table 2 of Section 3.2 for more details). We have also revised Appendix A.11 to include a discussion on the method's complexity and scalability.

Q3: Recent Usage of KANs
A3: We agree that KANs are relatively new and not yet widely adopted. Our focus was on using the KAN detector as a post-hoc method. This means that it can be applied to any existing backbone (e.g., CNN or Transformer) without influencing the classification output of the backbone itself. We show that this works for small- and large-scale datasets, image and tabular data, and different backbone models.

Q4: Training of Different KANs
A4: We apologize for the lack of clarity regarding the training process of different KANs. Each KAN is initialized identically, with the only difference being the data subset (partition) on which they are trained. The training task is the same for all models, and in our case, we used the same loss function as the backbone. We have clarified this point in lines 191-192 of our revised manuscript.

Q5: Integration with Pre-trained Models
A5: To clarify, our approach (like other post-hoc methods) does not replace pre-trained models but rather complements them. The pre-trained model, such as ResNet-18, is used for feature extraction. These features are then processed by our KAN-based detector (or any other considered post-hoc technique) in a subsequent phase. These backbone models do not have to be based on KANs; they can follow any architecture, such as fully connected MLPs, ResNets, or Transformers.

Q6: Use of Pre-trained Backbones
A6: The primary job of the backbones is to perform the classification or regression task. The OOD detector is applied afterward to detect semantically different samples (e.g., samples that do not belong to the training classes), which would yield incorrect predictions by the backbone. From the detector's perspective, the backbone's job is simply to provide the latent features. We clarified this in the revised manuscript at lines 266-268.

Q7: Generalization of Hyperparameters
A7: The validation set contains both InD and OOD samples. However, the OOD samples encountered at test time are of a different type as they belong to different datasets and hence classes. For instance, on the CIFAR-10 benchmark, the validation set includes only OOD samples of the "near" type (CIFAR-100 and TIN datasets) while the test set contains also four datasets with "far" OOD samples. Our method performs well on both categories indicating that the selected hyperparameters generalize well even when new OOD type are encountered.

We greatly appreciate the insightful feedback provided. We have implemented the recommended changes and believe these revisions have substantially improved the paper's quality and clarity.

We sincerely appreciate the time and effort invested in providing valuable feedback on our submission. We are pleased that our contributions and thorough experiments have been acknowledged. Below, we address the specific points raised:

Q1: Method structure and clarity (Eqn 5):
A1: We acknowledge the need for further clarity regarding the method structure. Intuitevely, many methods define the boundary surrounding the InD based on the training samples and classify samples at inference time based on their distance to this boundary. In our approach, the InD boundary is encapsulated within the spline trainable coefficients, while regions activated by InD samples are utilized to determine the distance from the boundary through the aggregation of the delta matrix. we clarified this in line 136-137 of the revised manuscript.

Q2: Generalization performance with KANs:
A2: We agree that integrating KANs in the training scheme can influence the final performance. We have included new experiments in Appendix A.2 to illustrate that our detector can be applied directly to the data features without an additional backbone model. However, it is important to emphasize that our method functions as a post-hoc processor, applied to a pre-trained backbone. Consequently, training our detector does not impact the backbone model as it function as a separate block. The advantage of post-hoc methods is their ease of integration with different backbones without requiring additional training, even in scenarios where no feature-extractor backbone is available.

Q3: Results on CIFAR-100:
A3: Although the improvements on CIFAR-100 are minimal, it is noteworthy that our method is either the best or statistically similar to the best-performing method across a wide range of benchmarks on the overall average AUROC metric. This consistency is not observed in other approaches. For instance, while MDS performs well on medical benchmarks, it shows poor performance on CIFAR-10 and CIFAR-100. Conversely KNN excels on CIFAR benchmarks but underperforms on the medical datasets. To highlight this aspect even more, we conducted additional experiments on the ImageNet-200 benchmark (Table 2 of Section 3.2) where the KAN outperforms the previous best method by approximately 4%. This highlights the robustness and versatility of our method.

Q4: Computational cost:
A4: We have extended Appendix A.11 to include not only inference time but also the setup time (which includes extracting the latent features from the backbone, the partitioning method and the trainings of the KANs). We reported the results as a function of the dataset size and the number fo partitions. The results shows that the setup time of our detector scales linearly with the dataset size, in-line with other methods.

Thank you for your thoughtful comments. We have revised the manuscript in response to your suggestions, leading to significant enhancements in both quality and clarity.

We sincerely thank the reviewer for the detailed feedback and the recognition of our method's clarity and reproducibility. We greatly appreciate the constructive criticisms and suggestions, which have been instrumental in refining our work. Below, we address each of the major and minor concerns raised.

Q1.1/Q2: Testing on other large-scale datasets and scalability and computational burden:
A1.1/A2: We understand your concerns regarding scalability, which were also raised by other reviewers. To address this, we included experiments on the OpenOOD ImageNet-200 leaderboard, where our method ranks first with an overall average AUROC approximately 4% higher than the previously best performing method (detailed results are available in Table 2 of Section 3.2 in the revised manuscript). Due to time constraints, we could not include all OpenOOD leaderboards and other suggested benchmarks. However, we believe that the ImageNet-200, with five times more samples than the CIFAR leaderboards and images seven times larger, effectively demonstrates our method's scalability. To further eliminate any selection bias, we opted for the full-spectrum version of the ImageNet-200 benchmark, which includes covariate-shifted InD samples, making the detection challenge more complex and closer to real-world applications. We also expanded Appendix A.11 with a detailed discussion on the method's complexity showing that the most impactful factor is the dataset size and that our method has a similar scaling law to other approaches such as KNN.

Q1.2: Choice of K-means clustering:
A1.2: The choice of K-means was motivated by its simplicity and low computational overhead. Based on this review, we conducted additional experiments testing several alternative clustering methods and found that the choice of clustering method does not significantly affect detection performance. These results are now included in the revised manuscript in Appendix A.7.

Q3: Regression task example:
A3: To demonstrate that our method performs well on regression tasks, we tested it on the California Housing and Wine Quality datasets and showed that our method outperfoms the KNN detector on both of them. To further validate that the partitioning method is a core component of our detector and not merely a performance-improvement trick, we also used the Friedman synthetic dataset. The results show that the partitioning method is effective even for regression tasks with complexly correlated input features. These experiments have been added to Appendix A.2 in the revised manuscript.

Q4: Backbone dependency:
A4: As suggested, we performed OOD detection on datasets without using a backbone or any other feature-extraction method. We used the same three datasets mentioned in the above answer A3 (i.e., California Housing, Wine Quality, and Friedman synthetic dataset). Our method showed superior performance compared to the KNN baseline on all three datasets, proving its applicability even in the absence of a backbone. In Appendix A.2, we also clarified that our method does not require any additional information from the backbone other than the features, unlike NAC, which requires the gradient of the backbone network.

Q5: Support for the statement on InD dataset size:
A5: To better support our claim, we repeated the same experiment on the CIFAR-100 benchmark, and the results show a similar conclusion (see Table 6 in the revised manuscript).

Q6: InD score negative values (line 183):
A6: It is correct that the InD score cannot be negative. Here the range [-1, 1] here refers to the support of the input space $x$.

Q7: Clarification on dataset splitting (line 187):
A7: When a positive class is OOD for a KAN trained on samples of a negative class, the InD score will be low for that KAN. If another KAN is trained on the positive class, the maximization procedure will flag this sample as InD, as this second KAN will return a high InD score. If the negative class is actually OOD, none of the KANs will return a high InD score, and the maximization procedure will correctly flag the sample as OOD. We clarified this relationship with the InD score at lines 196-197 in the revised manuscript.

Q8: Robustness for NaN-Enriched Data:
A8: We thank the reviewer for raising this interesting point. With modifications to the KAN grid, it should be possible to handle NaN values. We hypothesize that assigning an individual spline coefficient to handle NaN values should suffice. However, since there is no distance relation between NaN and other spline coefficients, the smoothing operations of splines around NaN will be affected.

We sincerely appreciate your detailed feedback. Your suggestions have been very valuable in refining our paper, and we believe that the added experiments and improvements in clarity enhanced the overall contribution of our work.

We further evaluated our method on the challenging ImageNet-1k (full-spectrum) benchmark.
Remarkably, our method ranks first, outperforming the previous best approach (NAC) by approximately 2% in the overall average AUROC metric. Detailed results can be found in the table below.

This benchmark includes over 1.2 million samples and utilizes a larger backbone network (ResNet-50). The results further validate our method's capability to handle complex, large-scale problems.
We currently cannot update our manuscript, but if given the opportunity, we will include these results in the camera-ready version of our paper.