Improving classifier decision boundaries using nearest neighbors

The main weaknesses, in my view, are the weak set of experiments and the weak literature review. There is also some arbitrariness in the method regarding the choice of layer and other parameters in the algorithm.

The models used in the experiments have surprisingly weak accuracies. Paper spends considerable space describing the models, the dataset, and the specific details of how they have been trained. However, the reported testing accuracies are quite low. Paper can download standard models pretrained on any of the common datasets with 2 or 3 lines of code. The accuracy of such pretrained models, for example, the ones available on the pytorch library, or Hugging Face, are considerably higher than the ones this paper has trained. For example, the accuracy of models such as ResNet on the CIFAR-10 dataset are above 95%. The accuracy of ViT model on CIFAR-10 is around 98%. However, the accuracy of models used in this paper are about 85%. Am I reading the numbers correctly? I see the accuracy of ResNet on CIFAR10 is about 85% and the accuracy of ResNet on CIFAR-100 is 58%.

Because of this, the results do not provide much insight in my opinion. If authors were proposing some new training method that was compromising the testing accuracy for some other purpose such as adversarial robustness, that might have been understandable. But, the paper emphasizes that the proposed method does not need to make any modifications to the models, and it can be applied to any model. So, I don’t understand why experiments are performed on models with such low accuracies instead of pretrained models that are freely available.

The other shortcoming, in my view, is that the majority of results are on the CIFAR datasets. A full set of experiments on Imagenet would be more insightful.

Choosing the layer seemed a bit arbitrary to me. I think it was not specified how one should choose the layer and how much the performance of the method would change if other layers are chosen. Authors probably have experimented with many layers and they are reporting the layer that was most effective? It would be useful if authors specify the variations of their results among the layers of the networks. It would also be useful to know how exactly authors have chosen those layers. Is that based on looking at the performance of the method on the testing set?

Another alternative is to just pick a specific layer and not change it among the experiments.

It seems hard to understand Figures 6 and 7 as they don’t have axis labels, and they do not seem to be explained clearly enough. Also, Figures 6 and 7 seem to be going over the page limit.

Literature review appears weak to me. There is no mention of literature on misclassification detection methods which directly relates to the paper’s claim that it can detect and reduce the mistakes of the models. Not only that literature needs to be mentioned, the results of this paper needs to be compared to those methods as well. One paper, as an example, from the misclassification detection literature:

-- Zhu, F., Cheng, Z., Zhang, X.Y. and Liu, C.L., 2023. OpenMix: Exploring Outlier Samples for Misclassification Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12074-12083).

Another paper that reviews that literature:

-- Jaeger, P.F., Lüth, C.T., Klein, L. and Bungert, T.J., 2022, September. A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification. In The Eleventh International Conference on Learning Representations.

The references used in the paper for comparison are not recent and the paper misses the most recent methods published in the last few years. For example, in table 1, most of the references are from 2019 and 2020. These references may need to be updated.

Moreover, in table 1, there are no references assigned to adversarial robustness and accuracy. Perhaps this is a typo?

On the topic of adversarial detection, the paper does not use the most recent methods of detection. For example, the paper below uses a self-supervised embedding space and it reports that the detection of adversarial examples are significantly better in that space compared to the last hidden layer of trained image classification models. Authors would need to compare their method to the following method and possibly other competing methods.

-- Moayeri, M. and Feizi, S., 2021. Sample efficient detection and classification of adversarial attacks via self-supervised embeddings. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 7677-7686).

The accuracy gains reported in Table 9 are negligible compared to the accuracy gains reported in the above paper.

If the author's method does not outperform some of the existing methods in the literature, that is okay in my view, because the proposed method is performing several tasks, and on the balance, it might be useful even if it does not outperform all the other methods. However, for each of the claims in the paper, the paper needs to make comparisons with the most recent methods in the literature and demonstrate its capability, even if it is not as good.

On the location of decision boundaries. The discussion around Figure 4 might be insightful, if authors go beyond a hypothetical decision boundary. My understanding from the paper is that Figure 4 is a hypothetical demonstration and not based on an actual model, is that right?

What is demonstrated in figures 3 and 5 do not seem to imply any capability for detecting the mistakes. The mistakes are relatively close to the intersection of classes 5 and 6, however, a rather large portion of correct classifications are also at that same intersection shadowing all the mistakes of the model. When there is such a degree of mixture between correct and incorrect classifications, and the correct classifications are overwhelmingly more in numbers compared to the misclassifications, how can the method be helpful in improving the accuracy?

Paper has missed the recent training methods that directly affect the decision boundaries. In particular, Mixup directly defines the location of decision boundaries in between the training samples. This has shown to have several advantages in adversarial robustness, accuracy, etc.

--Zhang, H., Cisse, M., Dauphin, Y.N. and Lopez-Paz, D., 2018, February. mixup: Beyond Empirical Risk Minimization. In International Conference on Learning Representations.

--Battleday RM, Peterson JC, Griffiths TL. Capturing human categorization of natural images by combining deep networks and cognitive models. Nature Communications. 2020.

Some of the discussions about the complexity of decision boundaries would not make much sense if the training method is mixup.

I agree with the argument that using KNN provides some degree of interpretability. However, comparing testing samples to training samples has been a common approach. I think paper needs to go further than this and demonstrate, clearly and via examples, how that interpretability can be useful.

Writing of the paper is weak, and at times, very ambiguous. For example, at the beginning of section 2, the paper states: “Second, the prediction is a combination of the network output of the sample to predict itself and its NNs, while for classical kNNs only the NNs are used to make a prediction”. I read this sentence a few times, and I don’t really get what it says. It is not clear to me what it means for “the sample to predict itself.”

Moreover, there are typos and grammatical errors.

Figure arrangements need improvement. Figure 4 comes two pages after it is discussed in the text. Figure 4 is discussed before figure 2. The arrangements seem premature and arbitrary.

Please see for further information Questions where I will explain these points in more detail.

1. some claims/statements in the paper are not sufficiently backed up with results or references. (quality)
2. the proposed method should be better discussed with respect to state of the art. (quality + originality)
3. The improvements are minor (as honestly stated by the authors). In general, IMHO, this is not a problem but the authors could have presented the results in a better way. For instance, they could have performed significance test to check if the differences are statistically significant. (quality)
4. Some design choices in the evaluation pipeline are nor properly motivated or justified.

• The overall writing and presentation is poor. In particular, it is difficult to make sense of many figures (e.g., Figs. 2, 3 and 5). Algorithm 1 is also quite messy, while not carrying much information.
• The claims are broad and not well-supported (see Table 1). For instance, the interpretability claim is not accurate nor properly illustrated with experiments. Being able to identify the nearest neighbors and use them to interpret classification of the current sample is a local and quite weak way to interpret the decision, we don’t really know how the decision is made for the current sample. This is far from being explainable AI. Likewise, the robustness claims are not well supported, with no comparison with state-of-the-art methods made for improving robustness (e.g. adversarial training).
• All results are reported on rather old and small backbones, that is ResNet-10 and VGG13, on CIFAR-10 and CIFAR-100. Baseline results are low, that is 85.4%/81.6% on CIFAR-10 with ResNet-10/VGG13, and 57.4%/50.5% on CIFAR-100 with the same backbones. Results with common backbones with are rather at 95.5% for CIFAR-10/ResNet-18 (https://paperswithcode.com/sota/image-classification-on-cifar-10) and 81.1% for CIFAR-100/Wide ResNet (https://paperswithcode.com/sota/image-classification-on-cifar-100). It is easy to show gains when starting with poor baselines, and the improvements are rather thin – less than 1% on CIFAR-10 and around 1-2% for CIFAR-100 (ref Table 6).
• The state-of-the-art is not very well developed, rather throwing ideas and discussing of too many aspects. The idea of using instance-based classification in feature space is not new and has been explored before. As relevant work, I point to Wang and Sabuncu 2022 (https://openreview.net/forum?id=iEq6lhG4O3) in TMLR, which present a much more convincing work around similar ideas (minus the adversarial aspects, which are not well supported in the current paper).