Neural Coherence

• Soundness: The results don't seem convincing, given the complexity of the method.
  • The results seem unreliable: The proposed method makes a number of design choices and approximations, such as approximating trajectory comparisons using only the first two moments of activations, assigning higher weights to layers with more divergent statistics and summing up the stopping timesteps across layers. Given all of these choices, it seems easy for the method to overfit to the limited set of evaluation tasks. To make the conclusions more reliable, I would like to see results on:
    1. More target datasets, such as the ones in [A].
    2. More architectures. Right now there seems to be just one architecture used in 3.1 and one different architecture in 3.2. These two architectures are vastly different, the one in 3.1 uses > 200 hidden layers whereas the one in 3.2 has 4. The method seems to be sensitive to the number of layers, since activation statistics at different layers are considered with different weights, so studying understanding how the performance depends on this parameter seems quite important.
    3. Larger numbers of target samples. Most results are shown over 5 target samples with ablations using up to 20 target samples. While these numbers show that the method seems to perform well when target data is very scarce and unlabeled, a larger range of samples is needed to judge whether this performance advantage translates to perhaps more typical transfer settings with 100s and 1000s of target samples. In particular I would imagine the Target-Val baseline to perform much better in these cases. If this is not the case then the method should be advertised specifically for low-data regimes or in cases where target annotations are not available.
  • The results are not convincing:
    1. In Table 1 and 2, the proposed method (NC) performs 0.6% and 2% better than Source-Val, respectively, in Scenario A and slightly worse in Scenario B. Given that Source-Val is target dataset agnostic, i.e. it can just be computed once to find the best model on a source dataset and then use it on multiple target datasets, the added complexity of per-dataset evaluation of NC seems unjustified, given the at best marginal performance gains.
    2. Table 3 seems to report better results, but I'm not quite sure how to interpret them, see clarity issues below.
• Clarity
  • Motivation and justification for a number of methodological choices are missing, which makes the approach feel rather ad-hoc in many places.
    1. What is the intuition behind expecting the activation statistics (mean and std) to capture alignment between different tasks, especially when they are from different distributions? Why should models have similar activation statistics across these distributions?
    2. You mention that the moments of distributions are in many cases sufficient to characterize them. However, there are a couple of approximations, such as only using the first two moments, aggregating the first two moments of the mean activations and the first moment of the activation std, approximating trajectories linearly. How do these approximations affect how well NC captures the divergence between activation distributions?
    3. In Figure 1 you say that you sort the checkpoints in the trajectories based on performance (I'm assuming source data performance). However, in the "Training data selection" part of Section 2.2 you mention that you use the implicit ordering given by training time to sort checkpoints. The two criteria seem aligned but if there is a conflict, which one would you use?
    4. How did you choose the layer weighting scheme based on divergence described in 2.2?
  • Details for interpreting the results are missing:
    1. Which datasets or dataset combinations does NC select in Section 3.2?
    2. Why do you only train for 10% of the full pre-training steps in Section 3.2? Some of the datasets seem rather modest in size.
    3. In Section 3.1 you show two scenarios, one in which NC breaks down before the end of training (A) and one in which it doesn't until the end of training (B). How do you control for when divergence breaks down? Did you select the datasets based on their behavior in this regard?
    4. How is the Oracle constructed in Table 3?
  • Some notation is not properly defined
    1. What is $t^*_{valid}$ in Eq. 5?
    2. In 2.1 you mention an objective function $d_{NC}$, but do not define it, as far as I can see. Is $d_{NC}$ the same thing as $NC$ in Eq. 4?
  • It would be easier to understand NC if both it's ideal version, as well as the approximated form were presented, not just the approximation.

[A] Salman et al., Do Adversarially Robust ImageNet Models Transfer Better?, NeurIPS 2020

• Many portions of the paper are poorly presented, and implementation details are obscured (See questions) Also, title of the paper does not provide any insight about what the paper is about.

• The method proposed seems to be very closely related to [1], which also proposes the method of computing the same statistics $m_1$ to $m_4$ in the exact same manner, and modeling the trajectories of these variables. While compared against in the experiments, it is not mentioned in the related works nor anywhere else how the proposed method is different from that of [1]. Can the authors elaborate on the novelty/difference of their method compared to [1]?

• Experiments section:

  • Sec 3.1: The method seems to perform on-par with Source-Val especially for experiments on zero-shot generalization and linear probing (beats NC for 50% of the evaluated datasets), which does not require any data from or knowledge of any particular downstream dataset. This also renders Source-Val a more robust method since it generalizes to any unseen downstream task. In contrast, NC makes stronger assumptions regarding knowledge of (few samples) from the target downstream task.
  • Sec 3.2: The authors evaluate on 5-way-1-shot classification tasks. It is not mentioned what value of $n$ is used here, if $n=5$ as per zero-shot generalization experiments, this amounts to using the entire downstream dataset (minus the labels) which seems a strong assumption (compared to Source-Val with zero knowledge regarding the downstream task).
  • No baselines other than Source-Val are compared against in Sec 3.2 experiments, even though there exist several works for pre-trained model selection without labels such as [2]

• Other comments

  • An algorithm pseudocode for the generalization and model selection tasks might be useful for improving the clarity of the paper.
  • In the definition of $z = \phi_l(x; t)$, none of the notations $l$, $x$, and $t$ are introduced at this point (and $t$ is never introduced in the following text either)
  • Typo in: ``... by decreasing $L_S$, the target empirical risk $L_S$ will also decrease"

[1] Guiroy, Simon, et al. "Improving Meta-Learning Generalization with Activation-Based Early-Stopping." 2022

[2] Wallace, Bram, Ziyang Wu, and Bharath Hariharan. "Can we characterize tasks without labels or features?." 2021

Problem Motivation: I actually think the oracle baseline here (e.g., either performing zero-shot or fine-tuning the model on each of the candidate checkpoints on the target dataset) is a pretty reasonable way to solve the task. Especially since you are looking at linear probing, actually computing the linear probe for each of the checkpoints is already very cheap, so its not clear to me why you would need this method. Full-network fine-tuning is maybe a better case (where fine-tuning is expensive) but the authors do not consider this task in their paper. Furthermore, in most cases, if you are about to perform transfer learning, you usually have a target dataset in mind (and have some non-trivial number of examples from that target dataset).

Experiments: In general, full-network fine-tuning is more common than either MAML or linear probing and should be considered in the paper. In particular, why was MAML chosen for 3.2?

Intuition: It's not clear to me (and not explained in the paper) why we should expect the trajectories between source and target to match for good transfer learning models. The authors should explain their intuition more clearly in the paper.

• The main weakness of the paper is that the experimental results are weak. In particular results on zero-shot tasks (Table 1) do not perform much better than the Source-Val baseline. This is consistent with findings in prior works that better ImageNet models transfer better: https://arxiv.org/abs/1805.08974 (i.e. doing well on in-dist validation set is a good measure for selecting checkpoints). In Table 2, this is again apparent on EuroSat, however Neural Coherence is better on Scenario A by 2%. In Figure 3, even though NC performs well with limited data, all curves are still below or similar to source-val. The main advantage of NC for checkpoint selection seems to only be where you have limited target and source data as in this setting, source-val would also drop in performance.

• In checkpoint selection experiments, the authors the authors test different forms of fine-tuning including zero-shot and linear probe. However, the authors do not test full fine-tuning. The authors should conduct this investigation to show NC is consistently better for all benchmark choices of fine-tuning.

• The proposed approach is similar to https://arxiv.org/abs/2104.11408 which has applied a similar measure which they call Neural Mean Discrepancy (NMD) to compare activations and detect OOD data. I encourage the authors to draw comparison with how the proposed method is different as the field of OOD detection is very similar to dataset selection.

Collectively, the method does not appear original in comparison with NMD, experimental results appear weak on checkpoint selection, and there are some outstanding concerns on dataset selection experiments concern small dataset size, and number of training steps.