ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning

1. The proposed algorithm seems to be a generalized version of feature-contrastive learning. Previous works like [1] and [2] have shown a close relationship between meta-learning and contrastive learning. Optimization-based models implicitly contain feature information [2], and metric-based prototypes directly encode feature information. In this sense, contrasting models may be inherently similar to contrasting features. The authors should provide more context on how model contrastiveness differs from or extends feature contrastiveness. Clarifying this relationship is important, even if model contrastiveness is a generalization.
2. The paper lacks comparisons with other contrastive-based meta-learning algorithms such as ANIL [1] and MAML with the Zeroing trick [2]. Including these baselines would help position the contribution more clearly.
3. It would be informative to see how the model alignment dynamics evolve during training. Plotting alignment metrics (e.g., d^in, d^in - d^out, d^out) for MAML and ConML could provide insights into the alignment/discrimination process.
4. An ablation study exploring different meta-contrastive objectives (e.g., only d^in or only -d^out) could help gain more insight into the relative importance of alignment vs. discrimination for meta-learning.

• Why is the proposed approach described as “universal”? It seems that it must be adapted to each specific meta-learning method, such as by defining $\psi$. Moreover, it cannot be applied universally to solve any task, but it is evaluated only on in-domain scenarios considering tasks sampled from the same dataset during meta-training and meta-testing.
• Does the projection function $\psi:H→\mathbb{R}^d$ need to satisfy certain properties? A more in depth explanation, also with some mathematical proof, and references, might be useful for the reader.
• Given that the projection function $\psi$ is method-dependent, would it be possible to define a universal function (or neural network) that could be applied across all meta-learning frameworks? This would align better with the learner-agnostic claim.
• In line 107, the authors provide a definition of meta-learning assuming that the meta-learned model generalizes to a new task $\tau'$ sampled from $p(\tau')$, which is different from the task distribution $p(\tau)$ used during meta-training. This is not common in "standard" meta-learning, as one of the assumption is that all tasks (both for meta-train and meta-test) are sampled from the same probability distribution. Do the authors refer here to the cross-domain scenario, instead? Is the proposed approach applicable also in this challenging scenario?
• Is there evidence or experimental support for the claim that $d_{out}$ enhances generalization? All experiments appear to be performed considering the in-domain scenario, which does not fully demonstrate generalization capabilities.
• Do the tests performed with CAML w/ConML consider the cross-domain scenario? Specifically, was meta-training performed on the same dataset combinations as in CAML (ImageNet-1k, Fungi, MSCOCO, and WikiArt) and then evaluated on the datasets in Table 3? If not, the results in Table 3 may not be directly comparable.
• Regarding the experiments in Figure 2.c, could you clarify why MAML’s performance remains unchanged as the number of shots increases? Additionally, why does MAML with ConML and MAML with $d^{in}$​ perform better with fewer shots than with more? I expected the performance to improve with more shots at test time. Also, I recommend conducting experiments with the same number of shots during meta-training and meta-testing.

1. In model space, as in MAML, the weights of the task-specific model may consist of multiple weight matrices, rather than a single weight vector as in regression. How is this complexity addressed
2. Bilevel meta-learning is also an important approach that the authors do not mention
3. In my opinion, MAML and ProtoNet aim to establish a common initialization or feature space across tasks. However, in this paper, the use of d_out disrupts this objective, potentially leading to instability during training
4. In a few-shot learning setting, the task-specific learning model cannot be fully optimized. Therefore, directly using e* as a reference does not seem advisable.

1. Which specific pain points in current meta-learning does task-level contrastive learning address and which challenges in meta-learning can it overcome?

2. What's an efficient way to tune certain hyperparameters (e.g., the weight parameter $\lambda$, sampling strategies, and distance functions)?

3. How effective is ConML on larger architectures, such as deep convolutional networks or transformers?

4. How would this framework perform in experiments on more realistic tasks, like segmentation or detection?

Please see Weaknesses for details.