On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift

We thank the reviewer for their feedback and for the kind comments regarding the strengths of the work.

[W1] Strong assumption that the shared, low-dimensional representation is even learnable across the private and public data.

[Q1] Referencing W1 it might be interesting to look at the naturally learnt representations, when the public data and private data is used independently to train different models. It would help verify the assumption that a common representation space is even learnable. What kind of representations are learnt? What is the difference between the representations learnt in a disjoint way and those learnt by forcing a common latent space?

The reviewer raises an interesting point regarding how realistic the shared low-rank subspace assumption is in practice, for deep models and real data. Answering these questions in the most rigorous way may be an interesting paper in itself; this would require defining a metric to measure similarity between trained models (or containment in a subspace) to distinguish the pretrained representation from the trained-from-scratch representation. Another subtlety is that the private trained-from-scratch representations in our setting are simply unusable, and are likely not representative of the "true" representation of the data—this raises a question of whether we should compare to a private or non-privately trained representation in order to measure containment in a subspace.

That said, as a preliminary exploration, we plotted the eigenspectrum of the feature covariance matrix computed after extracting features of PCam images from the ViT-B-32 pretrained model. This result is attached in the PDF in the global rebuttal.

From these results, we see that the pretrained features are approximately low-rank for the out-of-distribution task PCam, yet a linear probe over these features achieves good (83.5%) performance (Table 1). The fact that the representation still gives good performance when only a linear layer is trained on top suggests that the data does fundamentally lie in or near the low-rank space that is identified by the pretrained model. We would be happy to also provide this analysis for RESISC45 and fMoW in our revision.

• I appreciate the authors' effort to undertake the plotting exercise in the short rebuttal deadline. The outcome on PCam does look reassuring! The inclusion of the plots for eigenspectrum for RESISC45 and fMoW would help round off the paper quite well.

• I further agree that the measurement of distance between two representations is not quite obvious, especially for the high dimensional ones. A very very recent ICML paper (that I don't expect the authors to discuss at all now, but could be interesting for the future) looks into this problem it seems, and does a reasonably good job of providing measures to compare them. [1]

  • Also, as the privately trained representations are in fact not usable is a valid observation. I would personally still err on the side of comparison against the private one to understand what differences arise when using public pre-training, and how the change in representation helps the model learn better.
  • On the other hand, comparing representations against the well performing non-private training ones, could act like the other side of the spectrum?
  • I'm curious to see which side the public-pretraining but privately trained models lie in this hyper-spectrum (if they do even lie somewhere useful), or do they lie in a completely unrelated subspace to either of these in the latent space. Something as simple as representing the latent representations as vectors for some smaller (maybe even toy) datasets and then analyzing them could be a good starting point.
  • But then again, I do agree that this might require rigorous definitions in themselves and is too involved to be discussed in this paper. But I do find this direction fascinating and hope the authors' can follow up in a future version.

• All in all, I've read the comments and feedback provided by the other reviewers', and am happy to recommend acceptance for the submission and keep my score :)

[1] Wayland, Jeremy, Corinna Coupette, and Bastian Rieck. "Mapping the multiverse of latent representations." arXiv preprint arXiv:2402.01514 (2024).

We thank the reviewer for their thoughtful feedback and comments on our work.

The disconnect between the setting for the experiments and the stylized theoretical model is very apparent. It is difficult to say how much of the empirical results from section 4 can be explained by the theoretical results in section 5. [...] It would improve the paper if the authors compared to other potential models, if they exist, or stated that they do not.

Indeed, some other models and theoretical analysis techniques do exist in the non-private meta-learning literature. For example, [d] analyzes the gradient dynamics of model-agnostic meta-learning (MAML), and [e] proposes and analyzes a hierarchical clustering approach that uses multiple different MAML initializations to adapt well to new tasks.

Nevertheless, as evidenced by the works cited in our paper (lines 97-101, 226-227), due to the tractable and simple nature of the linear subspace model, it has attracted more attention than other analyses even in the nonprivate meta-learning literature. As we mention in the common response, this model is particularly well-suited for understanding transfer learning under concept shift. We will include a more thorough discussion of other theoretical models from the non-private setting in our revision.

The authors discuss public data assisted query answering in the background section, are you aware of this more recent work in the area?

Thank you for this reference; we were not aware of this work and will add it to our related work section.

An alternative to linear probing for parameter efficient fine tuning is LoRA, would you expect this method to perform similarly to linear probing in your transfer learning experiments?

This is an interesting direction for future exploration. In our results, linear probing already achieves performance much higher than standard fine-tuning. LoRA would still involve training more parameters than linear probing, though fewer than full fine-tuning. In our experiments, we observe that full private fine-tuning performs worse than private linear probing, and one explanation for this is the larger number of parameters involved in full fine-tuning. Thus we would expect LoRA to be another point on the tradeoff between the degradation from increased dimension vs. the improvement from accuracy.

How does the bound for your two stage algorithm compare to the error you would attain if you simply ignored the public data? It would be useful to say exactly what you gain from the subspace estimation step and if the naive approach would ever be better (like in a situation where the embedding dimension is almost as large as the latent dimension).

We point to the bound of Tripuraneni et al. (stated as Theorem 5.2 in our paper) as a starting point for this discussion. The error $\gamma$ of the subspace estimation algorithm given in that paper grows as $O(\sqrt{dk^2/n_1})$. As the reviewer suggested, this indicates that when the latent dimension is nearly as large as the embedding dimension, the subspace estimation error will outweigh the benefits of the dimension reduction. We would be happy to add a discussion about this in an updated version.

We thank the reviewer for taking the time to read and provide feedback on our work.

The advantage over prior work is mild for linear regression tasks. In linear regression, gradients, input data, or models being constrained to a linear subspace are all roughly the same thing.

We acknowledge that these three settings (gradient, input, or model/task subspaces) lead to similar analyses in the linear regression setting. That said, we focus on the shared subspace of the linear models in public and private data distributions because it cleanly models concept shift and shared representation. Such a model also has not been explicitly stated or studied in prior work on public-to-private transfer.

The paper does not attempt to reconnect the theory back to the original experiments. This explanation does not seem refutable.

We understand the reviewer's concern regarding the disconnect between theory and practice. As we mention in the common response, our theory provides one simple model in which public data can indeed help under significant distribution shift (and one which has been validated to be useful empirically, in the nonprivate setting). That said, there is significant current research interest in understanding whether this "shared representation" model is accurate in practice.

The paper does not quantitatively argue that there is limited overlap between the datasets considered and the base pretraining data [...] Can you verify that there is limited overlap between the benchmark datasets evaluated and the DataComp dataset? I think even some sort of textual overlap is sufficient if not exhaustive.

In our paper, we point to the low zero-shot performance of the base CLIP models on each dataset as a natural proxy for the absence of sufficient training data in the CLIP dataset. (In contrast, for example, CLIP has high zero-shot performance on ImageNet, which is contained in the training data, as well as on ImageNet variants [c].)

For RESISC45 and fMoW, which are remote-sensing datasets, we also refer the reviewer to [b], which states: "We obtain a remote sensing subset of [LAION-2B] by applying a binary classification model to determine whether an image in LAION-2B is a remote sensing image (see details in Appendix A.5). This subset, denoted as LAION-RS, contains 726K remote sensing image-text pairs—only 0.03% of all samples. This shows that web crawling cannot efficiently collect remote sensing image-text pairs at scale."

We would be happy to replicate this analysis for PCam in an updated version of the paper (for Datacomp rather than LAION as LAION data is no longer publicly available). Unfortunately, due to the size and download time of the Datacomp dataset we were not able to perform this analysis during the rebuttal period.

Hello, we would like to thank the reviewer again for their thoughtful feedback. We hope our responses have provided clarity on the concerns regarding the theoretical model and the composition of the datasets. We would like to gently follow up in case there are any further questions so that we have time to respond appropriately before the discussion period ends. Thank you!

We thank the reviewer for their time and feedback on our work.

The theoretical setup may be too simplistic. For instance, the resulting algorithm 1 is different from what was implemented in the experiments.

We understand the reviewers' concerns regarding the stylized theoretical model. We aimed to analyze the simplest model that would capture the main characteristics of public pretraining as well as the key property of concept shift. As we note in the common response, this model is commonly used in the non-private meta-learning literature to model similar experiments in the non-private setting. Nevertheless, we agree that building off our work to explore a more complex model capturing the performance of public-to-private transfer would be an interesting direction of future work.

The experiments only focus on image classification. Additional data modality could be utilized to broaden the interest of these results.

Thanks for this suggestion. We agree that experiments in another modality, such as language, could be interesting future work. Works such as [f, g] have shown that private training from scratch is ineffective for language models, while privately finetuning a public model can reach state-of-the-art (private) performance on standard NLP benchmarks. However, these works have not strictly explored the distribution shift setting. We expect that the benefits of a shared representation would extend across different language domains (due to the shared low-level features of language domains), but verifying this experimentally would be valuable. We will include a discussion of this in our revision.

Hello, we would like to thank the reviewer again for their thoughtful feedback. We hope our responses have provided clarity on the concerns regarding the theoretical model and the experimental evaluation. We would like to gently follow up in case there are any further questions so that we have time to respond appropriately before the discussion period ends. Thank you!