Enhancing Transfer Learning with Flexible Nonparametric Posterior Sampling

Thank you for your constructive and positive comment with an important question.

Q1 Is the NPL posterior robust to model misspecification compared to the regular Bayesian posterior on $\theta$?

Thank you to the reviewer for posing an important and insightful question. You are indeed correct that the parametric posterior and NPL posterior target the same parameter, that is $\theta^*$ which minimizes the KL divergence between the true $F^*$ and the model $F_\theta$ [Walker (2013)].

However, we highlight that NPL elicits a nonparametric prior, which induces a posterior distributions on $\theta^*$ that has superior asymptotic properties to the regular Bayesian posterior when the model is misspecified. Intuitively, the reason for this subpar performance for parametric Bayes is that the posterior is computed assuming there exists $\theta^*$ such that $F_{\theta^*} = F^*$. On the other hand, NPL does not make this assumption (i.e. model misspecification is acknowledged), and updates the posterior distribution $\pi_n(F)$ in a nonparametric fashion. This in turn leads to more robust posterior inferences on $\theta^*$, as well as asymptotically superior predictions. This improvement in prediction is indeed observed practically as well [Fong et al. (2019)].

We now outline what we mean specifically by superior asymptotic posteriors and predictions. Lyddon et al. (2018) show that the Bayesian bootstrap posterior (which has the same limit as NPL) asymptotically has the sandwich covariance matrix, which is known to be robust. On the other hand, the parametric posterior does not obtain this variance asymptotically.

For prediction, we are interested in the posterior predictive density, $p_n(y) = \int f_\theta(y) \pi_n(\theta)$, where $\pi_n$ is either the Bayesian or the NPL posterior. Theorem 1 of Lyddon et al. (2018) shows that asymptotically, the KL divergence between $F^*$ and $P_n$ is smaller for the NPL posterior compared to the Bayesian posterior. This asymptotic improvement is indeed due to the robust sandwich covariance matrix [Muller (2013)].

Thank you again for highlighting this important point. We will add a detailed discussion of how NPL addresses model misspecification in the revision paper. And if there are no additional questions or uncertainties, we kindly request you to contemplate revisiting your evaluation to ensure it accurately reflects the situation.

References

[1] Walker, S. G. (2013). Bayesian inference with misspecified models. Journal of statistical planning and inference, 143(10), 1621-1633.

[2] Lyddon, S. P., Holmes, C. C., & Walker, S. G. (2019). General Bayesian updating and the loss-likelihood bootstrap. Biometrika, 106(2), 465-478.

[3] Lyddon, S., Walker, S., & Holmes, C. C. (2018). Nonparametric learning from Bayesian models with randomized objective functions. Advances in neural information processing systems, 31.

[4] Müller, U. K. (2013). Risk of Bayesian inference in misspecified models, and the sandwich covariance matrix. Econometrica, 81(5), 1805-1849.

[5] Fong, E., Lyddon, S., & Holmes, C. (2019, May). Scalable nonparametric sampling from multimodal posteriors with the posterior bootstrap. In International Conference on Machine Learning (pp. 1952-1962). PMLR.

Your contribution to reviewing our paper is much appreciated. With the discussion period deadline nearing, could you kindly respond to our rebuttals? It would greatly assist us if you could indicate any further inquiries or uncertainties you might have regarding our paper.

Thank you for your constructive and positive comments. We will summarize and respond to the questions below:

W1 Discussion of limitations is lacking.

As you pointed out, the apparent downside of the suggested NPTL approach is the extra training cost. For instance, applying the proposed NPTL algorithm involves additional computational expenses to obtain $f_{\text{probed}}$ through linear probing for the provided downstream data. Nevertheless, the computational cost associated with linear probing is minimal in contrast to training the complete network. We will add a discussion on such limitations regarding computational costs in the next version of the paper.

Q1 (1) Using self-supervised pre-trained models

Thank you for posing an important question about constructing pre-trained models. Since the informative base measure $F_\pi(x, y)$ could be any probability measure representing our initial assumption about $F_0$ based on our prior knowledge of the actual data distribution, we have the flexibility to employ linear probed self-supervised pre-trained models as our model $f_{\text{probed}}. Indeed, our preliminary experimental results on Caltech101, utilizing ResNet-50-SimCLR as the pre-trained model, are as follows: Ensemble + L2SP attains ACC=0.875 and NLL=0.441, whereas NPTL attains ACC=0.879 and NLL=0.410. We believe that integrating the outcomes of self-supervised pre-training will undoubtedly enhance the paper, and we are grateful for constructive feedback. Additional results regarding self-supervised pre-training will be available in the camera-ready version.

Q1 (2) Experiments on semantic segmentation

Understanding semantic segmentation as pixel-level classification would simplify the implementation of the proposed NPTL in semantic segmentation experiments. More precisely, we first obtain $f_{\text{probed}}$ by exclusively training additional modules for semantic segmentation tasks while keeping the fixed (self-supervised) pre-trained backbone network. Subsequently, we proceed to train the entire semantic segmentation model, incorporating the proposed NPTL term.

In this manner, there is no obstacle to the conceptual application of our NPTL methodology to semantic segmentation tasks. We are of the opinion that providing additional experimental results for semantic segmentation, as per your suggestion, will enhance the emphasis on our contribution. Nevertheless, due to practical limitations like restricted computational resources and time constraints, we regretfully acknowledge that we may be unable to provide additional experiments on semantic segmentation during the rebuttal period. Even if the results are not ready by then, we will be committed to incorporating them into the camera-ready version. We earnestly request your understanding and consideration of this matter for your assessment.

Following your constructive suggestion, we expanded our experiments on semantic segmentation using the PASCAL VOC 2012 dataset, building upon [1]. Our backbone model was the pre-trained ResNet-50, while we adopted the DeepLabv3+ model as our complete model. Our experiments, apart from our model, were consistent with those in [1]. Table R.3 presents conclusive evidence that NPTL surpasses other baseline models.

Table R.3. Additional BMA Mean IoU performance of (a) SGLD with Non-Learned Prior, (b) SGLD with Learned Prior Supervised, (c) SGLD with Learned Prior SSL, and (d) NPTL with ResNet-50.

It would greatly assist us if you could indicate any further inquiries or uncertainties you might have regarding our paper.

References

[1] Shwartz-Ziv, Ravid, Micah Goldblum, Hossein Souri, Sanyam Kapoor, Chen Zhu, Yann LeCun, and Andrew G. Wilson. "Pre-train your loss: Easy bayesian transfer learning with informative priors." Advances in Neural Information Processing Systems 35 (2022): 27706-27715.

Thank you for your positive and constructive comments. We'll add our additional experiment results in new version of our paper.

I have read the response and I will keep my score.

We appreciate your dedication to reviewing our paper. However, it seems there might be a confusion with other paper, as we don't have any anonymous references in our paper or reliance on a theory submitted to ICLR 2024 in another submission. Could you please double-check and confirm your observations? Thank you.

As, we have identified what seems to be the accurate review for our paper in this link: https://openreview.net/forum?id=AweVGJeW47&noteId=4ovgy5lYGf.

We write responses based on that review.

Thank you for your constructive and positive comment. We will summarize and respond to the questions below:

W1 How we can get posterior samples from any model that has been pre-trained without a particular prior before. Are there conditions or requirements for pre-training the models?

Thank you for pointing out an important question. As we commented on the question from reviewer izux, there is no restriction to constructing pre-trained models. Since the informative base measure $F_\pi(x, y)$ could be any probability measure representing our initial assumption about $F_0$ based on our prior knowledge of the actual data distribution, we have the flexibility to employ any type of pre-trained models that contain information of generating process of the upstream dataset (MAP solution from supervised tasks or solution from self-supervised).

W2 (1) Is the method really playing a role given huge models with a large number of parameters?

As you pointed out, there is a trend in modern transfer learning to assume large pre-trained models on extensive upstream data. Consequently, it would be essential to assess the scalability of transfer learning approaches. To this end, we conducted experiments employing the sizable ViT-B/16 and RoBERTa-Base models (Specifically, ResNet-50 has approximately 23 million parameters, whereas ViT-B/16 and RoBERTa-Base have 86 million and 123 million parameters, respectively). We believe that our experiments using ViT and RoBERTa adequately validate the scalability of the proposed NPTL methodology.

W2 (2) Is it just because the pre-trained models are doing still well on similar vision tasks?

Your question brings up an intriguing point. Indeed, a well-pretrained model tends to perform effectively across related tasks. Yet, if the prior distribution isn't suitable, it can significantly impact the performance of the Bayesian model averaging (BMA) with posterior samples. To demonstrate this, we showcase the BMA performance using the zero-mean isotropic Gaussian prior, denoted as $\mathcal{N}(0,\sigma^2 * I)$. The empirical results in Table R.1. and Table R.2. distinctly illustrate how an unsuitable prior distribution can markedly degrade performance.

Table R.1. Additional BMA accuracy performance of (a) SGHMC with zero-mean isotropic Gaussian prior and (b) NPTL with ResNet-20x4.

Table R.2. Additional BMA negative log-likelihood performance of (a) SGHMC with zero-mean isotropic Gaussian and (b) NPTL with ResNet-20x4.

Thanks to the authors for the answers to my questions included in the (correct) review. Some short comments to their points to make a follow up message on this rebuttal period.

[W1] --- Ok, I see it now, also after section (2) and using section 2.3 and 2.4 from Fong et al. (2019). It makes sense to me, thank you.

[W2A] --- I agree that the order of parameters considered makes sense given the proposed method and approach for probabilistic transfer learning. To me sounds convincing. Additionally, the sort of models considered is also in line with other recent papers doing transfer learning approaches in similar scenarios like in (Matena and Raffel, NeurIPS 2022). Thanks for clarifying this out.

[W2B] --- Thanks for running out additional experiments to address my question. I see the point of author's response and makes sense to me. I also liked the experiment conducted. It would be definitely interesting to test this sort of impact with pre-trained models in the future. But I also understand it could be out of the paper's current scope.

In general, I am satisfied with authors' contributions and answers in this rebuttal. Therefore, I will raise my score accordingly.

Matena and Raffel. Merging Models with Fisher-Weighted Averaging. NeurIPS 2022

Thank you for your positive and constructive reviews and comments. We will summarize and respond to individual comments below:

W1 How do we get W*?

Your understanding is correct. We specifically fit the last fully-connected layer for the downstream task, ensuring that our $(\phi^\ast, W^\ast)$ retains the essence of the upstream data-generating process in $\phi^\ast$ and transfers the knowledge from the upstream through $W^\ast$.

W2 Initialization and learning rate.

Our proposed posterior sampling method is tailored for transfer learning scenarios, and we initiated the model parameters as $\mathbf{\theta} = (\phi^\ast, W)$, where $\phi^\ast$ represents a pre-trained feature extractor parameter, and $W$ is the randomly initialized task-specific head parameter. The linear-probed head parameter $W^\ast$ is solely utilized in constructing the informative base measure for NPTL (although it could potentially be initialized with $W^\ast$, we assert that using the same random initialization for all methods offers a more equitable basis for experimental comparison).

As you pointed out, when fine-tuning from a pre-trained solution, it is crucial to be cautious about avoiding substantial deviations from it. In our ResNet-50 experiments, we opted for a learning rate of 0.01, smaller than the conventional value used in from-scratch training (e.g. 0.1 in practice). Consequently, we did not encounter situations where the model significantly deviated from the pre-trained initialization. Additionally, when testing larger learning rate values such as 0.1 and 0.03, we observed a significant performance decline in line with your concern, indicating a departure from the pre-trained initialization.

W3 For the SGHMC baseline, how is the prior specified and how does the pre-trained model help the SGHMC method? And how does the pre-trained model help the ensemble baseline?

For the SGHMC baseline, we employed two distinct weight prior distributions as outlined below:

1. L2SP: Gaussian prior with a mean set as the pre-trained weight and a variance of $\sigma^2 * I$, where $I$ represents the identity matrix.
2. PTYL: Gaussian prior with a mean derived from the pre-trained weight, further trained on the upstream dataset (acknowledged as challenging to utilize, as emphasized in the paper). The variance is a non-diagonal matrix $\Sigma$ constructed with empirical covariance using weights gathered during additional training on the upstream dataset.

Considering the context of posterior sampling in the transfer learning scenario, both the ensemble baselines and the SGHMC method adopt the same initialization approach. Specifically, we initialize our method and all the baselines with pre-trained feature extractor parameters $\phi$ and randomly initialized task-specific parameters $W$.

Q1 The zero mean isotropic Gaussian is called a non-informative prior.

Thank you for bringing this confusion to our attention. We will seek more suitable language and make the necessary corrections.

Q2 For the Bayesian inference, one criterion of setting prior can be ensuring posterior concentration. Can the authors comment a bit on how the proposed prior related to those concepts?

Thank you for highlighting the importance of posterior concentration. As NPL is based on the Dirichlet Process, we can leverage standard results. In fact, more can be said than the posterior contraction rate - for example, Theorem 12.2 of Ghosal & Van der Vaart (2017) gives a Bernstein-von Mises theorem for the Dirichlet process.

References

[1] Ghosal, S., & Van der Vaart, A. (2017). Fundamentals of nonparametric Bayesian inference (Vol. 44). Cambridge University Press.

Your contribution to reviewing our paper is much appreciated. With the discussion period deadline nearing, could you kindly respond to our rebuttals? It would greatly assist us if you could indicate any further inquiries or uncertainties you might have regarding our paper.

Thank you for your valuable and positive comments. We will make the correction in the revised version of our paper.