Transfer Learning for Diffusion Models

We thank the reviewer for the comments and suggestions. We appreciate the time you spent on the paper. Below, we address your concerns and comments.

Q: The authors could discuss what is the Lemma 3.2 in the conditional version. What is the Lamma 3.2 in the conditional version?

A: Thank you very much for this kind reminder. Yes, the key idea behind Lemma 3.2 is that the conditional expectation is the optimal solution to the least regression problem. It can be easily extended to conditional generation, i.e., $h _{\psi^*}\left(\mathbf{x}_t, y, t\right) = \mathbb{E} _{p\left(\mathbf{x}_0 \mid \mathbf{x}_t, y\right)}\left[q\left(\mathbf{x}_0, y\right)/p\left(\mathbf{x}_0, y\right)\right]$. We give the Lemma and proof here for the sake of completeness, and will add them to the revised version.

Lemma: For a neural network $h _{\boldsymbol{\psi}}\left(\mathbf{x}_t, y, t\right)$ parameterized by $\boldsymbol{\psi}$, define the objective $\mathcal{L} _{\text{guidance}}(\boldsymbol{\psi}) :=\mathbb{E} _{p(\mathbf{x}_0, \mathbf{x}_t, y)}\left[\left\|h _{\boldsymbol{\psi}}\left(\mathbf{x}_t, y, t\right)-\frac{q(\mathbf{x}_0, y)}{p(\mathbf{x}_0, y)}\right\|_2^2\right],$ then its minimizer $\boldsymbol{\psi}^* = \underset{\boldsymbol{\psi}}{\arg \min } \ \mathcal{L} _{\text{guidance}}(\boldsymbol{\psi})$ satisfies: $h _{\boldsymbol{\psi}^*}\left(\mathbf{x}_t, y, t\right)=\mathbb{E} _{p(\mathbf{x}_0 |\mathbf{x}_t, y)}\left[{q(\mathbf{x}_0, y)}/{p(\mathbf{x}_0, y)}\right].$

The proof is straightforward and very similar to the unconditional version. Note that the objective function can be rewritten as

We thank the reviewer for the comments and suggestions. We appreciate the time you spent on the paper. Below we address the concerns and comments that you have provided.

Q: The title "Transfer Learning" is too broad. The paper focuses solely on (few-shot and supervised) domain adaptation, where the upstream and downstream tasks are similar in nature. General transfer learning encompasses a wider range of tasks, including those with different downstream tasks from the upstream one. For instance, transferring pre-trained text-to-image models to controllable generation or text-to-video generation are broader and more complex tasks that fall under transfer learning.

A: Thank you very much for this question. Actually, our proposed framework is general enough to deal with more complex tasks, e.g., fine-tuning text-to-image models or text-to-video generation. These models can be viewed as the conditional generative model with text embedding as the condition.

Q: The paper does not address transfer learning across different label spaces. Even within the same generation tasks, domain adaptation requires identical label spaces for source and target domains, which is not always practical. For example, transferring pre-trained conditional generation models on ImageNet to other fine-grained datasets with different label spaces presents a more challenging problem than the domain adaptation addressed in this paper.

A: Thank you very much for this question. Our framework (conditional guidance) does not assume identical label spaces for source and target domains. In the ECG experiments, the label set of target domain is the subset of the source domain. When the source and target domain contain different class labels, our framework is still applicable, i.e., when $y_t\neq y_s$, $\underbrace{\mathbf{s} _{\boldsymbol{\phi}^*}(\mathbf{x} _t, y _{t}, t)} _{\substack{\text {target source}}} = \underbrace{\nabla _{\mathbf{x} _t} \log p_t(\mathbf{x}_t | y_s)} _{\text {pre-trained conditional model on source}}+ \underbrace{\nabla _{\mathbf{x}_t} \log \mathbb{E} _{p(\mathbf{x}_0|\mathbf{x}_t, y_s)}\left[\frac{q(\mathbf{x}_0, y_t)}{p(\mathbf{x}_0, y_s)}\right]} _{\text {conditional guidance}}.$ To generate an unseen class $y_t$, the key problem here is to choose a particular class from the source domain $y_s$ such that we can borrow more information from the source domain to generate this unseen class from the target domain. The coupling between $y_t$ and $y_s$ can be learned by solving a static optimal transport problem.

We show our framework is general enough to deal with homogeneous transfer learning together with heterogeneous transfer learning. More in-depth design (e.g. coupling solved by static optimal transport) can be left to future work.

Q: More experiments on various modalities and datasets are necessary to showcase the generalization ability of the proposed method.

A: Thank you very much for this suggestion. We refer to the "general" response and Table 1 in the attached file.

Q: While several ablation studies on simulations are included in Appendix C.2, they are not comprehensive.

A: Thank you very much for this suggestion. We refer to the “global” response and Figure 1 in the attached file.

Q: The consistency regularization requires optimization of the gradient term $\nabla _{\mathbf{x}_t} \log h _{\boldsymbol{\psi}}\left(\mathbf{x}_t, t\right)$. Could the authors elaborate on the optimization process for this term?

A: Thank you very much for this question. Optimization of the gradient term is widely used by the meta-learning method. There is only one-line difference in practice to implement it, i.e., "torch.autograd.grad($\log h_{\boldsymbol{\psi}}\left(\mathbf{x}_t, t\right)$, $\mathbf{x}_t$, retain_graph=True)" to retain the computational graph for the backpropagation of the weights of guidance network $\boldsymbol{\psi}$, which can be found at line 249 in density_ratio_guidance.py. We will add additional clarifications to the revised version.

Q: Effectiveness of TGDP.

A: Thank you very much for this question. Both of the two claims are correct. (The training of the domain classifier is not significantly affected by the number of samples in the target domain since the domain classifier can achieve more than 90% accuracy learning from only 10 samples in the target domain. And the training of the guidance network with $\mathcal{L}_{\text{guidance}}$ does not require target data.) The key problems are: 1) The density ratio estimator may not be accurate in some regions since the coverage will not be large enough when the number of training samples is 10; 2) The $\mathcal{L} _{\text{cycle}}$ and $\mathcal{L} _{\text{consistence}}$ depends on the number of samples in the target domain.

Q: Explanation of Figure 3.

A: Thank you very much for your question. From Table 2, we demonstrate improvements in the diversity and fidelity of TGDP against two baseline methods. For each sub-figure in Fig 3, we get the embedding for T-SNE by using the same encoder from learning directly on target samples. If we change the encoder to the learned classifier by each method, we may see a big difference between Finetune Generator and TGDP.

Thank you for your response. However, some of my concerns still remainsunsolved.

• Q1: The title "Transfer Learning" is too broad. One typical advantage of fine-tuning and other transfer learning methods (compared with score shaping) is to adapt to a new downstream task. For example, stable-diffusion is a text-to-image generation model, and [1-2] transfer it to controllable generation and text-to-video generation, respectively. Could the authors provide further details about how TGDP handles these tasks?
• Q7: Explanation of Figure 3. Why does using the same encoder learn on target samples lead to similar T-SNE visualizations, although the diversity and fidelity are improved? Are there any results following "change the encoder to the learned classifier"?

[1] Adding Conditional Control to Text-to-Image Diffusion Models, ICCV 2023
[2] Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models, CVPR 2023

Thanks for your response and my concerns are addressed. I appreciate the contribution of TGDP in motivation novelty and theoretical soundness, but yet expect the scope to be more clear (since the title is Transfer Learning), and believe more discussions about TGDP on other fine-tuning scenarios could make the paper better. Above all, I raise my score to 5, i.e., a borderline score.

Hello authors of the paper,

It would be great if you could respond to reviewer UUwS's most recent comments before the author/reviewer discussion period closes (at Aug 13, 11:59pm AoE).

Thanks, Your AC

We thank the reviewer for the comments and suggestions. We appreciate the time you spent on the paper. Below we address the concerns and comments that you have provided.

Q: The first experiment on a 2D Gaussian illustrates the performance of the method in comparison to baselines decently, but it is in my opinion ultimately too trivial to be really meaningful (since the density ratios of the 2 Gaussian mixtures can be easily learned, because they are well separated, I would expect that a method based on learning density ratios outperforms baselines here).

A: Thank you very much for this question. We agree that the two-dimensional Gaussian mixture setup is a relatively easy task and serves as a proof-of-concept numerical example to validate the proposed method. To further demonstrate the effectiveness of our method, we have conducted experiments on real-world ECG datasets, and on image datasets in the "general" response above and Table 1 in the attached file therein. Additionally, we would like to highlight a challenge associated with the two-dimensional Gaussian example: when two distributions differ significantly, the density ratio estimator struggles to accurately estimate the density ratio. Due to the well-separated nature of the distributions, the density ratio at some points can be as large as $10^5$. One key takeaway from our work is that our method is robust to the magnitude of the estimated density ratio.

Q: The ablation in Figure~4 in the appendix is conducted too superficially. It consists of scatterplots, no real quantitative analysis, and doesn't evaluate all relevant loss terms of the model. It would have been nice to see the ablation for all combinations that contain $\mathcal{L} _{\text{guidance}}$, and in particular $\mathcal{L} _{\text{guidance}}$ alone (and not $\mathcal{L} _{\text{consistence}}$). In this state, it is not clear what, e.g., $\mathcal{L} _{\text{cycle}}$ or $\mathcal{L} _{\text{consistence}}$ contribute separately.

A: Thank you very much for this suggestion. We refer to the “global” response and Figure 1 in the attached file.

Q: The evaluation on the ECG data is not fully clear, since meta-information, such as sampling rate, and other details are not given (as far as I can tell). For readers who do not know what a "12-lead ECG data" is, the supplement should contain this information (at least a rough overview). How does 12-lead ECG data look? Are they one-dimensional time series? At what rate are they sampled?

A: Thank you very much for this suggestion. A 12-lead ECG (electrocardiogram) refers to the 12 different perspectives of the heart's electrical activity that are recorded, which is 12-dim time series data. We use the data from PTB-XL dataset and ICBEB2018 dataset at a sampling frequency of 100 Hz, which means 100 samples per second. We include this necessary information in the revised version in section 4.2.

Q: How are the hyper-parameter $\eta$ chosen?

A: Thank you very much for this question. The guidance term is calculated based on data from the source distribution, while two regularization terms are calculated based on data from the target distribution. To choose a good $\eta_1$ and $\eta_2$, we must take the number of samples in the source $m$ and target distribution $n$ into consideration. Therefore, we initially set $\eta_1= \eta_2 = \sqrt{n/m}$ and rely on the grid search to determine the $\eta_1$ and $\eta_2$.

Q: A quantitative evaluation on at least 2-3 more high-dimensional real-world data sets should have been conducted in addition.

A: Thank you very much for this suggestion. We refer to the "general" response and Table 1 in the attached file.

Q: Task 4.2.2 is a bit unclear. The authors should be more detailed what the goal is and what they are exactly doing (even though one can infer it from the text).

A: Thank you very much for this suggestion. We include a paragraph in section 4.2.2 to describe the ECG classification task in the revised version.

Thank you for the response to my review and comments.

I believe the method would make good contribution to the community, but a sparse and not revised experimental section prohibit a higher score. Having read the reviews of the other reviewers, I agree that the method clearly needs more evaluations.

We thank the reviewer for the comments and suggestions. We appreciate the time you spent on the paper. Below we address the concerns and comments that you have provided.

Q: The paper primarily validates TGDP on Gaussian mixture simulations and ECG datasets. Its performance on other types of data or domains is not empirically tested or discussed. More experiments on real-world datasets are required to further validate the effectiveness of the proposed framework.

A: Thank you very much for your comments. We refer to the "general" response and Table 1 in the attached file.

Q: The computational cost of training the guidance network and the diffusion model should be discussed.

A: Thank you very much for this question. We provide the computation cost of training in Table 2 on page 9. The number of parameters in the guidance network (domain classifier) is much less than that of the diffusion model, which reduces the computational costs against fine-tuning based methods.

Q: In what scenarios does the proposed framework work best?

A: Thank you very much for this question. In this work, we prove that using the guidance $\nabla _{\mathbf{x} _t} \log \mathbb{E} _{p(\mathbf{x}_0|\mathbf{x}_t)}\left[q(\mathbf{x}_0)/p(\mathbf{x}_0)\right]$ (or $\nabla _{\mathbf{x}_t} \log \mathbb{E} _{p(\mathbf{x}_0|\mathbf{x}_t, y)}\left[q(\mathbf{x}_0, y)/p(\mathbf{x}_0, y)\right]$ for conditional generative model), we can generate the samples from target distribution with the source pre-trained generative model. Therefore, The performance of our method depends significantly on the accuracy of the estimated guidance network, particularly the estimated density ratio between the target and source distributions. When this density ratio is accurately estimated (especially when the relative magnitudes are accurately captured), our method achieves optimal performance.