[1] Arbel, M., Matthews, A. and Doucet, A., 2021, July. Annealed flow transport monte carlo. In International Conference on Machine Learning (pp. 318-330). PMLR.

[2] Girolami, M. and Calderhead, B., 2011. Riemann manifold langevin and hamiltonian monte carlo methods. Journal of the Royal Statistical Society Series B: Statistical Methodology, 73(2), pp.123-214.

[3] Geffner, T. and Domke, J., 2021. MCMC variational inference via uncorrected Hamiltonian annealing. Advances in Neural Information Processing Systems, 34, pp.639-651.

[4] Geffner, T. and Domke, J., 2023, April. Langevin Diffusion Variational Inference. In International Conference on Artificial Intelligence and Statistics (pp. 576-593). PMLR.

[5] Doucet, Arnaud, Will Grathwohl, Alexander G. Matthews, and Heiko Strathmann. "Score-based diffusion meets annealed importance sampling." Advances in Neural Information Processing Systems 35 (2022): 21482-21494.

[6] Del Moral, P., Doucet, A. and Jasra, A., 2006. Sequential monte carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(3), pp.411-436.

[7] Zhang, Q. and Chen, Y., 2021. Path integral sampler: a stochastic control approach for sampling. arXiv preprint arXiv:2111.15141.

[8] Vargas, F., Grathwohl, W. and Doucet, A., 2023. Denoising diffusion samplers. arXiv preprint arXiv:2302.13834.

Dear Reviewer ndxa,

We thank you for the time and effort taken to review our manuscript and for the many careful and insightful comments presented.

We believe that there might have been a small confusion and you have accidentally reviewed the wrong manuscript. It might be possible that you wrote your review based on a much earlier workshop version of this paper that was publicly available online before the start of the review period. Alternatively it may have been the case that you did not see the main contributions of our paper being a new sampling algorithm, which our extensive experimental section focuses on, instead providing comments on some additional toy 2D experiments that are only in the appendix.

We believe this may be the case since the focus of your feedback is oriented to experiments that are not present in the main manuscript that we have submitted on openreview.

Furthermore, we sincerely request the reviewer to revisit the experimental section of our openreview submission, where we have high dimensional non-toy experiments on sampling from densities, which we believe is the main focus of this work, and quite different from the workshop submission.

To explain why we believe this is the case we would like to point to the following comments and feedback the reviewer has provided:

1. “The ablation study of HJB-regulariser is weak.”
2. “that Figure 2 is more well behaved than non-regularized transport, we still found that Figure 3 is good enough in real practice.”

If you revisit the experiments in the main pdf, you will notice we have experiments on 8 different datasets (target densities) ranging in dimensions from 2 to 1600, and averaging a dimension of ~ 231 which is quite high-dimensional for our focus of sampling from unnormalised densities. We sincerely request that the reviewer draw conclusions based on these 8 experiments in the main text where we compare them to 7 different competing methods.

The reviewer's feedback refers only to the toy 2D experiments that are at the end of the appendix, which we are happy to address but we would reiterate that our main experimental section on sampling experiments is the focus of the paper, not the final appendix section, which we don't mention in the main text and is mainly included as an interesting aside.

We would further like to remark that the HJB-regulariser is not our main proposed method / algorithm and as the title of the paper motivates, the controlled Monte Carlo diffusion sampler CMCD (which we have ablated thoroughly in our experiments) is the main contribution of this work. We apologise if this was a point of confusion. The HJB regulariser is mostly a conceptual/theoretical remark that is in line with our overarching framework whilst our experimental focus is on our proposed method for sampling (CMCD). Therefore, the minor empirical ablations on the HJB regularizer are left to the appendix and are by no means the central contribution of our paper.

You will see in our main manuscript submitted here you can find quite a comprehensive array of experiments for sampling comparing our CMCD method to prior work (note these are not toy experiments):

1. Ionosphere Logistic Regression Target. Dimension = 35
2. Log Sonar Logistic Regression Target. Dimension = 61
3. Brownian Motion Posterior Target. Dimension = 32
4. Lorenz Model Posterior Target. Dimension = 90
5. Seeds (Random effect regression) Target. Dimension = 26
6. Log Gaussian Cox Process Target. Dimension = 1600
7. Neals Funnel Target. Dimension = 10
8. Toy Difficult GMM [1] Dimension = 2

Note that across all these targets we significantly outperform an array of sampling methods that we compare to:

1. ULA [2]
2. UHA [3]
3. MCD-UHA [4,5]
4. MCD-UHA [4,5]
5. SMC [6]
6. PIS [7]
7. DDS [8]

We would also like to highlight that these experiments are standard baselines for sampling used consistently across many prior works focused on sampling and partition function estimation [1,2,3,4,5,6,7,8]. Note that for sampling, unlike generative modelling, dimensions >= 50 are already considered to be very high dimensional, as many of the provided references suggest.

Finally, to further the extensive nature of these experiments we would like to cite reviewer LiYc’s remark:

“Extensive experiments show that the CMCD method outperforms existing methods in relatively high dimensional examples.”

We hope the reviewer can agree that these experiments are significantly different and much more extensive than the toy 2D experiments from the appendix. We sincerely request the reviewer to reconsider the comprehensive sampling experiments we ran to support our new sampling algorithm as the core contribution, and we would be grateful if they could reevaluate their score accordingly.

Dear reviewer ndxa,

Thank you for the prompt response. It is possible you are looking at a different paper, as DNF is not applicable to our task

Our paper is focused on sampling from unnormalized densities. DNF is a method for generative modeling when you have access to data, this method is not applicable or relevant to the sampling task that we consider (where we do not have access to data or samples).

We cannot compare/do ablations to DNF as DNF is focused on a different task tangential to our paper. Could you please check the experiment section here https://openreview.net/pdf?id=PP1rudnxiW (this is the pdf for this paper on openreview).

Our proposed algorithm CMCD has nothing to do with the HJB regularisation remark, and this is what we explore empirically. We do a careful ablation of its performance across different timesteps, furthermore, in Appendix F.3 you can find further ablations.

Dear Reviewer ndxa,

We would like to thank you for the time you spent reading our work and the feedback you have provided. In what follows we aim to address your main question and detail how our revised version clarifies this.

We have uploaded a revised version of the manuscript, which introduces the sampling problem, that is the task of sampling from an unnormalised density and estimating its normalising constant. This is a seminal task in both statistics and machine learning [1-8] that predates generative modelling.

We further reiterate in the revised version that our task is focused on sampling, not generative modelling. The goal here is as follows; given we have access to a probability density function of the form:

$p(x) = \frac{e^{-U(x)} }{\int e^{-U(x)} dx}$

Where $U(x)$ is a function that we can evaluate and differentiate pointwise, however we are unable to evaluate the normalising constant $\int e^{-U(x)} dx$.

Given such a distribution $p(x)$, the goal is to learn or design a model that can generate samples that are distributed according to $p(x)$, notice unlike generative modelling we do not have access to any data or samples. One way to do so is via minimising KL w.r.t to a parametric distribution $q_{\theta}(x)$:

$\arg_{\theta}\min D_{KL} (q_{\theta}(x) || p(x))$

Note that the above KL can only be computed up to a constant in practice ($\int e^{-U(x)} dx$), and thus an ELBO is what we typically minimise. This family of approaches is typically also referred to as Variational Inference, and is a superset of techniques involving approximating intractable distributions [13], VAEs are simply a particular, unrelated sub-instance of VI and are not particularly applicable to our task. Therefore, there are no low-dimensional latents in this formulation.

Typically one wants 2 things from $q_\theta$:

1. It is easy to sample from numerically (not intractable)
2. It is a very flexible distribution so that $D_{KL} (q_{\theta}(x) || p(x))$ can be brought very close to $0$.

A common way this is achieved is through [9,14] applying a series of tractable transformations to a simple Gaussian, e.g. :

$

z_0 \sim \mathcal{N}(0,I) \

$

$

z_1 \sim \mathcal{N}(f_{\theta_1}(z_0), \sigma_1^2I) \

$

$

\vdots \

$

$

x = z_K \sim \mathcal{N}(f_{\theta_{K-1}}(z_{K-1}), \sigma_{K-1}^2I) ,

$

the resulting $q_\theta(x)$ is then a very flexible distribution that is easy to sample from.

Notice there is no need for the latents to be lower-dimensional as our goal is to minimise $D_{KL} (q_{\theta}(x) || p(x))$ so that we can sample $p(x)$. We do not care about $z_0$ being an interpretable quantity, we just care about it being a tractable quantity. This is a very common use case of VI [9,10,11,12].

Also, notice that these flavours of deep latent models with equal latent dimensions have been studied quite extensively already in the context of both neural ODEs and SDEs [14,15,16].

Our main contribution is a novel algorithm (CMCD) that allows us to parametrise $q_{\theta}(x)$ with a tractable and learnable SDE that starts at a Gaussian $z \sim \mathcal{N}(0,I)$ and then approximately (with guarantees that we prove) transports the Gaussian to the target distribution $p(x)$.

We hope this answers your question on VI and equal dimensional latents:

“In Variational inference literature, the hidden dimension is a low dimensional manifold that can be interpretable factors? The latent of VAE uses smaller latent dimension than the data... ”

We look forward to any further doubts you might have and are happy to provide any further information. In particular, we hope now that the task and focus of this paper are clear and you might be able to appreciate our sampling experiments and ablations better, as they are by no means toy examples, and are a comprehensive ablation on standard benchmarks, targets and baselines across sampling literature.

[9] Rezende, D. and Mohamed, S., 2015, June. Variational inference with normalizing flows. ICML

[10] Doucet, Arnaud, Will Grathwohl, Alexander G. Matthews, and Heiko Strathmann. "Score-based diffusion meets annealed importance sampling." Nuerips

[11] Geffner, T. and Domke, J., 2023, April. Langevin Diffusion Variational Inference. In AISTATS

[12] Wu, H., Köhler, J. and Noé, F., 2020. Stochastic normalizing flows. Neurips, 33

[13] Blei, D.M., Kucukelbir, A. and McAuliffe, J.D., 2017. Variational inference: A review for statisticians. Journal of the American Statistical Association,

[14] Tzen, B. and Raginsky, M., 2019. Neural stochastic differential equations: Deep latent Gaussian models in the diffusion limit.

[15] Chen, R.T., Rubanova, Y., Bettencourt, J. and Duvenaud, D.K., 2018. Neural ordinary differential equations. Neurips

[16] Xu, W., Chen, R.T., et al. 2022, May. Infinitely deep Bayesian neural networks with stochastic differential equations.

Dear Reviewer ndxa,

Thank you so much for engaging with us and for the very helpful comments that are significantly improving our manuscript.

Upon your recommendation, we have added the following additional experiments and ablations (see Appendixes F.2. - F.5. in the revised manuscript https://openreview.net/pdf?id=PP1rudnxiW):

1. A VI Normalising flow baseline (VI-NF) on the GMM, Funnel and LGCP targets.
2. Comparisons to VI-DNF across the GMM, Funnel and LGCP targets. As expected (from our theory but also comparisons to DDS and PIS) we significantly outperform VI-DNF.
3. Comparisons to AFT, across all datasets with tractable distributions, where outperform on 6/8 targets and remain competitive on 2.
4. Comparisons to VI-NF across all datasets , where we outperform on 5/8 datasets and remain competitive on 1. The 2 target densities VI-NF performs better are on the more lower-moderate dimensional end and are unimodal.
5. We compare to Underdampened Langevin Annealing comparisons in the main section of the paper (UHA) is the acronym we use for it.
6. Wallclock times comparing different methods. Note all methods are trained for the same amount of iterations and parametrised (if needed) by neural networks of the same size across methods.

These ablations can now be found in Appendix F.4. - F.6 colored in teal, Tables 5,6 and z (apologies if color is unhelpful). It is important to note that for the VI-NF we match $K$ to the number of IAF flows which quickly results in VI-NF having much more parameters than our modest drift network used in CMCD.

Now we would like to address your questions:

Moreover, some other baselines such as Underdamped Langevin, is missing

We already compare to Unaderdampened Langevin Dynamics. Underdampened Langevin Dynamics is in fact Hamiltonian dynamics + Brownian noise, and we use the acronym UHA (Unadapted Hamiltonian Annealing) to refer to it. If you look carefully at our plots in Figure 1 a) and b), you will notice there is a column for Underdampened (UD) methods and a column for Overdamped methods (OD). UHA corresponds to Underdampened Langevin dynamics. We further explain the OD and UD acronyms in the experiment section.

Also, it is not fair to compare $K$ with ULA, as you introduce additional parameterization in the model

I think there is a tiny misconception here. Our algorithm provides us with an optimisation objective that basically allows us to train / improve on ULA in way that variance is reduced and we can obtain better estimators for ln Z . Notice when $\phi=0$, our approach is in fact exactly the same as ULA. Our propositions and lemma provide us with an objective to improve on top of ULA. As the reviewer pointed out, the $\phi$ parametrisation will increase the compute time, but correspondingly, there is significant improvement in performance over ULA that cannot be achieved otherwise. We trade off some extra compute for much lower variance and better performance.

Rather than an unfair comparison, the ULA line serves as a baseline to see how much our approach improves from its starting point. However, for transparency, we have added wall-clock times for ln Z estimation, to showcase the additional compute that is required for this significant improvement in performance.

We have ensured the other methods that are neural SDEs have the same number of parameters, and their computational complexity is the same as CMCD for sampling, However, we have also added wallclock times for some these (e.g. MCD).

Finally, as Reviewer yH6G mentioned, the presentation of the paper needs to be revised extensively

We wholeheartedly agree with the reviewer that the initial presentation needed extensive revision, and we have significantly changed our presentation to address reviewer yH6G’s remarks. This can be seen in the revised pdf (https://openreview.net/pdf?id=PP1rudnxiW) ). Additionally, we wrote a 2nd alternative draft for the the introduction with a different presentation flow so that reviewers had an extra option to choose from (https://anonymous.4open.science/r/CMCD-6BF6/rebuttal_version.pdf ), please note this version does not contain the updated experiments the only bit to read here is the different introduction (presented differently).

We would like to remark that reviewer yH6G was very positive about our new presentation, and they increased their scores from a 3 to an 8, stating that we addressed their remarks:

I sincerely thank the authors for their comments and their extensive modifications. I think the presentation is much clearer now and I also now finally understand the greater context … I believe with this modification the paper has reached finally a place where it is broadly significant and I have modified my original rating to reflect this..

We would love to hear your feedback on our revised ablations and presentations, and we hope that this has addressed some of your concerns and questions.

Wallclock Times

Here we provide comparisons for wallclock times for sampling and lnZ estimation (the number of samples kept the same across methods, more precise details can be found in our appendix). Notice how the overhead from our proposed approach whilst higher is still not a large computational overhead.

Method Descriptions + References

• AFT: Anealed Flow Transport Montecarlo [1], a more modern and scalable flow method than NFVI
• VI-NF: Stacked Inverse autoregressive flows (IAF) for variational inference (uses the codebase of [1]) . We match $K$ to the number of stacked IAF layers.
• DNF-VI: Diffusion Normalising Flow Baseline adapted to VI suggested by the reviewer.

Note: AFT was unable to train successfully (became unstable and did not converge) on the Lorenz and Brownian targets. We sweeped over learning rates and hyperparameters to try and stabilise this but were not able to produce reasonable results.

[1] Arbel, M., Matthews, A. and Doucet, A., 2021, July. Annealed Flow Transport Monte Carlo. In International Conference on Machine Learning (pp. 318-330). PMLR.

Dear reviewer ndxa,

With the revision of our manuscript for which reviewer yH6G stated “has reached finally a place where it is broadly significant“ and “the presentation is much clearer”, we hope to have addressed any concerns you may have had regarding the presentation of our manuscript.

Furthermore, as per your suggestion, we have implemented new baseline algorithms (NF-VI, DNF-VI, AFT) and compared our algorithm against them on a suite of target distributions, showing the strong performance of our algorithm.

Finally, we also provide wallclock times of evaluation of our baselines and computational complexity comparisons. We look forward to engaging with you further during the rebuttal period to address any concerns you might have remaining and hope to hear your feedback and review of our revised manuscript.

Dear Reviewer nqRv,

We thank you so much for the very positive comments on our work, we are thrilled to hear such positive and constructive feedback. We thoroughly agree that more work can be done towards improving the details in the experimental section, and thus we have uploaded some changes to the experimental section within the appendix as requested (we have highlighted your requested changes in orange).

In what follows we will go over each of the points raised:

In my opinion the paper could benefit from a shorter introduction and motivation and instead an extended experimental section with method limitations, experimental details,

We have moved the HJB remarks which were not central to our story to the appendix. As a result we expanded the practical motivation of the work further and added Algorithm 1 detailing CMCD which should heavily benefit more applied audiences.

computational trade-offs, etc. for more applied audiences. Since this is likely not possible due to page limits, an extended treatment in the appendix would be very welcome.

We have added Table 8 which details the sampling and loss complexity of the main methods that we compare to. In particular, we have also added wallclock times for inference (sampling + ln Z estimation) comparing several of the ULA + MCD-based methods and ours. We can observe that once trained, sampling time is comparable across approaches.

What are the training times of the method, for instance, in comparison to running SMC?

It is not entirely straightforward to compare SMC to our approach taking training into account, as given our fixed training iterations we dramatically outperform SMC across challenging high dimensional targets.

To carry out an insightful comparison, we chose the LGCP target, which is our most numerically intense target, and we phrase the following question:

“For how long do we have to train CMCD to outperform the best-run of SMC”

For this, we look at Figure 1 and see that at $K=8$, CMCD already outperforms SMC at $K=256$ with 2000 particles. So we choose these two approaches to compare to each other. Below is a brief comparison of training-time calculation, where we have included the tuning time for SMC, which is akin to our training time (as without tuning SMC’s hyperparameters, ELBOs and ln Z estimations were much worse).

How were the neural networks architectures chosen and how is the hyperparameter selection and other details impacting the experimental results (e.g., see Table F1).

We did not tune the architectures, we inherited them from the LDVI [1] paper, they are super minimal feedforward architectures, where the only interesting design choice is on the embedding layer used to encode the timesteps. We have added a diagram for the architecture as well as some additional details like activations (see Figure 2 in the revised version).

weaknesses other than that the source code for the experimental validation needs to be provided in the supplement

We have uploaded a zip file to the supplement with our source code as requested. We will also open-source the code on Github at the camera-ready stage (if the paper is accepted), where it will be available publicly.

We hope that with the added figures, discussions, and further ablations we have addressed the reviewer’s remarks and look forward to further feedback!!

[1] Geffner, T. and Domke, J., 2023, April. Langevin Diffusion Variational Inference. In International Conference on Artificial Intelligence and Statistics (pp. 576-593). PMLR.

Dear Reviewer 2i1s,

We thank you for your comment and are glad to see that our response addresses your question. With this additional information and clarification that we have provided, as well as our experimental results establishing SOTA across 7 datasets, we humbly ask if the reviewer could consider re-evaluating their score if their major questions have been addressed. Alternatively, we are very happy to address any additional suggestions that could improve our contribution and make it a stronger candidate for acceptance with a higher score and are glad to engage with the reviewer throughout the rebuttal period!

Update: We have updated a revised version of our manuscript, including your suggested comments and additional information. These changes have been colored in blue and can be found on pages 6 and 31.

Dear Reiveiwer 2i1s,

We would like to thank you for the very helpful comments and for taking the time to read our work with such care.

Before we upload an updated version addressing your feedback, we would like to clarify some of your questions if possible in order to understand how we can improve the EM contributions and their clarity:

1. “I am unsure how novel the restricted correspondence between IPF and EM is”

To the best of our knowledge, our paper is the first to make this connection. In case this is not true, we would appreciate it if you could point us to any references where this connection is already proved. Is there any other aspect of our contribution where you might be uncertain about the novelty, which we would be happy to clarify?

2. “EM does not allow for the forward generative model $\pi^{2 n+1}(x, z)$ to have as a marginal $\pi_x(x)=\mu(x)$ and the backward to have as a marginal $\pi_z(z)=\nu(z)$ … .”

When you say the general setting, do you mean the case where $p^\theta(x|z)$ and $p^\phi(z|x)$ are restricted? You are completely right that the connection between EM and IPF only holds when these transitions are made to be flexible enough. We mention this briefly in the introduction, but will indeed emphasize this more explicitly. Indeed if the transition densities are restricted it is not possible to match the marginals via EM, however in our paper we focus on the unrestricted setting and thus our construction of EM⇔ IPF is valid.

3. I think it would be very helpful to have some comments on this point in the paper.

We aim to address this in the main text by adding the following remark:

“Note that the correspondance between the EM and IPF iterates can only be made true if the transition densities are unrestricted. This can be achieved by parametrising them as transition densities of SDEs and thus can be carried out approximately via discretising these SDEs in practice. For the setting where one may restrict them to Gaussians as done with VAEs it becomes clear that the EM iterates cannot be minimised fully to enforce the constraint required by the IPF iterates.”

We hope that this remark addresses your concerns, and would be happy to add any additional details that might clarify this further. We would like to highlight that this is mostly a theoretical connection that unifies classical algorithms from VI and OT. We intended this as a bridging element that can allow readers from VI to familiarise themselves with OT through something they are already familiar with and vice-versa.

4. they propose a score-based annealing (Controlled Monte Carlo Diffusion) that betters the state of the art on two datasets: funnel and GMM

We thank the reviewer for pointing out the state-of-the-art results that our algorithm achieves, however, we would like to clarify that our method does indeed obtain state-of-the-art performance on 7 of the 8 datasets we consider, and is highly competitive on the remaining dataset (Lorenz, d=90). All of these 8 datasets are highly popular in the literature as standard benchmarks.

Finally in order to further enhance the EM contributions, we have recently been able to show that our EM⇔ IPF correspondence extends beyond KL divergences and to f-divergences (via applying results from Proposition 6 in [1]). We will add a further proposition and proof to the appendix extending our EM⇔ IPF result to f-divergences, and we hope that this further novel connection strengthens the novelty of this connection for the reviewer and the community.

Once again, we thank the reviewer for their insightful comments, and would be very grateful if the reviewer could clarify whether these additions address their concerns regarding the EM contribution, and any further clarifications we could make in case we don’t address their concerns.

[1] Baudoin, F., 2002. Conditioned stochastic differential equations: theory, examples and application to finance. Stochastic Processes and their Applications, 100(1-2), pp.109-145.

Dear Reviewer LiYc,

Thank you for your review and very insightful comments which have led to major improvements in our revised version.

It is quite unfortunate that there are no experiments for the loss proposed in Proposition 3.2 to back the fact that the proposed loss does not suffer from mode forgetting.

Out of the alternate losses we discuss, it is the log-variance divergence (introduced after Equation 24) that can handle mode collapse:

$\mathcal{L}^{\mathrm{Var}}(\phi)=\mathrm{Var}\left(\ln \frac{\pi_T\left(\boldsymbol{Y}_T\right)}{\pi_0\left(\boldsymbol{Y}_0\right)}+\int_0^T \Delta \phi_t\left(\boldsymbol{Y}_t\right) \mathrm{d} t-\sigma \sqrt{2} \int_0^T \nabla \ln \pi_t\left(\boldsymbol{Y}_t\right) \circ \mathrm{d} \boldsymbol{W}_t-\sigma^2 \int_0^T\left|\nabla \ln \pi_t\left(\boldsymbol{Y}_t\right)\right|^2 \mathrm{~d} t\right)$

The loss with the HJB/PINN regularizer in Proposition 3.2 does not address the issue of mode collapse, and its application is more relevant to different problems (relevant to molecular dynamics).

Furthermore, it is not clear how this is implemented in practice since a Laplacian term is involved.

As you have pointed out, there are some Laplacian terms that are not immediately obvious how we could compute. However, as it turns out the term inside the variance is simply the RND between the forward and backwards SDEs, so we can use the exact same discretisation that we used for KL to discretise it and thus:

$\mathcal{L}^{\mathrm{Var}}(\phi) \approx \mathrm{Var}\left[\ln\frac{\pi_0(Y_0)}{\hat{\pi} (Y_T)}\prod_{k=0}^{K-1}\frac{\mathcal{N}(Y_{t_{k+1}} |Y_{t_k} + (\nabla \ln \pi_{t_{k}}+\nabla \ln \phi_{t_k})(Y_{t_k}) \Delta t_k , 2 \sigma^2 \Delta t_k)}{\mathcal{N}( Y_{t_k} |Y_{t_{k+1}} + (\nabla \ln \pi_{t_{k+1}}-\nabla \ln \phi_{t_{k+1}})(Y_{t_{k+1}}) \Delta t_k , 2 \sigma^2 \Delta t_k)}\right]$

This makes this loss as scalable as the KL divergence CMCD objective that we ablate. In particular, it is a bit more scalable as the Variance can be taken w.r.t to any measure, thus allowing us to detach the samples and circumvent backpropagating through timesteps.

.. There are no experiments for the . ..

To address your main weakness, we explore the log-variance (logvar) loss in the very challenging 40 GMM target [2] for which reverse KL-based methods all mode collapse. We compare CMCD KL to CMCD with logvar, and we demonstrate how it significantly outperforms the KL-based loss, both in ELBO but also qualitatively in its ability to overcome mode collapse. The results and plots illustrating mode collapse can be found in Appendix F.3. (highlighted in olive) in particular Tabe 2, Figures 3 and 4, and here is a table to summarise them

… for the loss proposed in Proposition 3.2 to back the fact that the proposed loss does not suffer from mode forgetting ..

The loss in proposition 3.2 (now moved to the appendix) solves a different problem to mode collapse. Rather than addressing mode forgetting, it deals with forgetting the reference SDE. This does not solve practical issues (e.g. such as mode collapse) in sampling or generative modelling, but it might be useful in transition path problems in MD. In appendix G, we do some small ablations and demonstrate across a series of toy targets (including an MD double well problem) that it does indeed do well in not forgetting the reference process. To address the Laplacian term computation in this loss in this case, we used specialised architectures from OT [1], which allow for fast calculation of these terms.

Finally, we note we have added a series of further experiments and ablations to address the other reviewers as well as improved the presentation of our work.

We hope that this addresses your main weakness, and we sincerely request any further feedback that you may be able to provide us with regarding our comprehensive changes to the presentation of the paper, as well as additional ablations and baselines. Thank you for your continued feedback!

[1] Onken, D., Fung, S.W., Li, X. and Ruthotto, L., 2021, May. Ot-flow: Fast and accurate continuous normalizing flows via optimal transport. In Proceedings of AAAI

[2] Midgley, L.I., Stimper, V., Simm, G.N., Schölkopf, B. and Hernández-Lobato, J.M., 2022. Flow annealed importance sampling bootstrap. ICLR 2023

Dear Reviewer yH6G,

We thank you for the time and the very detailed feedback, especially since the presentation difficulties you have encountered must have required more time in parsing through our manuscript, and we apologise for the same.

We would also like to thank you for the very positive comments regarding the strength of the technical contribution of our work. We really appreciate your feedback and suggestions for improvement despite the presentation of the paper, while still being able to provide excellent actionable feedback on our technical contribution.

We would like to work with you as much as possible through this discussion period to understand how we can further improve the presentation. We appreciate that you must have many other commitments but we would greatly appreciate it if you could engage with us in improving the presentation of our work.

We have uploaded a revised version of the manuscript where we have attempted to address some of the changes recommended by the reviewer, these changes have been coloured in magenta/pink.

The revised manuscript (https://openreview.net/pdf?id=PP1rudnxiW) contains the following changes which are highlighted in the revised version.

1. We have mentioned that our focus is the sampling task from a very early stage in the introduction and we formally introduce the sampling task in a titled paragraph, connecting it to framework 1 and thus providing motivation to our setup.
2. We have reworded the abstract to acknowledge the focus on the sampling task and we have removed the remark on the HJB regularisers from the abstract as this is not our central contribution.
3. We have also highlighted that the EM ⇔ IPF correspondence mostly serves a conceptual purpose that connects two adjacent fields rather than an algorithmic one.
4. We have changed our contributions to shift the focus to CMCD and the sampling task.
5. On page 6, we removed non-central results such as the paragraph on HJB regularizers and the corollary that followed. As before, we highlight that the EM and IPF connection is conceptual rather than practical.
6. Also on page 6 we explain more clearly the issues present with EM and IPF, emphasising that an accumulation of errors occurs due to their sequential nature. We then signpost to the next section motivating that our proposed method CMCD can be trained end to end with simultaneous updates and thus solve this sequential error accumulation issue.
7. We have also added pseudocode for sampling and $\ln Z$ estimation with CMCD and we hope that this also draws the focus/attention to the task and enhances the readability.

Additionally, we have also tried restructuring the introduction in a different way which can be found in this anonymised link https://anonymous.4open.science/r/CMCD-6BF6/rebuttal_version.pdf (same link as the anonymised codebase), here we changed the way the framework is introduced to try and make it more amenable to sampling. We hope that one of these revisions addresses some of the presentation issues raised by the reviewer.

We hope that the revised presentation and rebuttal pave the way to addressing the reviewer's concerns and we are eager to hear the reviewer's feedback plus further points of improvement.

In what follows we will directly address some of your other questions/comments (which have been updated in the revised version of the manuscript):

Is eq. (10) the same as eq (1)? I failed to find the difference.

The following equations are taken from the unrevised version of the manuscript that you read.

Equation (1):

$\boldsymbol{z} \sim \nu(\boldsymbol{z}), \quad \boldsymbol{x} \mid \boldsymbol{z} \sim p^\theta(\boldsymbol{x} \mid \boldsymbol{z})$

Equation (10):

$\mathcal{L}^{ext}(\phi, \theta) :=D(q^{\mu, \phi} ( y_{0: L} ) \vert\vert p^{\nu, \theta} ( y_{0: L}))$

Equation (1) specifies a “static” generative model whilst equation (10) details the divergence between two pathwise (many latent) generative models. Is it possible the reviewer was referring to a different pair of equations? Maybe they meant equations 10 and 5:

Equation (5)

$\mathcal{L}_D(\phi, \theta):=D\left(q^\phi(\boldsymbol{z} \mid \boldsymbol{x}) \mu(\boldsymbol{x}) \vert\vert p^\theta(\boldsymbol{x} \mid \boldsymbol{z}) \nu(\boldsymbol{z})\right)$

Both Equations 10 and 5 define the loss as a divergence between backwards and forwards generative models. However, Equation 10 (as the text preceding it details) is a path KL divergence because the generative model has many intermediate latent variables (which becomes an SDE in the limit) whilst Equation 5 does not have any of these intermediate latents. It is a static/non-dynamic setup (you can think of the intermediaries as being marginalised out).

The purpose of having both these equations is, as reviewer LiYc highlights (i.e. “I enjoyed reading this paper; it is highly pedagogical and sheds light … ”), pedagogical. We present our work and framework in a pedagogical manner, first starting from simple static generative models that readers may be familiar with from VAE literature, and then we slowly build up from there to the non-static / dynamic SDE setting.

Proposition 3.1 explains why IPF fails to perform well. there is no explanation afterwards to say why going through the lens of EM solves this issue.

Indeed this was presented in a very confusing manner and has been changed in the revised version.

The purpose of the connection to EM was not to solve an issue with IPF but to highlight how there exists a correspondence between these two seminal algorithms that have been developed almost in isolation from one another. This provides us a conceptual link between VI and OT which can help when learning one field given knowledge of the other.

Thank you for bringing this point to our attention.

. Rather, the text proceeds to describe optimality conditions in what seemed to me to be like a tangent.

This is a really good point we were unable to explain this fully due to the page constraint, we have moved this section to the appendix and taken more time to explain the issues with VM and IPF.

It pains me to say that I cannot suggest the authors any single thing that could drastically improve the paper. I appreciate the technical contribution.

We are very sorry that the initial presentation of our work has made you feel this way, however, we still believe your comments have been very helpful and have led us to make significant restructuring changes across the paper.

At the very least, I think the first two pages of the introduction should be thoroughly revised to focus more on the problem addressed in the paper

We have added the sampling task (which is the task CMCD solves) to the start of the introduction and modified the signposting and motivational elements in the introduction to refer more to this task.

I also think the propositions which are not key contributions of the pape … could be better placed in the supplements.

We have moved proposition 3.2 to the appendix as well as the section on HJB reguralisers which are not part of our central algorithmic contribution/experiments. We hope this allows the reader to have a more clear and contextual focus on the story.

Rather than overloading on related technical terms and connections.

We have not fully removed the connections / related work from the introduction as you requested for the following reasons

1. Variational inference is effectively the flavour of the task we are solving. We are doing sampling through VI using ideas from optimal transport. This requires us to introduce and discuss VI (Framework 1) in the introduction.
2. ELBOs are used for solving the second part of the sampling problem, namely estimating the normalising constant, and thus it is essential for us to introduce these earlier on.
3. OT and coupling allow us to highlight some of the pitfalls that doing full-blown VI can have, and as part of the motivation we want to discuss these issues in the introduction since they motivate our proposed improvements.

Instead, we have tried to reinforce their relevance to the sampling problem.