Dear Reviewer HM7Y,

Thank you for your thoughtful and encouraging review. We appreciate your recognition of the contributions and evaluations of our work. Below, we address your comments in detail.

W1: The RND weight varies between several orders of magnitude for different tasks. An ablation study would help to understand the sensitivity of the method for different RND weights.

To resolve your concern, we provide experimental results that verify that intensive hyperparameter search is not needed, and the results from different RND weights are not sensitive.

Taking these factors into account, we conducted experiments across approximately four different RND weight values per task to ensure stable training and meaningful results. The calibration of the RND weight $\alpha$ is important but simple: run 20% of the second round to measure the performance.

Table 1. ELBO-EUBO gap at 20% of the second round for Manywell-128, LJ potentials, and ALDP.

We found that reasonable values could be selected with minimal tuning, though a more systematic approach could be explored in future work.

---

W2: I currently doubt that the method outperforms forward KL training combined with one of parallel tempering, replica exchange MCMC, or MD.

To dispel your doubt, we show that our method outperforms MLE (or forward KL) training on samples from parallel tempering (or replica exchange) MD in Table 2.

Table 2. ELBO-EUBO gap comparison between parallel tempering + MLE (or forward KL) training and ours with parallel tempering searcher. We align the number of energy calls by collecting more data for MLE.

---

W3: The ablation study on ManyWell shows that the RND searcher does not significantly boost performance.

We believe the RND searcher significantly boosts the performance since the reported values are in log scale. The presence of the RND searcher results in an improvement of $6.4$ of ELBO in the ablation study, which actually represents $\exp(6.4) \approx 403.43$ times better performance in lower bounding the marginal probability of the Learner generating the true sample.

W4: A comparison to Adjoint Sampling seems missing, which has shown to be very scalable and sample efficient as well.

We reproduced Adjoint Sampling algorithm and conducted the experiments to compare SGDS and Adjoint Sampling on LJ-13 and LJ-55 potentials. The results show that SGDS outperforms Adjoint Sampling in the Wasserstein distances of positions and energy histograms.

Table 3. Comparison of Equivariant position Wasserstein-2 and energy histogram Wasserstein-2 between SGDS and Adjoint Sampling on LJ-13. The parameters of LJ-13 energy function are set to be the same as the settings in our experiments.

Table 4. Comparison of Equivariant position Wasserstein-2 and energy histogram Wasserstein-2 between SGDS and Adjoint Sampling on LJ-55.

We reproduced Adjoint Sampling by closely following the public codes and variable settings, but the training on LJ-55 exploded. So we instead compare our method against the value reported in Adjoint Sampling paper for LJ-55 task.

---

Q1: What does LP stand for?

LP stands for Langevin Parameterization—a neural‑network design where the input includes the energy gradient as mentioned in Section 1, and the network outputs a scaled and biased version of this gradient, thus embedding a Langevin‑style inductive bias [2].

---

Q2: What is the benefit of this intrinsic motivation term, compared to just running MCMC on a higher target temperature?

The intrinsic-motivation term benefits from the ability to incorporate previously visited samples to better guide the exploration process.

---

Q3: Did the authors try the Log-variance loss instead of TB?

Yes, we did. To provide a clean and fair comparison for your question, we conducted experiments using four different seeds across 40GMM and Manywell-32/64,/128. Table 5 is the experiment results of comparing the EUBO-ELBO gap between TB and Log-variance for the Learner's objective:

Table 5. Comparison of EUBO-ELBO gap between TB and Log-variance on 40GMM, Manywell-32/64/128.

Here, we believe that TB outperforms Log-variance since it estimates $\log Z$ using samples from the Searcher, while Log-variance estimates $\log Z$ on-the-fly using a smaller number of samples collected from the Learner.

---

[1] Sugita et al., "Replica-exchange molecular dynamics method for protein folding", Chemical physics letters 1999.

[2] Sendera et al., "Improved off-policy training of diffusion samplers", NeurIPS 2024.

---

We hope our responses have addressed your questions and concerns. Please let us know if there is anything we may have missed or if any points require further clarification.

Dear Reviewer fmfd,

Thank you for your thoughtful and encouraging review. We appreciate your recognition of the clarity of our experiments and evaluations. Below, we address your comments in detail.

W1: The method seems to be primarily an incremental improvement on existing work, in particular, Sendera et al. It would be helpful to qualitatively discuss the differences.

Our contribution is a conceptual shift in a mixture of training schemes compared to prior works. While existing diffusion samplers rely on full online learning [1,2] or use a buffer dependent on the current model's performance [3,4], SGDS utilizes both offline guidance from an independent MCMC-based searcher and online learning from the diffusion model itself. This is a meaningful improvement in the context of the training structure.

---

W2: It is difficult to read the tables because of their heavy use of abbreviations.

To resolve your concern, we plan to incorporate the following clarifications for Table 1 in the future manuscript as an extension of the description written in Section 5.1.

• PIS (Path Integral Sampler) [1], TB (Trajectory Balanced) [5], and FL-SubTB (Forward-Looking SubTB) [6] denote the types of objectives. GAFN (Generative Augmented Flow Networks) [7] is a GFlowNet-based method that directly injects intrinsic rewards into TB loss. AT (Adaptive Teacher) [8] is a framework that introduces an additional neural sampler (the Teacher) trained to focus on high‑loss regions of the Student.

• LP (Langevin Parametrization) [4] refers to a drift construction technique where the model’s drift term is combined with the gradient of the target energy function, helping the trained dynamics to better follow the target energy landscape.

• LS (Local Search) [4] denotes a refinement step where a short Markov chain, guided by the target energy gradient, is applied from the final state of the generated trajectory to improve sample quality.

• Expl (Exploration) [4] represents a noise-scheduling technique applied during trajectory generation, in which additional stochasticity is injected more in the early training phase to promote broad exploration and gradually reduced over time for exploitation later.

---

Q1-1: Why is there such a large discrepancy between TB+LS and SGDS?

The differences of RND reward, initial distribution, and search length make a large discrepancy between TB+LS and SGDS.

While TB+LS relies on only Langevin dynamics with the target energy function, SGDS utilizes RND reward together for sample-efficient exploration, which is one of the key components in our framework.

And TB+LS always seeds each walk from the current diffusion sample [4]; if that sampler is low‑quality in training, exploration is constrained, while SGDS does not depend on the current training states.

Also, TB+LS search is too short to reach high‑reward regions on larger, sharper tasks. SGDS extends each walk, letting trajectories climb farther before termination.

These three factors synergistically contribute to the substantial performance gap between SGDS and TB+LS.

Q1-2: When running TB+LS with different hyperparams, would you effectively recover SGDS (except for re-init and RND reward)?

No, they share some motivations, yet the methods are fundamentally different in where to begin the search. TB+LS starts search from the samples generated by the current Learner; if that sampler is low‑quality in training, exploration is constrained, while SGDS does not depend on the current training states.

Additionally, we conducted an experiment on Manywell-128 to address your question of how TB+LS performs with a double-fewer freuquency but double deeper chains.

Table 1. ELBO-EUBO gap of SGDS and TB+LS with different hyperparameters for local search on Manywell-128.

Fewer and deeper local search with TB could make TB+LS better, but SGDS is still superior in performance and sample efficiency.

---

Q2-1: How are the MCMC chains for the searcher initialized?

We initialize the MCMC chains for the searcher with a Gaussian prior in MALA and AIS, and relaxed positions starting from a fixed position in pdb file in MD.

Q2-2: Would it make sense to draw the initial states from forward trajectories?

Drawing the initial states from forward trajectories does not make sense in our algorithm. We initialize the MCMC chains with a Gaussian prior. Seeding the search with forward trajectories reduces exploration and biases it toward the learner’s current distribution—especially problematic when the learner is immature. This choice helps explain why SGDS outperforms TB + LS, which draws its initial states from the diffusion sampler’s forward policy.

---

Q3-1: Why does SGDS require fewer evaluations (for example, compared to TB+LS)?

SGDS requires fewer energy evaluations than other methods since it consumes energy calls for collecting the Searcher samples, and it recycles the samples and rewards for off-policy training. For example, TB+LS methods waste too many samples for searching high-reward regions as they initialize the local search start point too frequently using the Learner.

Q3-2: Is it correct that SGDS only consumes energy evaluations during search (1 per chain and MCMC step) and on-policy steps (B evals per opt step), but none during off-policy steps?

Yes.

---

Minor1: Eq (3): The indices of the product in $P\_B$ seem to be off by one.

Minor2 : Equations after line 84 and 86: Shouldn't $P\_F(\tau;\theta)$ also be conditioned on $x\_0$ (analogous to $P\_B$ being conditioned on $x\_1$)?

We thank you for pointing out the minor typos in the equations. We will correct $\sum_{i=0}^{T-1}P_B(x_{(i-1) \Delta t} | x_{i \Delta t})$ to $\sum\_{i=1}^{T}P_B(x\_{(i-1) \Delta t} | x\_{i \Delta t})$, and $P\_F(\tau;\theta)\mu\_0(x\_0)$ to $P\_F(\tau|x\_0;\theta)\mu\_0(x\_0)$ in the future manuscript.

---

[1] Zhang et al., "Path Integral Sampler: a stochastic control approach for sampling", ICLR 2022.

[2] Vargas et al., "Transport meets variational inference: Controlled Monte Carlo diffusions", ICLR 2024.

[3] Akhound-Sadegh et al., "Iterated Denoising Energy Matching for Sampling from Boltzmann Densities", ICML 2024.

[4] Sendera et al., "Improved off-policy training of diffusion samplers", NeurIPS 2024.

[5] Malkin et al., "Trajectory balance: Improved credit assignment in gflownets", NeurIPS 2022.

[6] Zhang et al., "Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization", ICLR 2024.

[7] Pan et al., "Generative augmented flow networks", ICLR 2023.

[8] Kim et al., "Adaptive teachers for amortized samplers", ICLR 2025.

---

We hope our responses have addressed your questions and concerns. Please let us know if there is anything we may have missed or if any points require further clarification.

Dear Reviewer DVcd,

Thank you for your thoughtful and encouraging review. We greatly appreciate your recognition of the clarity of our writing, the importance of our field, and the significance of our experimental results. Below, we address your comments in detail.

W1: It would be good to compare with some papers that target this task, e.g., the baseline from Midgley et al. [1].

Thank you for the suggestion. We did not include FAB [1] in our comparison because, as noted in [6], FAB cannot amortize sampling and relies on annealed importance sampling, which demands many energy evaluations. Instead, we compare against Adjoint Sampling method from [6], which also targets small peptides like Alanine Dipeptide. As shown in Table 1, when the model is trained solely on Alanine Dipeptide, our method outperforms Adjoint Sampling.

Table 1. Comparison of Wasserstein-2 distance of positions and energy histogram on Alanine Dipeptide.

---

W2: Unclear novelty in combining two existing methods.

We resonate with your concern, but we also believe that our method is not just a combination of two methods. The diffusion‑sampling field sits at the crossroads of diffusion models, probabilistic inference, stochastic control, and reinforcement learning, so progress often comes from properly hybridizing techniques that each community describes in its own language.

• Prior work has imported existing stochastic‑control formulations into diffusion models [1],
• utilized existing replay buffer auxiliary [6],
• applied existing local credit‑assignment tricks [7] from RL, and
• mapped existing search algorithms onto diffusion sampling [8].

Our contribution follows this line by combining sample-efficient RL techniques with MCMC inside a diffusion sampler, which is not previously explored and therefore a meaningful addition to the literature.

---

Q1: Does the paper's limited consideration of real-world datasets reflect what is standard in the literature?

Yes. In fact, our work expands the standard benchmarks [7-9], i.e., GMM, Lennard-Jones, and Manywell, by additionally including Alanine Dipeptide experiments. In the future, we expect researchers to further the algorithms and apply them to real-world problems, e.g., fine-tuning biomolecular foundation models like BioEmu [10] with energy function evaluations.

---

Q2: In the limitations, it is discussed that calibration of alpha is important. How simple is the calibration of RND weight?

We simply choose from four possible RND weights (1, 10, 100, 1000) based on the performance at 20% of the second round, while the performance has a dependency on the RND weight.

To alleviate your concern, we conducted experiments across approximately four different RND weight values per task to ensure stable training and meaningful results in Table 2.

Table 2. ELBO-EUBO gap at 20% of the second round for Manywell-128, LJ potentials, and ALDP.

We found that reasonable values could be selected with minimal tuning, though a more systematic approach could be explored in future work.

---

[1] Midgley et al., "Flow annealed importance sampling bootstrap", ICLR 2023.

[2] Noe et al., "Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning", Science, 2019.

[3] Jing et al., "Torsional diffusion for molecular conformer generation", NeurIPS 2022.

[4] Hassan et al., "Et-flow: Equivariant flow-matching for molecular conformer generation", NeurIPS 2024.

[5] Havens et al., "Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching", ICML 2025.

[6] Akhound-Sadegh et al., "Iterated Denoising Energy Matching for Sampling from Boltzmann Densities", ICML 2024.

[7] Zhang et al., "Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization", ICLR 2024.

[8] Sendera et al., "Improved off-policy training of diffusion samplers", NeurIPS 2024.

[9] Zhang et al., "Path Integral Sampler: a stochastic control approach for sampling", ICLR 2022.

[10] Lewis et al., "Scalable emulation of protein equilibrium ensembles with generative deep learning", Science 2025.

---

We hope our responses have addressed your questions and concerns. Please let us know if there is anything we may have missed or if any points require further clarification.

Dear Reviewer Kvbn,

Thank you for your thoughtful and encouraging review. We appreciate your comments on the novelty of the methodological incorporation and several evaluations of our work. Below, we address your points in detail.

W1: It seems like the MCMC-based searcher might not scale well for expensive energies. It would be good to see neural network energy examples other than Alanine Dipeptide and their run-times.

To resolve your concern, we provide additional results on Alanine Tripeptide and Tetrapeptide considered in Tan et al.[1].

Table 1 shows that the searcher is not the bottleneck for scaling to larger systems. Table 2 shows that our method still outperforms MLE (or forward KL) training on samples produced by the searcher. All model architectures and training settings for the tripeptide and tetrapeptide follow those used for the dipeptide, with the only change being batch sizes of 8 for the tripeptide and 4 for the tetrapeptide (versus 16 for the dipeptide). Upon further inspection, we observed that our model successfully generates low‑energy conformations, and we plan to include 3D visualizations of these structures in the future manuscript.

Table 1. Runtime (minutes) of neural network energy for a single round using NVIDIA GeForce RTX 3090.

Table 2. Performance comparison with MLE on Alanine Tripeptide with the same energy calls (1M).

Table 3. Performance comparison with MLE on Alanine Tetrapeptide with the same energy calls (1M).

---

W2-1: The Searcher exploration reward with RND won't prevent the previously visited mode from being lost once it has been visited.

It seems that you are concerned about the following case: when the RND method assigns too small exploration reward for a previously visited mode, the Learner loses this mode during training.

Our algorithm avoids this issue for several reasons. First, in our training, the RND reward decays slowly even for previously visited modes, so that the Learner properly learns the mode before the RND reward becomes negligible. Next, we use a top-k priority buffer to keep the low energy states, hence the Learner does not lose the mode. Finally, even if the RND reward becomes negligible, the Searcher will have a relatively high probability of re-visiting the mode due to its low energy.

W2-2: In the ablation on Manywell 128, “w/o RND-searcher” performs only marginally worse than with RND.

The improvement is significant since it is in log scale. In the ablation on Manywell 128, the RND searcher improves ELBO by $6.4$ of ELBO, which actually represents $\exp(6.4) \approx 403.43$ times better performance in lower bounding the marginal probability of the Learner generating the true sample.

---

Q1-1: Since the ground-truth is also generated by some MCMC variant, why wouldn’t we just do flow/score matching onto the MCMC samples and achieve better performance?

Our algorithm outperforms flow/score matching on MCMC samples since our online learning collects informative samples in a sample-efficient manner with a good exploitation-exploration trade-off. That is, once the Learner is sufficiently trained, the Learner generates low-energy samples that are hard to collect from short MCMC trajectories (exploitation), while the Searcher is still necessary to collect samples from unseen modes (exploration).

We already verified this in Figure 2-$c$, where TB loss outperforms MCMC + MLE equivalent to score matching on the MCMC samples. Also, we show the debiasing effect of TB on Manywell distributions in Figure 6 in Appendix B.2.

Q1-2: How good do these proposal samples from MCMC need to be to effectively learn a diffusion sampler with the TB loss?

For Manywell 128 case, 300 parallel chains for Annealed Importance Sampling (AIS) with length 100 are enough to help the model learn well. For LJ-55, a single chain for Metropolis-Adjusted Langevin Algorithm (MALA) with length 10K is enough to learn well. For Alanine Dipeptide, four parallel chains of high-temperature molecular dynamics (MD) with length 110K are enough to capture diverse modes and train the learner well.

---

Q2: A run-time comparison would be helpful, especially for the case of Alanine Dipeptide, which uses the neural network energy function.

We present the table of runtime of the searcher and the learner in Alanine Dipeptide. Table 1 shows that the run-time of Searcher is marginal compared to that of Learner.

Table 4. Runtime (minutes) of neural network energy using NVIDIA GeForce RTX 3090. Note that the first round searcher requires computing only neural network energy (51 minutes), but the second round searcher utilizes the RND network additionally (128 minutes).

---

Q3: How is the ELBO evaluated for diffusion samplers? It’s my understanding that for samplers like PIS, you use the path-weights via the Radon-Nikodym derivative, which is not exactly the model-likelihood of the sample $x_1$.

We evaluate the ELBO by the following steps:

• Compute $\log P_F(x_{t+\Delta t}|x_t ; \theta)$ along the forward path with the drift $u_\theta(x_t,t)$ and the noise schedule $\sigma$, and $\log P_B(x_t|x_{t+\Delta t})$ along the Brownian bridge for $t\in[0,1]$.
• Average $\log\frac{R(x_1)\sum P_B(x_t|x_{t+\Delta t})}{\sum P_F(x_{t+\Delta t}|x_t ; \theta)}$ along the trajectories generated by the Learner.

We follow prior works that use the ELBO as a performance metric [2-4] as a lower bound to the model likelihood of samples.

---

[1] Tan et al., "Scalable Equilibrium Sampling with Sequential Boltzmann Generators", ICML 2025.

[2] Sendera et al., "Improved off-policy training of diffusion samplers", NeurIPS 2024.

[3] Kim et al., "Adaptive teachers for amortized samplers", ICLR 2025.

[4] Vargas et al., "Transport meets variational inference: Controlled Monte Carlo diffusions", ICLR 2024.

---

We hope our responses have addressed your questions and concerns. Please let us know if there is anything we may have missed or if any points require further clarification.

I want to thank the authors for their clarifications and their efforts for addressing my concerns.

One last question regarding the result about MCMC+flow-matching, so I can understand it correctly. Shouldn't just running MCMC longer completely solve LJ55 since that is what generated the ground truth? I would think that fitting a generative model to MCMC data run on a longer chain would eventually do better than your proposed method. So your result is showing that with some fixed energy call budget, your method performs better?

That being said, I will raise my score accordingly.