# Rebuttal Reviewer S3Vr

We would like to thank the reviewer for their time, feedback, and positive appraisal of our work. We are heartened to hear that the reviewer feels that the “contribution made by this paper should significantly change the course of future work in this area.” We also thank the reviewer for acknowledging that the main claims of our work are “well supported by experiments”. We now address the questions and suggestions raised by the reviewer in order, and note that additional results are included in this link: https://anonymous.4open.science/api/repo/sbg/zip

ESS after SMC and SBG without resampling

The reviewer makes an astute observation that using SMC with adaptive resampling, ESS can be maintained to an artificially high value. We agree that such a metric in this case is not meaningful, and we had opted to include this to be in line with the broader literature, e.g. in pure sampling NETS includes ESS after SMC. We will update the paper to remove ESS after SMC and, as the reviewer suggested, include SBG without resampling (SBG-AIS) which is now included as another row in our rebuttal document Tables 1 and 2. We find that SBG with AIS outperforms the previous SOTA which is ECNF and our introduced ECNF++ for AL3-AL6 across sample-based metrics $\mathcal{T}-\mathcal{W}_2$ and $\mathcal{E}-\mathcal{W}_1$, and is slightly worse in terms of ESS to ECNF++.

We thank the reviewer for allowing us to strengthen and clarify our empirical results, and we hope this new SBG-AIS variant alleviates this particular concern.

Wasserstein distances of proposal w/o reweighting

We thank the reviewer for this insightful idea. We have included Table 3 in the rebuttal link to quantify the performance of our proposal without reweighting, with importance sampling i.e., just BG, and with SBG. We observe a drastic improvement in $\mathcal{E}-\mathcal{W}_1$, for IS over the proposal, and an even greater improvement with SMC. We will include these results in our updated draft.

Wasserstein in TICA space

Thank you for the suggestion! We have now included in the rebuttal experiments (Tables 1, 2 and 3) TICA Wasserstein metrics for all AL3+. We find SBG outperforms all baselines (ECNF/ENCF++) in this metric. We will include these results in our updated draft.

High energy structures

That is a very interesting question. Upon further investigation we found that the highest energy AL6 samples result from steric clashes, as visualized in Fig 4. In Fig 5 we show the histogram and log histogram of shortest non-bonded interatomic distance between MD (ground truth) and SBG samples. An additional source of high energy samples is covalent bonds of insufficient length (Fig 6). We will include these results in our new appendix.

Mixing energy-based training

We thank the reviewer for this great suggestion. We have experimented with mixing energy-based training with normal data-based training and find preliminary evidence that this leads to better performance on $T-\mathcal{W}_2$ and the newly introduced TICA metric but marginally worse $\mathcal{E}-\mathcal{W}_1$ vs normal SBG (Tables 1-2). Given these promising initial results we will run a full set of experiments using this method and report these in the updated paper.

Transferable BG

We appreciate the reviewers' comments that operating in Cartesian coordinates more easily leads to training transferable models. In this work, our primary focus was on proving the scalability of SBG as it relied on normalizing flows and inference scaling through SMC. Consequently, we believe that extending this to transferable systems is a natural direction of future work, but out of the scope of the current paper.

Other questions

The definition of T-W2 distance is not clear.

This is a multi-dimensional Wasserstein distance on torsion angles, accounting for the torus geometry of the angle space.

100k vs 10k samples

The subsampling is done after the final SMC step at the end of inference.

Proposition 1

To give further empirical credibility to CoM adjustment, we perform additional ablations in rebuttal Fig 2 and 3. We first find that the $||C||$ of the proposal samples does indeed follow an approximate Chi distribution, as expected given the training data augmentations. We also find that CoM adjustment is both important for stable IS reweighting (with a large but finite number of samples) and that sample metrics improve when the adjustment is employed in SMC.

We will update the proposition statement and proof to better convey our theoretical result.

Closing comments

We would like to thank the reviewer for their time and effort. We hope all our answers here allow the reviewer to continue to positively endorse our paper, and we would love to have the opportunity to clarify any lingering questions should the reviewer have them.

# Rebuttal Reviewer HyU2

We thank the reviewer for their time and effort. We are glad that the reviewer found our empirical results to “demonstrate the effectiveness” of our method SBG. We next clarify the main points raised in the review and note that additional results are included in this link: https://anonymous.4open.science/api/repo/sbg/zip

Novelty

We acknowledge the reviewer's concern that SBG can be thought of as an application of NETS-style inference on a learned proposal. We first highlight that our approach differs from NETS as we do not learn an extra drift term through an auxiliary loss e.g. A PINN objective. In NETS, such a term is crucial to reduce the variance of the importance weights during a linear interpolation. However, SBG does not need to employ this computationally intense learning objective precisely because of our learned proposal. In contrast to an uninformative proposal, which has a very small overlap with the target, a learned proposal mitigates the need to learn a drift term. Thus, we would like to politely push back against the assertion that SBG is a simple instantiation of NETS, as it in fact SMC with a computationally efficient manner to perform Langevin dynamics that is unlocked by using a normalizing flow rather than an equivariant CNF which we argue is a novel insight that we exploit in the BG context.

With regards to our framework, we again would like to politely disagree with the reviewer as our design choices fundamentally challenge the direction of BG research. More precisely, our main technical novelty lies in demonstrating the scalability of non-equivariant classical flows in contrast to the dominant trend to leverage equivariant CNFs. In addition, we include new algorithmic novelties such as CoM adjusted resampling; rebuttal Fig 2 and 3 highlight the improved stability and performance of IS / SMC when accounting for CoM augmentation. Furthermore, the overall scalability of our method to hexapeptides is a novel result, and is due to a combination of each component in SBG: 1) a non-equivariant classical flow and 2) SMC that leverages the exact energy of a classical flow. We highlight that these findings enabled Reviewer S3Vr to remark in their review, “The contribution made by this paper should significantly change the course of future work in this area.”

Finally, we highlight that our paper contains several theoretical results that provide quantification of bias added through various thresholding schemes. Such schemes have routinely been employed in existing literature without justification or analysis. Our paper is the first to provide an exact characterization of the bias added—allowing practitioners to negotiate a select a problem-dependent thresholding value.

Training details

We appreciate the reviewer's comment regarding the training details for training the proposal flow. Whilst many of the training, inference and dataset details are included in Appendices D and E, we recognize the importance of including details in the main paper. We will update this in future versions of the work to include additional details in section 4 but briefly state that for all TarFlow models we train for 1,000 epochs with lr 1e-4 weight decay 4e-4 using the AdamW optimizer with a cosine lr schedule and EMA with decay 0.999. Furthermore, directly answering the reviewer's question concerning training samples, Appendix E includes a description of dataset construction from MD trajectories for each of the peptide systems. For each system we use a training set of 100k contiguous samples from a single MD trajectory. We understand that aspects of such details are important to be included directly in the experiments section, and we will revise the paper in the next draft to include the key details in the main body.

Background on CNF

We value the reviewer's feedback that the discussion of CNFs might seem ancillary to a paper that leverages classical normalizing flows as the model. The key reason to introduce CNFs is to illustrate that our inference time scaling through SMC, while theoretically possible using a CNF proposal, faces significant challenges in scalability due to the need to simulate the ODE using equation 4 to compute $\log p_{\theta}$ for every energy evaluation along the interpolation used in Langevin dynamics. In contrast, our TarFlow requires only 1 call for each Langevin step. However, we understand that this section may not have been as tightly integrated into the paper, and we will improve the clarity of presentation in the updated draft.

Closing comments

We thank the reviewer again for their questions, which allowed us an opportunity to clarify our paper. We hope that our rebuttal responses fully address all the important questions raised by the reviewer, and we kindly ask the reviewer to potentially upgrade their score if the reviewer is satisfied with our responses. We are also more than happy to answer any further questions that arise.

# Rebuttal Reviewer 9RzA

We thank the reviewer for their thoughtful comments and feedback. We value that the reviewer found that most of our claims were supported by “clear and convincing evidence” and that our mathematical claims were “theoretically sound and mathematically consistent”. We are also glad that the reviewer found our use of SMC in BGs to be “novel” and allows for “scalability,” which is supported by our “insightful” comparison on computational efficiency. We next address all the salient points raised in the review and note that additional results are included in this link: https://anonymous.4open.science/api/repo/sbg/zip

(Auto) Correlation of samples

We acknowledge the reviewer's request for the inclusion of autocorrelation analysis. However, we emphasize that the samples generated by SBG can not suffer from autocorrelation. This is because only samples from $\tau = 1$ are returned as generated samples, hence there is no time dimension on which autocorrelation could exist. This is in contrast to methods such as MD in which a single trajectory is returned as generated samples, and samples may exhibit autocorrelation over sampling time.

To bolster our study, we have included a new variant of SBG (SBG-AIS) that does not perform any resampling during the annealing process (See Tables 1 and 2). This ablation principally serves to eliminate the correlation that SMC might introduce during inference scaling. As we observe SBG-AIS, it achieves acceptable ESS and outperforms other baselines in sample-based metrics. We hope that this makes the impact of correlation during inference clearer in SBG.

Benchmarking MCMC

We thank the reviewer for the suggestion of additional baselines. We however, are unsure exactly which MCMC methods they believe to be suitable for benchmarking on the evaluation systems. We note that the goal of BGs is to amortize sampling and train a model which is able to quickly draw many uncorrelated samples. This is a fundamentally different approach with both benefits and drawbacks as compared to MCMC methods.

Evaluating SBG beyond peptides

We value the reviewer's feedback as exploring SBG’s generalizability to experimental settings beyond peptides considered in this paper is interesting. We first note that before SBG, modern BG’s utilized equivariant CNFs instead of classical normalizing flows that employed invertible architectures. This was due to the widely held belief that equivariant CNFs were more expressive and more amenable to larger-scale problems of interest. Despite this promise, these equivariant CNFs—as demonstrated in our ablations—struggle on datasets beyond AL3/4 and are much slower to evaluate due to the need for simulation. Thus, we argue that even demonstrating that a classical normalizing flow paired with an inference scaling strategy allows tackling even larger peptides in hexapeptide (AL6) is extremely interesting. These results demonstrate the expressive power of our framework in comparison to previous BGs.

Such results enabled Reviewer S3Vr to highlight in their review, “The contribution made by this paper should significantly change the course of future work in this area.” As a result, we believe the experimental validation of our introduced SBG approach is well supported in the current peptide datasets and testing on other chemically diverse systems, while extremely interesting, is beyond the scope of the current paper, but remains an exciting direction of future work. At this time we don’t anticipate any limitations specific to chemically diverse systems such as small organic molecules or metal-organic frameworks, but chose to evaluate on peptides as has been the norm in many prior BG works.

Closing comments

We thank the reviewer for their time and effort in reviewing our work, and we hope the reviewer will kindly consider a fresh evaluation of our work, given the main clarifying points outlined above. We are also eager to engage in further discussion if the reviewer has any lingering doubts.

Rebuttal Reviewer VNPc

We thank the reviewer for their time, feedback, and nuanced comments. We are glad that the reviewer found the non-equivariant NF an “interesting alternative” which is “necessary as we scale up to even larger systems and datasets”. We also appreciate that the reviewer recognized our use of SMC instead of IS as a “smart drop-in design choice”. Finally, we are heartened to hear that the reviewer agrees that Prop 3 “is a principled technique”. We now address their key questions raised in the review and note that additional results are included in this link: https://anonymous.4open.science/api/repo/sbg/zip

iDEM as a Baseline

We appreciate the reviewers' valuable comments regarding iDEM as an additional baseline. We would like to first politely recall the setting of iDEM and other diffusion samplers as completely data-free (i.e. there is no training set), in contrast to the BG setting which includes training on (biased) data. This allows BGs to scale much more easily than the data-free amortized samplers. Consequently, we argue that scalability claims in each setting cannot be meaningfully compared. To our knowledge only one sampling method has successfully scaled to any molecular task, which is FAB on ALDP using an intrinsic coordinate system, while molecular tasks are the focus of most works on BGs. To our knowledge, no sampling method has successfully scaled to any molecular task on cartesian coordinates, the focus of this work.

To investigate this setting, we had private correspondence with the iDEM authors, who told us that iDEM failed to scale to ALDP even with substantial effort; to their knowledge no diffusion-based sampler can scale to molecular tasks. Following the reviewer's suggestion we trained iDEM on ALDP ourselves. In this case we use a vacuum instead of the implicit solvent used in our main results to speed up training (see Fig 1 in our link). We observe that iDEM is unable to successfully sample in this easier setting with most modes missing and a poor energy distribution.

Novelty

We value the reviewer's feedback that the application of SMC, and a general-purpose Normalizing Flow may initially appear to have limited technical novelty. We would like to politely push back against this assertion, as our framework and design choices fundamentally challenge the predominant approach in BG. Prior to SBG, all modern BGs in Cartesian coordinates have resorted to equivariant CNF’s; the fact that we can omit both exact equivariance and use an exact NF—is a novel insight. Moreover, our CoM adjustment strategy is also new. We ablate its importance in new plots in the rebuttal link (Fig 2 and 3), which shows that IS reweighting significantly benefits from this adaptation strategy.

In fact Reviewer S3Vr notes, “The contribution made by this paper should significantly change the course of future work in this area.” We further wish to highlight that the paper contains numerous theoretical results on thresholding, some of which are utilized in existing papers without proper justification. Our theoretical results allow us to quantify the impact of thresholding schemes in this process. We hope that the reviewer may join us in agreeing that the design choices utilized in SBG allow for a fresh approach to building BG’s using non-equivariant components with soft penalties that demonstrably scale better to larger peptides in Cartesian coordinates—which remained an open challenge until SBG.

Additional References

We acknowledge the reviewer's comment regarding the inclusion of more non-BG based samplers. We will update the paper with a dedicated discussion on non-BG based sampling and include the references suggested by the reviewer in FAB, PIS, PDDS, and the concurrent work SCLD.

LJ Potential

The reviewer is correct to highlight that many previous samplers and BG papers also evaluate on LJ potential systems. Such systems are however less challenging than even small peptides (e.g ALDP), making them of limited interest once peptides can be successfully tackled. We thank the reviewer for their suggestion that this may help better place the work, but instead draw attention to the evident failure of iDEM on ALDP in Fig 1 as clear evidence that BG methods are superior in the data-available setting.

Computational expense

We thank the reviewer for enquiring regarding the efficiency of our method with respect to energy / force evaluations. During sampling a single force evaluation is required per-particle per timestep, hence if the force is expensive this will present a computational cost.

Closing comments

We thank the reviewer for their valuable feedback and great questions. We hope that our rebuttal fully addresses all the important points raised, and we kindly ask the reviewer to potentially upgrade their score, as they indicated, if they are satisfied with our responses. We are also more than happy to answer any further questions that arise, please do let us know!