Molecular Conformation Generation via Shifting Scores

Weakness

1. Mischaracterization of prior works.

In GeoDiff, authors stated that under the common assumption that $d$ also follows a Gaussian distribution (p7 Chain-rule approach before Sec 4.4). GeoDiff injects the noise on the coordinates and then transforms the distribution of the coordinates into the distribution of the distance. By such a transformation, GeoDiff attains the SE(3) equivalence. We suggest reviewer go through the official code of GeoDiff to have a better understanding of GeoDiff's implementation.

2. Missing baseline.

We strongly suggest reviewers go through the whole paper more carefully. We mentioned Torsional diffusion in Introduction (paragraph 2, line 3-4) and Related work (line 6). Even in Appendix F, we use a full page to discuss our work and Torsional diffusion. Our modeling method is quite different from Torsional diffusion. Our model is purely data-driven while Torsional diffusion uses strong chemical prior. We think that chemical prior can greatly enhance the model's performance. In [1], the authors claim that using the RDkit and a simple clustering algorithm can surpass most of the deep learning models. Hence, we think that including Torsional diffusion is not fair and did not include modeling methods that use strong prior in the comparison.

3. Weird theoretical assumption

We plot the case when $\sigma=50$ as an extreme example to show that when $\sigma$ is sufficiently large, the distribution indeed trends to the MB distribution. In practice, the maximum $\sigma$ is chosen to be round $12$. We can use the Figure 2 when $\sigma=10$ as an illustration. In this case, the distribution is still close to the MB distribution.

4. Unconvincing results.

The main contribution of this work is to propose a new shifting distribution for a more accurate approximation of the distance distribution, we did not greatly change the modeling method (i.e., model architecture, modeling manifold) for a fair comparison. Hence, the improvement may not seem to be very large. About the benchmarks released in [2], it is essentially the same as the benchmarks in our paper but with a different error tolerance. We set a toy Drugs dataset that only contains 1/10 of the original training set and the same test set as SDDiff. We trained 2 models whose distance distribution hypotheses are Gaussian and the shifting distribution, respectively. The results can be seen in the following table:

5. Unexplained analysis

To clarify, in this work, we do not apply the diffusion on distances. We apply diffusion on coordinates, model the distribution of distances, and then transform this distribution back to the distribution of the coordinates. Since it is extremely difficult to model the dependence of the distances, hence, we hypothesis that this dependence has a limited infect on the final results. In the introduction, our motivation is that by applying diffusion on coordinates, the relative ``speed'' of atoms can be considered as the MB distribution which is the distribution of the speed of particles when particles move freely with the bare occurrence of collisions.

6. While I think diffusion on distances rather than particles is interesting for ML on molecules, the formulation in this work confuses me of why it is beneficial (if at all). The results are not convincing and prior works are either mischaracterized or left out. Due these issues, I recomend reject.

We believe that reviewer XVGd has mischaracterized the work of both GeoDIff and ours. We went through the official code of the GeoDiff and we are sure that GeoDiff indeed applies the assumption that the distribution of the distance follows the Gaussian distribution. Also, in this work, we did not apply the diffusion to distances. As indicated by unnumbered equations in p5, we directly inject noise on the coordinates and then model the distribution of the distances, which is similar to GeoDiff's implementation. However, we design a more accurate approximation of the distance distribution, which we believe can enhance the model's performance. The results in the ablation studies show that when the dataset is small, such a more accurate approximation of the distribution can greatly enhance the performance.

[1]https://arxiv.org/pdf/2302.07061.pdf [2]https://arxiv.org/abs/2206.01729

Weaknesses

The main weakness from my perspective is the significance of empirical comparison. The improvement over GeoDiff is not significant to me. Could the author provide more ablation study about the function in Eq7, which can help to verify the importance of the proposed MB diffusion distribution.

SDDiff outperforms GeoDiff greatly when the dataset is small. We set a toy Drugs dataset that only contains 1/10 of the original training set and the same test set as SDDiff. We trained 2 models whose distance distribution hypotheses are Gaussian and the shifting distribution, respectively. The results can be seen in the following table:

Questions

I may miss some details, but feel a little confused about defining the diffusion process on the distances. The motivation is "Gaussian on coordinates results in MB distribution on distances". Then, why not add noise on coordinates which can also enable the MB diffusion on distances? I feel the direct perturbation in distances will also potentially result in infeasible geometry?

When the noise is injected into the coordinates, if the noise is small, then the distances follow a Gaussian distribution as proposed by GeoDiff. However, if the noise is large enough, the distribution of the distances follows the MB distribution. That's why we proposed a shifting distance distribution between Gaussian and MB distribution for a more accurate approximation of the exact distribution. Directly injecting noise on the distance results in infeasible geometry. This method was proposed in ConfGF, which was mentioned in related works. From the experimental results, SDDiff and GeoDiff are better than ConfGF.

Weakness

While the observation is interesting and it's great that the authors are able to demonstrate its superior performance, the GEOM benchmark has been used for quite some time now and probably over-optimized, so it's difficult to argue true superiority marginal gain on one benchmark alone. In addition, the majority of the mathematical framework for score matching involving langevin dynamics are not new to this problem either.

In this work, we are not mainly aiming to propose a new baseline model in the task of conformation generation task. We are considering how to model the distance distribution if we apply the diffusion process to coordinates. Such modeling does not only contribute to molecular generation tasks, but can also be applied to any generation task that takes SE(3)-invariance as an important property. We use the GEOM benchmark mainly because many previous works model the distribution of distance such as ConfGF, and GeoDiff. Hence, we compare these models of the distance distribution and demonstrate our distribution approximation is better compared with theirs. For a fair comparison, since other models apply the Langevin dynamics for generation, we still apply the same sampling method.

Questions

I am happy to re-evaluate my rating if the authors can provide more compelling evidence that the proposed methods are not just another conformer generation model. For instance, showing superior downstream application impact would be very helpful.

Thanks for your suggestions. Our method does not require any prior knowledge and pre-processing, thus it can be applied to any other tasks in SE(3) space like point cloud. Actually, the generalization of this method is what we plan to do in the next step, we have tried a toy example to generate simple shapes such as cubes without any other input, and it indicates that this method can be generalized to other scenarios.

I appreciate the reviewers' response and agree that reframing the paper to focus and motivate a new kernel for modeling distance distribution would greatly strengthen the paper, especially if the authors can include more experiments and results in other problem settings since these issues are also brought up in other reviewers' feedback.

However, I cannot change my rating as the paper currently stands. I hope reviewers' feedback here can help the author iterate further.

Questions

1. Especially in the introduction several papers are missing the year of publication, e.g., Xu et al., Jing et al, Zhu et al., it is therefore not clear which paper is being cited and if multiple occurrences denote the same paper.}

Thanks for your pointing out it. We have revised the references.

2.What do the authors mean by marginal distribution of interatomic distances? If the full set of 3N(3N-1)/2 number of distances are included, this distribution would be even higher dimensional than the 3N-dimensional Boltzmann distribution of Cartesian coordinates.

We refer $p(d_{ij})$ as the marginal distribution of interatomic distances where $d_{ij}$ is the distance among the $i$-th node and the $j$-th node. Indeed such modeling would increase the modeling manifold but still this is the so fared best choice to avoid directly dealing with the SE(3)-invariance in Cartesian coordinates.

3. It is not clear how equation 7 follows from 6 nor how it justifies to “simply” use a Gaussian kernel.

Eq.6 gives constraints that the distribution of perturbations on distance is proportional to the normal distribution when $\sigma=\sigma_0$, and proportional to MB distribution when $\sigma=\sigma_T$ We construct $f_\sigma(\tilde{d}, d)=1-e^{-\sigma / d}$, then substitute it into $p_{\sigma_t}(\tilde{d} \mid d) \propto \tilde{d}^{2 f_\sigma(\tilde{d}, d)} e^{-\frac{(\tilde{d}-d)^2}{4 \sigma_t^2}}$ and then we can obtain the score function shown as Eq.7 (the $\nabla_{\tilde{d}} \log q_\sigma(\tilde{d} \mid d)$ should be $\nabla_{\tilde{d}} \log p_\sigma(\tilde{d} \mid d)$, we have fixed it).

The justification to "simply" use a Gaussian kernel is detailed after Eq 7 in those unnumbered equations. We perturb the coordinates with Gaussian noise, then the conditional perturbed inter-atomic distance are Gaussian and MB distributions.

4. On the bottom of page 5, where the authors state that n has to be greater-equal to 5. It would be good to mention there that “Each individual atom must be associated with a minimum of four distances, in order to establish isomorphisms between …”}

Thanks for your suggestions, We are open to revisiting and refining the wording here in accordance with your suggestions.

5. The implications of the “Note…” after eq 9b remain unclear.

Here, we are trying to remind readers that the score function of conformation is not simply $-\frac{\tilde{\mathcal{C}} - \mathcal{C}}{\sigma^2}$. This is because the considered probability density function has the SE(3)-equivalence. Hence, for example, if $\tilde{\mathcal{C}} = \mathcal{C} + t$, where $t$ is a scaler indicating that we move the atoms in $\mathcal{C}$ along x, y, z axis by $t$. In the conditional states of $p(\tilde{\mathcal{C}} \mid \mathcal{C})$, $\tilde{\mathcal{C}}$ and $\mathcal{C}$ represents the same state since we are considering in a SE(3)-invariant space. Hence, the score function of $p(\tilde{\mathcal{C}} \mid \mathcal{C})$ cannot be easily computed. That's why we transform the score function of distances to the score function of conformations.

6. The paragraph on optimal transport, says that the authors use the regularized Wasserstein barycenter but then continue saying that it is not suited. So what exactly do the authors do?

We are approximately the distribution between the Gaussian distribution and the MB distribution. Hence, one may consider using the OT to solve the intermediate distribution. However, we discussed that using this method has numerical stability issues. Hence, we are using the proposed methods in our paper to estimate the intermediate distribution. Discussion about OT is to save time for those trying to solve the intermediate distribution by OT method.

7. The caption of Table 1 is too short. The caption should at least explain the meaning of COV and MAT and refer the reader to their definition in the text.}

Thank you for your suggestions. We have expended it.

8. Why exactly is planarity a problem? Don’t 4 neighbors define any point in 3d exactly?

Consider the case where 4 points are all in the same plane. You may consider there are 4 points in an A4 paper. Then, given the distances between them and the other additional points, you cannot decide whether this point is on the top of the A4 paper or the downside of the A4 paper.

9. In Section 5.1, The MAT(-R) metric should be corrected as follows: MAT=1/|S_r | ∑_(C∈S_r)min┬(C^'∈S_g )⁡RMSD(C,C^' ) .}

Thank you so much for pointing it. We have made corrections.

Weakness

1. Probably my biggest problem with the paper is the motivation for using the Maxwell- Boltzmann distribution (MBD). The MBD describes the distribution of the length of velocity vectors in an ideal gas, and to some good approximation even in a real molecule. The authors generate molecular conformations based on distances, velocities are never generated nor used (e.g., the math on page 5 after eq 7 only includes distances). It is also unclear how interatomic distances (which are by definition purely positive) or the length of velocities could ever follow a Gaussian distribution. Therefore, the appearance and meaning of velocity v in eq 5 needs to be explained clearly.

We are not claiming that the distances between atoms follow the Gaussian distribution, we are proving that under small perturbations, the change of the distances, i.e. $\tilde{d} - d$ follows the Gaussian distribution. Also, $d$ nor $\tilde{d}$ is relative to the concept of speed. However, $\tilde{d} - d$ is relative to the speed since it measures the change of the distances.

2. Regarding Section 4.5, if a molecule were to break apart, there will be some correlation of atom-atom distances, i.e., if an atom's distance to another one which is far away in the graph increases this will almost certainly also mean that the distance of the neighboring atom to the far away atom increases. This is more correlation than causation, but it does contain information. Additionally, instead of hypothesizing can the authors at least show empirically that their statement is true?

We generate samples $\tilde{d}_1, \tilde{d}_2, d_1, d_2$, where $d_1, d_2$ are the edge lengths of two incident edges and $\tilde{d}_1, \tilde{d}_2$ are edge lengths after perturbation on coordinates. We compute the Pearson r correlation between random variables $\tilde{d}_1 - d_1$ and $\tilde{d}_2 - d_2$. The results show that under different levels of $\sigma_t$, the correlation coefficient is always round 0.24. Hence, we ignore such a correlation.

3. Regarding the measures COV and MAT. If we assume that a molecule has, e.g., 3 major conformers which are all very close in RMSD. Wouldn’t a model that samples always only one conformer achieve a COV of 1, even though it has never generated the other 2? Also, the definition of MAT seems odd, is there sum over S_r and maybe a min() missing?

The COV and MAT measures can cause this problem. We notice that COV-P and MAT-P can reflect the quality of all generated conformers, we will include these metrics later. And thanks for pointing out the wrong definition of MAT in our paper. It should be $\operatorname{MAT-R} =\frac{1}{|S_r|} \sum_{\mathcal{C} \in S_r} \min_{\mathcal{C}' \in S_g} \operatorname{RMSD} (\mathcal{C}, \mathcal{C}')$. we have revised it.

4. On page 9: The authors write that it is evident that the proposed distribution matches the Gaussian closely, however, in Fog. 4 the orange distribution appears to be tri-modal Can the authors compute the overlap of orange and blue for the two values of sigma?

The orange distribution is not tri-modal, it is unimodal, and the distribution is very close to Gaussian.

5. "In Section 4.2, even though D is defined as image(d), R^(n×3)/SE(3) is not isomorphic to D: a molecule and its mirror image would have the same distance matrices, but they are not the same element of R^(n×3)/SE(3) if they are chiral. In this regard, it would be valuable for the authors to clarify how the proposed method handles the generation of conformers for stereoisomers or enantiomers."

When we say the distance matrices, we are referring to the adjacent matrix, i.e., we consider a fully connected graph. Although two chiral molecules have the same set of bound lengths, their adjacent matrices are different. Hence, $D$ is isomorphic to $\mathbb{R}^{n \times 3} / \operatorname{SE(3)}$ if we define $D$ to be the manifold of adjacent matrices (not the manifold of bond length).

6. In Section 5.1, the COV and MAT metrics introduced correspond to the “Recall” version (COV-R and MAT-R, without the typo mentioned below). Some of the baseline methods under comparison, such as GeoDiff (Xu et al., 2022) and Torsional Diffusion (Jing et al., 2022), also include the “Precision” version (COV-P and MAT-P) to assess the quality of the generated conformers. The authors should include the “Precision” metrics in Table 1 as well.

Thanks for your suggestion. we will include these metrics later.

Could the authors explain what the TD and clustering comparison is trying to explain in relation to SDDiff? What is the issue with using RDkit as a strong prior? If the task was molecular design, then using RDkit to initialize would be cheating but that is not this. I am not convinced TD and SDDiff cannot be compared quantitatively on this task.

Where are these numbers from? These rdkit+clustering numbers do not match what's reported in Table 1 of Zhou et al. For TD, why are the recall numbers used for QM9 while the precision numbers are used for Drugs?