DMol: A Highly Efficient and Chemical Motif-Preserving Molecule Generation Platform

We thank the reviewer for the constructive suggestion.

FragFM is indeed a very recent paper that has not been published yet (to the best of our knowledge). We were unable to locate publicly available code for FragFM, which limits our ability to conduct a direct empirical comparison at this time. We will definitely add a citation to FragFM in our revised manuscript and acknowledge it, and, if time permits, try to implement their software. Also, upon reviewing the FragFM paper content, we note that while both DMol and FragFM address molecular generation, they employ fundamentally different methodologies: FragFM utilizes fragment-level discrete flow matching with a hierarchical coarse-to-fine autoencoder and stochastic fragment bag strategies, whereas DMol employs diffusion models with ring compression, codependent node-edge perturbations with deterministic number of changes per step which give rise to valuable loss penalties, and specialized noise schedules for motif preservation. We observe that FragFM's use of discrete flow matching is related to DeFoG, which we already included as a baseline in our comparisons.

Thank you again for the extensive feedback on our work and for raising such interesting questions.

Reviewer FMwP, submitting a “Mandatory Acknowledgement” without leaving any comments for the authors in the discussion is not allowed. Could you provide feedback on the authors’ responses to your review comments? AC

We thank the reviewer for constructive suggestions. The letter.number W.X and Q.X in our response refers to the bullet listed as weakness and question, followed by its number.

W1: We respectfully disagree with the “old trick” characterization. While concepts similar to supernodes have been employed in methods like JT-VAE, our approach addresses fundamental limitations of VAEs. JT-VAE requires minimum spanning tree construction during encoding and faces significant challenges in decoding due to the complexity and ambiguities that need to be resolved when reattaching molecular substructures with multiple bond types to scaffolds. JT-VAE also uses several hundred supernodes, which is unacceptable for diffusion models. Furthermore, the authors of DiGress clearly stated that their attempts at supernode/motif masking failed (see their GitHub issue #15, and they remark that “It (meaning masking) would probably work better if we preserved the motif during diffusion in training,” but they did not come up with a satisfactory resolution). In contrast to JT-VAE, our method only uses 15 carbon ring motifs that can always be reintegrated into the scaffold in a chemically valid manner (our selection was based on ring frequencies and was done in consultation with chemists). Note that these motifs change for each dataset as different sets of drugs do have different motifs (e.g., high frequency substructures).

W2: For DMol, there is no concern pertaining to different molecular sizes since this issue is addressed through its design which preserves the joint distribution of individual atoms and motifs. Our approach does not arbitrarily replace motifs with single atoms, but motifs with other motifs or single atoms, because we explicitly model and preserve the distribution of motifs in the training samples. The diffusion process reconstructs both individual atoms and motifs according to their empirical distributions, ensuring that molecules maintain appropriate structural complexity. Additionally, our model incorporates a unique feature where the number of diffusion steps depends on the size of the molecule ($T = 2n$), and our objective function accounts for this relationship through penalty terms that ensure consistent node and edge count modifications. This design prevents problematic size shrinkage.

W3: It appears that there is a misunderstanding about our approach. While our predefined motifs (such as chlorinated carbon rings) cannot be directly transformed into nonmotif rings (such as brominated rings) during diffusion, this limitation does not impair our model's generative capabilities. Rings that are not used as motifs, of which there are many, can still be generated through atom-by-atom and edge-by-edge modifications during standard atom diffusion, which is how ours and other diffusion models manage to generate new ring structures in the first place. For example, a brominated ring can be generated by modifying an individual chlorine atom to a bromine atom through single-node state transitions. Our hybrid approach preserves the probability distributions of both motif and nonmotif structures. The key idea is that motif compression enhances generation of frequent rings while retaining full flexibility for generating arbitrary molecular patterns through an atom-by-atom level diffusion process.

W4. We respectfully disagree with the claim. By definition, a motif is a substructure that appears with significantly higher frequency than would be expected by random chance in a given dataset. Different molecular datasets naturally exhibit different motif structures/frequencies because they represent distinct chemical compounds with varying biological activities and synthetic accessibility constraints. Hence, MOSES, Guacamole etc all have different motifs (highest frequency substructures). As stated in line 302 of our paper, we systematically select the most frequently occurring carbon rings that can always form single bonds with scaffolds through available carbons. This dataset-specific selection is not only appropriate but necessary to capture the underlying chemical diversity of datasets.

W5 & Q1: There may be a misunderstanding how our forward diffusion process operates. In Appendix B (lines 669-679), we provide a comprehensive comparison of the number of edges modified at each time step of DMol and DiGress. In our approach, since the selection of a deterministic number of nodes at each time step is random, every edge in the graph has a nonzero probability of being selected and modified during the diffusion process. The probability that any edge remains unchanged throughout the entire diffusion process is the product of these probabilities across all time steps and our number of steps ensures that each edge has a nonzero probability of being selected. It is nevertheless important to understand that no diffusion model will or has to modify every node and edge.

Table 1: The result of conditional generation on MOSES dataset

W6& Q2: Regarding conditional generation, we actually did multiple experiments on this topic already. Rather than resort to complicated and time-consuming conditional generation, we opted instead to classify the molecules based on their scaffold (using four classes, aromatic monocyclic, aromatic monocyclic heterocycle, fused bicyclic, aromatic heterocyclic, following RDKit recommendations) or based on their graph embeddings (obtained by using GraphSage GNNs, followed by Kmeans++ clustering). In the latter case, we selected five clusters with the largest number of samples and best cluster separations. The results(Table 1) demonstrate that DMol perform significantly better on the GNN-governed clusters (validity: 80.7-83.1%, ChEMBL: 4.11-4.21) compared to scaffold based clusters (validity: 75.8-77.6%, ChEMBL: 3.60-3.91). This is a consequence of the fact that RDKit scaffold classes exhibit significant overlaps, while the GNN-based classes are well-separated. The GNN-based results also show slight degradations in performance compared to the full datasets which may be attributed to a significant reduction in the number of training samples.

Thank you again for the extensive feedback on our work and for raising such interesting questions.

We thank the reviewer for insightful comments and for taking the time to review our work.

Weakness

Technically impressive, but of limited practical value. This work brings little improvement in validity and ChEMBL-likeness and accelerates the sampling process by a large margin, compared to other methods. However, (i) the wet-lab experiment takes much more time than the in silico one, (ii) the sampling time of current methods is acceptable (e.g. DEFOG 5.81s). It means that the sampling time should not be the bottleneck in drug development.

We thank the reviewer for acknowledging the large improvements in validity, ChEMBL-likeness, and sampling speed of DMol. We agree that wet-lab experiments dominate the timeline during the later stages of drug development. However, sampling efficiency remains a critical factor that influences early-stage drug design pipelines, particularly in the de novo hit-to-lead phase where no prior reference compounds are available for a target protein. This is especially the case for high-throughput screening scenarios, for which the availability of a very large number of diverse and chemically valid candidates is essential.

For example, among molecules generated by all diffusion models including DiGress and DMol, those with a ChEMBL-likeness (CLscore) above 5.5 account for fewer than 0.5% of total outputs. While DMol significantly outperforms other models in this regard, each sampling run typically yields only a few dozen molecules that meet minimal thresholds for chemical likeness (note that scores closer to 8/9 are desirable, yet still not sufficient guarantees that the molecules can actually be synthesized). For example, Contract Research Organizations (CROs) such as Evotec and Nuvisan offer high-quality drug compound libraries containing over one million molecules, which are purchased and screened by pharmaceutical and biotech companies and it is easy to see that to get to such numbers purely through AI generation poses significant time constraints. This highlights the scale of high-throughput requirements and the need to generate large volumes of candidate molecules in de novo pipelines.

Hence, improvements in sampling speed directly enable more comprehensive chemical space exploration. Although methods like DeFoG already demonstrate efficient generation times (e.g., 5.81s/64 molecule), one has to note, as pointed above, that most of the molecules will not be chemically valid to the point that they can be synthesized. DMol offers further acceleration as well as a significantly better chance that the molecules will be valid due to their motif conservation. We believe this performance gain is meaningful in practical settings, especially when millions of candidates must be explored.

In conclusion, the general goal is not to merely generate as many molecules as possible, but to generate high-quality molecules, which is an inherently difficult task. This is not a critique of DeFoG or any other method specifically, but rather a shared limitation among current generative approaches in the field.

Poor formatting. The authors try to plug too much information into the main text, resulting in a tight page layout (the algorithms, the equations, etc). Besides, the font size in Figure 1 is too small to read. Ambiguous notations. On Line 230, the paper uses index j to represent both state and class, which may lead to confusion. It’s better to assign the indexes properly throughout the paper.

We thank the reviewer for this helpful feedback and apologize for the lack of readability. We acknowledge that we included a large amount of technical content in the main text. Given the complexity of drug discovery—requiring both rigorous mathematical formulation and deep domain knowledge—we hoped that including all core ideas would demonstrate the multifaceted ideas we tried to integrate to make the model more successful. We will nevertheless follow the advice and revise the formatting to improve clarity. Regarding line 230, the state j and class j are both referring to node type j. We apologize for this ambiguous use of the same symbol.

Over-simplified metrics on experimental tables. It seems that there is enough space for the tables to write down some of the full names of the evaluation metrics. For instance, the V, U, N metrics are really indirect for the readers who only read the tables.

We will revise all the tables to include the full names of evaluation metrics, improving readability for readers unfamiliar with our shorthand notation.

Questions

On Line 245, DMol samples the nodes and edges from specific marginal distributions. Why not introduce the motivation of the choice of the prior distributions?

In standard diffusion models, Gaussian (for continuous) or uniform (for discrete) priors are commonly used. However, in molecular generation tasks, such priors can be suboptimal due to the highly skewed distribution of atom and bond types in real-world molecules (e.g., carbon atoms are predominant in molecular graphs). Using a uniform prior significantly increases the learning burden on the model, as it must transform a flat distribution into a highly structured one. By contrast, using the empirical marginal distributions as priors gives the model a better inductive starting point, improving convergence and sample quality. This approach is supported by the recommendation in the landmark paper DiGress (see Section 4 and Table 6), where marginal priors resulted in better performance than uniform ones. Note that our priors are marginals pertaining to both atoms and the selected motifs.

Since the paper mentions the JT-VAE on line 307, why doesn’t it compare with this model in the experiments?

JT-VAE is a landmark model that has been widely used for benchmarking in earlier molecular generation work. However, it is based on VAEs, which operates in a latent space and requires highly complex decoding procedures to ensure chemical validity, such as enforcing valency constraints through hard-coded rules. DiGress also highlights these “hard-coded rules” in Table 2, which we believe makes direct comparison with diffusion-based models unfair. In contrast, our method—and the baselines we compare against—are based on more recent paradigms such as diffusion models and flow matching. To ensure a fair and up-to-date comparison, we followed the common practice in the field by benchmarking against these more recent and directly comparable diffusion approaches.

One of your strengths is fast sampling. Why not plot a figure to show your sampling speed compared to other methods?

We appreciate the reviewer’s suggestion. In our submitted paper, we indeed presented the sampling times (rather than speeds) of each method in Table 4. We believe that this table provides a direct and quantitative comparison and clearly demonstrates the efficiency of DMol.

One of your strengths is motif-preserving. Why not show some experimental results in the main text to show this property?

We thank the reviewer for highlighting DMol’s motif-preserving capabilities. Since molecular motifs (also known as shingles according to RDKit) play a crucial role in determining the chemical properties of a molecule, metrics such as ChEMBL-likeness and QED are specifically designed to reflect how well a molecule captures and preserves important biomedical shingles/motif substructures. For a more detailed explanation of the ChEMBL-likeness and QED metric, please refer to Appendix F of our paper.

Why not introduce your model’s name in the abstract?

We thank the reviewer’s suggestion - we will add the model’s name in the abstract for better recognition of the method.

Thank you again for the extensive feedback on our work and for raising such interesting questions.

Reviewer 6j7c, not responding to the authors / submitting a “Mandatory Acknowledgement” without posting a single comment to the authors in the discussion is not permitted. Could you provide feedback on the authors’ responses to your review comments? AC

We thank the reviewer for constructive suggestions. The letter.number W.X in our response refers to the bullet listed as weakness, followed by its number.

W1. We completely agree that the validity of generated molecules should ultimately be judged based on their utility in downstream pharmaceutical tasks. Unfortunately, the generative AI community as a whole currently does not have validation metrics that can be used to guarantee downstream performance. As a small step in this direction, we would like to draw your attention to the downstream validation results presented in our Supplement. For proprietary reasons, we cannot comment about our target selection rationale, but we indeed evaluated the AI-generated drugs as inhibitors of two proteins, 1PXH (Protein-tyrosine phosphatase) and 4MQS (human M2 muscarinic acetylcholine receptor). We ran DMol and identified molecules with fairly large CheMBL scores (>5) and certain desirable similarity with known drugs targeting the same proteins. We then ran docking simulations with AutoDock Vina (a simulation software that is not as accurate as an experiment, yet still one of the most sophisticated docking methods available). As shown in Figures 12 and 13 of the Supplement, our generated drugs illustrate both the strengths and limitations of AI approaches. For the 1PXH target (Figure 12), our selected generated molecules showed higher (less favorable) free energies (-6.4 and -6.0 kcal/mol) compared to the reference drug (-8.7 kcal/mol), which may be, among other reasons, attributed to the small molecular size of the drug candidate compared to the size of the reference inhibitor. We therefore focused on generated molecules that have similar sizes, motifs and compositions as the reference drugs and for the 4MQS target (Figure 13), we ended up with a molecule that significantly outperformed the best reference, achieving a free energy of -9.1 kcal/mol compared to the reference’s -6.4 kcal/mol. As a result, we made sure that for each target protein we sub-selected AI-generated drugs with appropriate molecular mass and motif structures. We chose not to emphasize this test in the main text due to space limitations and in order to maintain the coherent structure of our exposition. However, we can easily comment on more sophisticated downstream validation approaches, including similarity metrics for the molecules based on their masses, motives, 3D shapes as well as their “distances” of embeddings (generated by graph neural network) from known references. We also had guidance from molecular dynamics simulations, but did not include the conclusions to once again maintain a coherent exposition. Bridging the gap between generation quality metrics and real-world utility is and always was our biggest priority.

Table 1: The result of ablation study for the forward process schedules on MOSES dataset

Table 2: The result of scaffold-constrained generation on MOSES dataset

W2. To address the question regarding the cosine schedule, we reran DMol on the MOSES dataset using the cosine schedule with different values of the parameter $c$ (mentioned in line 217 of our paper) as well as a linear schedule ($\alpha = t/T$). The experimental results are shown in Table 1. As can be observed from the experiments, varying the value of $c$ has small impact on the performance metrics, with the optimal performance achieved for $c=0.008$ (which is the result presented in our paper). This finding aligns with Digress's selection of the hyperparameter $c$, as its code also adopts $c=0.008$ as the default setting. When comparing cosine and linear schedules, the cosine schedule is clearly superior to linear schedules. This advantage primarily stems from the fact that the cosine schedule adds less noise per timestep during both the initial and final phases of the diffusion process compared to intermediate stages. This distribution better aligns with the learning characteristics of diffusion models. Specifically, maintaining relatively small noise levels during the early diffusion phase/the last diffusion phase facilitates the model's learning of fine-grained data features, and ensure smoother convergence. In contrast, the uniform noise distribution in linear schedules may prove either overly aggressive or overly conservative at certain stages, leading to compromised training efficiency and ultimate performance.

Regarding scaffold-constrained generation, we actually did multiple experiments on this topic already. Rather than resort to complicated and time-consuming conditional generation, we opted instead to classify the molecules based on their scaffold (using four classes, aromatic monocyclic, aromatic monocyclic heterocycle, fused bicyclic, aromatic heterocyclic, following RDKit recommendations) or based on their graph embeddings (obtained by using GraphSage GNNs, followed by Kmeans++ clustering). In the latter case, we selected five clusters with the largest number of samples and best cluster separations. The results demonstrate that DMol perform significantly better on the GNN-governed clusters (validity: 80.7-83.1%, ChEMBL: 4.11-4.21) compared to scaffold based clusters (validity: 75.8-77.6%, ChEMBL: 3.60-3.91). This is a consequence of the fact that RDKit scaffold classes exhibit significant overlaps, while the GNN-based classes are well-separated. The GNN-based results also show slight degradations in performance compared to the full datasets which may be attributed to a significant reduction in the number of training samples. Furthermore, the excellent performance on individual GNN-classes shows that the cosine schedule does not impose special constraints for different scaffolds.

Thank you again for the extensive feedback on our work and for raising such interesting questions.