Weakness 1 Response:

Thank you for this comment, we agree that the preliminary section present in the original manuscript was insufficient for understanding the method. We revised it to include the following information, which hopefully will address Reviewer’s remark:

Another way to rephrase the flow-matching constraints is to learn a forward policy $P_F(s_{i+1}|s_i)$ such that trajectories starting at $s_0$ and taking actions sampled by $P_F$ terminate at $x \in \mathcal{X}$ proportional to the reward.

Trajectory balance. Several training losses have been explored to train GFlowNets. Among these, trajectory balance [citation] has been shown to improve credit assignment. In addition to learning a forward policy $P_F$, we also learn a backward policy $P_B$ and a scalar $Z_\theta$, such that, for every trajectory $\tau = (s_0 \rightarrow s_1 \rightarrow \dots \rightarrow s_n = x)$, they satisfy:

$

Z_\theta \prod_{t=1}^{n} P_F(s_t|s_{t-1}) = R(x) \prod_{t=1}^{n} P_B(s_{t-1}|s_t)

$

However, please note that we are still fairly constrained in terms of space, hence the description has to remain brief.

Weakness 2 Response:

Thank you for the suggestion. We included a fragment-based GFN that used the average of proxy value and SA score as the reward as another baseline (specifically, we used $R(x) = exp(\beta * (0.5 * proxy(x) / max_{proxy} + 0.5 * (10 - SA_{score}) / 10)$, with $\beta$ adjusted per proxy to match the original reward range. We attached the results in the PDF with the main answer to the reviews. As can be seen, using the SA score as a part of the reward slightly reduces the obtained proxy values, while improving the SA scores (which is expected). However, as can be seen, it does not translate to AiZynthFinder scores on par with RGFN, which highlights the issue of poor reliability of SA scores that was discussed in the manuscript.

Weakness 3 Response:

Thank you for the suggestion. As requested, we investigated computing the modes based on SA score-thresholded molecules only (with SA <3, which should correspond to easy-to-medium synthesis difficulty; lower values yielded similar trends). We conducted this experiment specifically for ClpP, with the results attached. However, this did not turn out to change the results significantly, since as discussed in the original manuscript, SA scores are a poor approximation of synthesizability. We could not conduct a similar experiment with AiZynthFinder in time due to much larger computational cost. We agree with the Reviewer that selecting only synthesizable modes should significantly improve RGFN’s performance, but the issue is once again evaluating the synthesizability.

Weakness 4 Response:

On a single RTX Quadro 8000 GPU, generating 1000 molecules from a trained FGFN model took ~7 seconds, while evaluating the top 100 molecules with AiZynthFinder took 2206 seconds. On the other hand, generating 1000 samples using RGFN took ~14 seconds. It’s worth noting that FGFN used a heavily optimized implementation [1], while our approach didn’t leverage multiprocessing for sampling, which might have produced additional overhead. Still, this illustrates that when taking into account the cost of synthesis, using RGFN becomes much more efficient.

Minor

Thank you for spotting this, it was revised!

[1] https://github.com/recursionpharma/gflownet

Thank you for the detailed and informative clarification. Most of my concerns are resolved. I believe that the paper has a basic value to be accepted. Therefore, I'd like to increase my score (5->6).

Significance

We thank the Reviewer for pointing out additional references that were included in the revised version. While the idea of extending GFNs to operate in the reaction space might seem straightforward, significant work was needed to implement the approach, not emphasized enough in the original manuscript. These include:

• Handling multiple possible products of a given reaction, which results in a de facto non-deterministic environment. This is not an issue in FGFN, as operating on graphs allows us to specify the place of inserting a new fragment. We deal with this by introducing a product selection step (Eq. 5).
• Since GFNs require computation of parent states and masking invalid actions, this required implementing efficient recursive decomposition of molecules into BBs and crafting specific SMARTS templates for reactions that would limit the possible number of parents.
• Crafting a specific set of high-yield reactions and low-cost fragments that would enable synthesis.
• Improving scalability by introducing fingerprint-based action embeddings for fragment selection.

Additional discussion can be found in Overall Response 2.

Quality

We included the DRD2 benchmark from TDC. We attach the results. The trends are comparable to other oracles.

W1

Please refer to the answer above regarding the originality of the approach. The comment regarding needing to emphasize more the differences between our and existing methods, was addressed in the Overall Answer.

W2

When implementing SynNet, we were unable to reproduce a decoder that reliably generated valid SMILES strings. As the initial trained weights provided did not match the most recent version of the US Enamine BB stock and generated nonsensical SMILES, we retrained the model’s four MLPs on the recent stock, which still did not result in reliable SMILES outputs. Due to time constraints, we instead opt for RXNGenerator, a VAE model designed for generating synthetic trees. For a general response regarding other synthesizable molecule generation methods, see Overall Response 2.

W3

We indicated in the paper checklist that the code would be provided upon acceptance and hope it wasn’t misleading to mark it as such in the checklist. We provide the code as is in the AC comment.

We did mistakenly omit the computational resources in the original manuscript, we apologize for that. Evaluating fragment scaling took approximately 800 RTX4090 hours in total. Remaining experiments took roughly 24 hours on Quadro RTX 8000 for sEH, DRD2 and senolytic proxies, and roughly 72 hours on an A100 for docking-based proxies.

W4

See Overall Response 1.

Q1

Vast majority of the molecules were generated using 4 reactions (maximum allowed number). We applied this constraint because larger molecules tend to achieve higher docking scores, and GFNs generate samples with the probability proportional to the reward, leading to a higher likelihood of generating larger molecules. Achieving higher size diversity would be possible by penalizing larger molecules.

Q2

The state space size is precisely the number of unique molecules that can be generated from the listed reactions and reactants. We have tried to explain this more clearly in the revised text.

Q3

Unfortunately, we were unable to explore using different FGFN fragments in time for the rebuttal. Other factors we suspect were 1) larger state space size of FGFN (which makes convergence more difficult), and 2) implementation differences (we based FGFN on [1], while RGFN was implemented from scratch).

Q4

Our hypothesis is that the QED differences stem mostly from different average molecular weights. While we tried to achieve comparable weight ranges for different methods, it’s still task dependent (Table 1). SyntheMol uses shallow synthesis trees which lead to smaller molecules. We note that QED could easily be improved upon by adding it directly as a reward term.

Q5

Synthetic routes from RGFN were used. Costs of synthesis were estimated according to literature reaction yields, building block availability, and unit price. The molecules generated in the paper were not synthesized, but we are in the process of doing that for another batch.

Q6

It is unlikely actual synthetic routes would fully follow those generated by RGFN due to the linear nature of RGFN compared to a retrosynthetic analysis that would aim to maximize the yield RGFN is not intended to optimize the synthetic route but rather to generate synthesizable compounds.

Q7

• Yes, we use the architecture expressed in Eq. 4 rather than Eq. 7, which outperforms with larger fragment libraries. However, for the fragment library outside Section 4.3 which contains 350 molecules, both architectures perform on-pair, as illustrated in Figure 5. We opt for Eq. 4 only because it is simpler. We note this in the text.
• The embedding scheme proposed in Eq. 7 can be used for reaction templates. We did experiment with using RxnRep [2] embeddings in the early stage of the project but observed only a negligible performance improvement.

Q8

We employed the standard techniques of the Leader algorithm as implemented in RDKit. While in theory this process is dependent on the order of the molecules, in practice the molecules are always shuffled and the standard deviation for the number of modes across 10 shuffled runs is less than 0.5%. We now add a note to this in the manuscript.

L1

Regarding limited library size, please refer to the Overall Response. We agree with the oracles being a limiting factor, but as the Reviewer states, this is not a unique problem to RGFN.

L2

The GFlowNet models each visited 400k molecules, the budget set for baselines, with the exception of SyntheMol. However, our models were able to converge much earlier, and allowing RGFN/FGFN to run for the remaining time was meant to allow exploration. We attach convergence plots for RGFN to illustrate this.

[1] github.com/recursionpharma/gflownet
[2] doi.org/10.1039/d1sc06515g

Thank you for following up. Yes, while I agree your curated set of reactions and fragments could be an interesting contribution, currently it is hard for me to judge given that they are not actually included in the submission (related to point W3). I believe (from the initial rebuttal) you have provided this to the AC though (note I cannot see the AC comment) so leave it with them.

Anyway, I have edited my score to reflect many of my original points getting addressed and to reflect the additional experiments carried out (including the one recently posted), showing that the diversity of the produced molecules could be a reason to prefer the approach here over other previously proposed synthesis-based de novo design methods.

Thanks also for the discussion!

Weakness 1 Response:

Thank you for pointing out the omission of the Qiang et al., 2023 manuscript (“Bridging the gap..”) and the work by Nguyen et al. (“A generative model … reaction trees”) , we added the references to the revised version. In the related work section we cited a review by Meyers et al., 2021 (reference #42 in the original manuscript) that covers many other relevant works. The manuscript “Generating molecules via chemical reactions" by Bradshaw et al., was actually cited by us (reference #8). The cited manuscript (“A Model to Search for Synthesizable Molecules”) refers to the NeurIPS version, while the version pointed out by the reviewer (“Generating molecules via chemical reactions”) refers to an earlier ICLR workshop version. We also include several references mentioned by another Reviewer.

Weakness 2 Response:

Please refer to Question 1 Response for the answer to the stated limitation being reliance on pre-trained models, and Limitation 2 Response to the stated limitation being reliance on reaction templates.

Reaction templates are necessary to avoid post facto evaluation of synthesizability for analogs generated solely based on physico-chemical parameters without regard to the realities of chemical synthesis. Synthetic intractability has in fact been a major issue with previous generative methods. The use of reaction templates solves this problem without imposing detrimental constraints on chemical diversity. Reaction templates are thus inherent to our approach as the only reliable means to avoid infeasible structures and/or to enforce focussing on robust chemical reactions (or those available at a preferred chemical supplier, like Enamine).

We have added additional text to the manuscript to clarify this important point on the challenges of real-world synthetic chemistry. We thank the reviewer for highlighting this issue and hope the reviewer concurs with our arguments.

Weakness 3 Response:

We apologize for the lack of clarity on this point. By the definition of our search space, all molecules generated are inherently valid (e.g., correct valence, reasonable size, etc.) because all compounds are generated by bona fide synthetic reactions. We also demonstrate diversity of generated compounds in Figure 4, which shows RGFN finds more modes (i.e., molecules dissimilar to each other) compared to GraphGA and SyntheMol. We note that because the GFN model is trained from scratch, all generated molecules are “new” because the model is not trained on any known molecules. They are also highly different from known ligands, as demonstrated in Appendix L. We also note that the current largest space of synthesizable molecules at Enamine (~40 billion) is not systematic in terms of reaction combinations and is more than an order of magnitude smaller than even the limited RGFN space we present here. We have revised the text of our manuscript to better reflect these points.

Question 1 Response:

We apologize for a lack of clarity on this key issue. While some of the considered oracles are indeed pre-trained ML models (sEH proxy and senolytic proxy), we specifically include the GPU-accelerated docking as an alternative reward function to alleviate this issue. Our approach is agnostic to the reward function, and could even use something like a QED score as the reward. We emphasize that all of the considered baselines also rely on the exact same oracle models (including the pre-trained proxies). We believe that a strength of our approach is its adaptability to all possible reward functions.

Question 2 Response:

Although having a limited set of known reactions in principle does restrict diversity, the space of synthesizable molecules is still vast and greatly exceeds all synthesized molecules to date. A huge space of compounds remains to be queried for biological activity based only on established chemical reactions, due to the massive number of related building blocks that can be plugged into any given reaction. Thus, there are approximately 219M substances known to be produced and characterized in the literature [1] and the evaluated size of produced chemical space in RGFN is already larger, despite using only a modest number of input reactions and building blocks. The total chemical space encompassed by these reactions and associated reactants has barely been explored.

Limitation 1 Response:

As above, we emphasize that RGFN does not heavily rely on pre-trained models. Please refer to Question 1 Response for the answer to this perceived limitation.

Limitation 2 Response:

We would like to clarify the difference between reactions and reactants. The number of feasible reactions is limited, the number of reactants that can be joined together by these reactions is very large, and hence the number of combinations is extremely large. To give a simple example, naturally occurring peptides are generated by a single reaction type (amide coupling) with only 20 reactants (the L amino acids) yet can linearly generate tremendous chemical diversity. The reaction templates need not be updated as more reactants are added - the diversity of possible reactants that can be accommodated alone enables a combinatorial explosion. In practical terms, it is also possible to add any reaction to the dataset using the SMARTS language, and those that we present in the paper are known to be generally very robust and reliable. While this method obviously doesn't allow us to discover ligands that could be produced from yet unknown reactions, it is not an aim of this work to devise new synthetic reaction methods. On the contrary, our stated goal is to ensure "out-of-the-box" synthesizability by using selected reactions that are more likely to deliver the product and thus shorten the discovery process, while at the same time ensuring massive chemical diversity.

[1] https://www.cas.org/cas-data/cas-registry

Weakness 1 Response:

Regarding limited library size, please refer to the Overall Response. We agree with the oracles being a limiting factor, but as the Reviewer states, this is not a unique problem to RGFN.

Question 1 Response:

The reviewer raises a good point. We agree that the SA scores are inaccurate, and in our opinion using them is not really necessary. However, we have tried as best as possible to demonstrate the synthesizability aspect in a rigorous way, and including the SA scores was one data point (in addition to the AiZynthFinder and manually showing the synthesis paths of top-10 molecules). Huge differences in SA scores can thus serve as a rough guideline. We also anticipated that not providing SA scores might lead to reviewers requesting the scores be included in our analysis, since the application of such scores is fairly common. We acknowledge that demonstrating synthesizability is difficult in general, and more work into this problem is required (but is outside the scope of this paper). We have added some text to highlight the caveats with current synthesizability assessment methods.

Question 2 Response:

At this juncture the platform does not automatically recommend specific synthetic routes. However, a trained synthetic chemist can readily propose reasonable synthetic routes for any proposed structure from RGFN, particularly given that the reaction templates used are well known. Synthetic routes may also be proposed by well-developed stand alone retrosynthetic software packages.