It Takes Two to Tango: Directly Optimizing for Constrained Synthesizability in Generative Molecular Design

We thank the reviewer for their constructive feedback. We respectively suggest that our work has been misinterpreted and we want to clarify our framework.

1. Clarify retrosynthesis uses single-step model + search algorithm

We will clarify this in the updated manuscript.

2. Starting-material constraint is trivial, not novel, nor innovative

We want to clarify what starting-material constraint means. The reviewer points out that in retrosynthesis, all leaf nodes are starting materials. However, all leaf nodes only need to be commercially available for a search to be successful. We denote commercially available nodes as $B_{stock}$ (this work uses 17.7 million blocks derived from ZINC). Starting-material constraint means that at least 1 leaf node at the max depth (first step of synthesis) must be from $B_{enf}$ (in this work, $B_{enf}$ $\in$ [5, 10, 100] and $B_{enf}$ $\subset$ $B_{stock}$). This is non-trivial because it heavily constrains the search. In our notation, we use $m = b \in B_{enf}$ and $depth(m)$ = max depth, as the reviewer also writes.

3. Retrosynthesis models are not oracles because they are not authoritative and top-1 accuracy is low

We believe the fact that “oracles” are imperfect does not preclude its use as a term. In one of the most popular benchmarks in drug discovery, the Practical Molecular Optimization benchmark [1], the term oracle is introduced as any computational predictor. In that work, surrogate models are considered oracles, even though they are trained with limited data so they are not expected to have global coverage. For retrosynthesis models, the reviewer rightly points out top-1 accuracy is imperfect. However, top-1 accuracy itself is an imperfect measure of oracle effectiveness as there are many ways to make the same molecule. However, how would one assess the usefulness of such alternative pathways? We believe it is most convincing to reference real life medicinal chemistry case studies, for example presented in this paper [2], where retrosynthesis models positively impacted 7 commercial drug discovery synthesis projects.

One can argue in LLMs, RLHF preference fine-tuning is also not an authoritative source of truth as humans have biased preferences. Yet, there is still value in tuning with these reward models.

Therefore, we believe the term oracle is reasonable, even if they provide approximations to the "true" value. The most important aspect is that one is aware of the limitations of the oracle, which we explicitly acknowledge in our conclusions section.

4. Why use $B_{ref}$ instead of entire block space? Large block space = sparse reward. Small block subset = reward denser but worse generalization

We use $B_{enf}$ (we assume the reviewer meant this, please correct us if this is not true) because we are tackling constrained synthesizability. We kindly refer to our response to Reviewer 5JW4.

Contrary to what the reviewer suggests, a large block space actually makes the reward more dense, rather than more sparse. More building blocks generally means more molecules can be “synthesized” from them. More synthesizable molecules means more molecules will have a reward as TANGO assigns a reward to all synthesizable molecules (Fig. 2). Non-synthesizable molecules receive 0 reward.

The reviewer is correct that enforcing the presence of specific blocks may restrict generalization (in the sense of what is synthesizable). But this is exactly our problem setting and why constrained synthesizability is challenging.

We recognize that the reviewer may have wrote this comment due to interpreting our framework as RL from scratch instead of post-training. However, we wanted to provide an answer to help clarify our work.

5. TANGO resembles PRM but we should use outcome reward model

We would like to clarify that TANGO is an outcome reward model already. We reference Fig. 2 in our main text. Every synthesizable molecule has a synthesis pathway. TANGO measures the max similarity between every molecule node and the set of enforced blocks ($B_{enf}$). There are no intermediate rewards in our framework. Generating molecules in token space only rewards the final molecule to ensure the molecule is chemically valid. Therefore, TANGO returns an outcome reward given the final molecule and its synthesis pathway.

6. Use post-training instead of RL from scratch

We are already using post-training. Our model is pre-trained on PubChem which is a dataset of bioactive molecules. This pre-training procedure is detailed in Appendix A. This pre-training is performed once. Next, we post-train the model using RL. This is akin to LLM pre-training and RLHF fine-tuning.

We hope to have clarified our work and thank the reviewer for their time. Please let us know if there are additional questions!

[1] https://arxiv.org/abs/2206.12411

[2] https://pubs.rsc.org/en/content/articlelanding/2024/md/d3md00651d

Thank you to the reviewer for their feedback and for the opportunity to clarify our work.

FP = fingerprint

NN = nearest-neighbor

1. Why is constrained synthesizability important?

We kindly refer to our response to Reviewer 5JW4.

2. Why existing synthesizable design works cannot be easily adapted to this problem setting

We first highlight characteristics of recent synthesizable design works and then discuss why adapting these models for constrained synthesizability is non-trivial.

SynNet [1]

• 147,505 blocks
• FP + NN to select blocks
• Pre-training dataset prepared by sampling blocks-reactions to generate synthetic trees

SynFormer [2]:

• Diffusion to select blocks, allowing scaling to longer FP
• 223,224 blocks
• Pre-training dataset prepared by sampling blocks-reactions to generate postfix notations [3] of synthesis

RGFN [4]:

• Either 350 or 8350 blocks
• Memory constraints limiting scaling to larger block sets (see Appendix B.5 and Appendix D.3 in [6])

SynFlowNet [5]:

• Scaling to up to (as stated in the paper) 200k blocks but most experiments are run with 10,000 blocks
• Uses masking to enforce compatible blocks and reactions
• Memory constraints limiting scaling to larger block sets (see Appendix B.5 and Appendix D.3 in [6])

RxnFlow [6]:

• Scale up to 1.2M blocks
• Non-hierarchical MDP formulation compared to hierarchical in RGFN [4] and SynFlowNet [5] allowing adaptation to changing blocks

In all methods above, blocks are selected either by FP NN or softmax sampled. Constrained synthesizability means that a small set (in our work we considered sizes of 5, 10, 100) of enforced blocks are present in the synthesis pathways. With FP, the reviewer suggests doing NN search on the enforced blocks. However, it is non-trivial to decide at which step of the synthesis pathway to do this in the general case, e.g., choose a "normal" block at step $t$ and then force step $t+1$ to choose an enforced block? What if the enforced blocks are incompatible with the reaction? One may need to encode explicit biases into the generation process which TANGO overcomes by incentivizing the learning process. One could mask the actions to guarantee enforced blocks, but the same consideration remains in when to do this and what to do for block-reaction incompatibility. For SynNet [1] and SynFormer [2] which require pre-training with enumerated “pseudo” routes, should one generate more routes with the enforced blocks to increase their sampling probability? Would this diminish the model’s ability to generate diverse molecules? We believe these are open questions for model development.

We next highlight a scalability limitation. We used 17.7M blocks which is 2-3 orders of magnitude larger than existing works which need to change model dimensions and/or more memory to increase block set. We can scale to 100M blocks since we only need to store the SMILES. But why use so many blocks? It is not unreasonable to consider any and all blocks that are commercially available. Our framework also allows freely changing block sets while the existing methods need to re-initialize (see [7] which applies Saturn using 5 distinct block stocks without re-training).

TANGO guides Saturn, which is a model with no synthesizability inductive biases, to optimize for constrained synthesizability. TANGO is general and can augment these existing models to do this and designing this function is the contribution of our work. However, we note that it may still not be straightforward as the lower sample efficiency of GFlowNet models can make this computationally prohibitive, i.e., constrained synthesizability alone is not enough, the molecules' properties must also be optimized (see [8] for a benchmark where GFlowNets are ranked 16/25 for sample efficiency and [7] for a Saturn/GFlowNet comparison). See also Appendix A.5 in SynFlowNet [5] showing that its sample efficiency is lower than REINVENT [9], which Saturn significantly outperforms (see Table 3 in [10]). Lastly, we report docking scores across thresholds to show that our framework performs multi-parameter optimization.

We now have results showing TANGO can augment GraphGA (genetic algorithm) for constrained synthesizability and will update the manuscript when we are able to do so.

We are eager to continue discussion and hope we clarified some points. Please let us know if there are follow-up questions!

[1] https://openreview.net/forum?id=FRxhHdnxt1

[2] https://arxiv.org/abs/2410.03494

[3] https://openreview.net/forum?id=scFlbJQdm1

[4] https://openreview.net/forum?id=hpvJwmzEHX

[5] https://openreview.net/forum?id=uvHmnahyp1

[6] https://openreview.net/forum?id=pB1XSj2y4X

[7] https://pubs.rsc.org/en/content/articlehtml/2025/sc/d5sc01476j

[8] https://arxiv.org/abs/2206.12411

[9] https://jcheminf.biomedcentral.com/articles/10.1186/s13321-017-0235-x

[10] https://arxiv.org/abs/2405.17066

Thank you to the reviewer for their positive assessment of our work. The point about true synthesizability is very important to us, as experimental validation is always the end goal of generative design.

As we are unable to update the manuscript version at this time, we wanted to provide more discussion in our response here. The major limitation of retrosynthesis models is that proposed pathways might suffer from general feasibility (does the reaction actually proceed?) and selectivity problems - regioselectivity, i.e., does the reaction proceed at exactly the position we want? And stereoselectivity, i.e., does the reaction yield the correct enantiomer as the major product? To this end, reaction feasibility/selectivity in retrosynthesis has been explored by integrating quantum chemistry information either through simulations [1] or the use of machine learning force fields [2] (we cite a recent example each and a recent review here [3]). Effectively and efficiently leveraging quantum chemistry information for reaction feasibility is an open research problem. However, going forward, more capable retrosynthesis models would immediately benefit our framework.

Finally, in the reaction pathways shown in the main text, amide coupling reactions are predominantly present (partially owing to how common they are). In general, this is a relatively robust reaction. In the updated manuscript, we will explicitly comment on the feasibility of the reactions.

[1] https://chemrxiv.org/engage/chemrxiv/article-details/671f791c1fb27ce124d5c98c

[2] https://chemrxiv.org/engage/chemrxiv/article-details/67d7b7f7fa469535b97c021a

[3] https://pubs.rsc.org/en/content/articlelanding/2025/sc/d5sc00541h

We thank the reviewer for their feedback and an opportunity to clarify our problem setting.

Generative approach for constrained synthesizability. Why is this useful?

We answer all questions from the reviewer in this single response since they are related.

What is the problem of constrained synthesizability?

Synthesizing molecules can be thought of as stitching together “building blocks” (commercially available chemicals) with reactions. We will refer to this set of building blocks as $B_{stock}$. Constrained synthesizability extends this problem setting to also enforce that a specific set of building blocks is used in the synthesis and denote this $B_{enforced}$. We emphasize that there are much fewer $B_{enforced}$: $|B_{stock}|$ is 17.7 million while we used $|B_{enforced}|$ $\in$ [5, 10, 100]. This is a significantly harder problem because the model must generate molecules that can be synthesized by using both $|B_{enforced}|$ and $|B_{stock}|$ and blocks cannot be freely combined together as they must be chemically compatible. TANGO enables a completely unconstrained generative model to learn how to satisfy this constraint using reinforcement learning, while performing multi-parameter optimization of other properties.

Why should we care about enforcing specific building blocks?

Building blocks have variable costs. One can envision enforcing the use of the cheapest set of blocks from a commercial vendor to manage costs.

We may want to re-purpose certain blocks into useful molecules. For example in [1], waste molecules from industrial processes are repurposed into medicines. By definition, this is constrained synthesizability because one wants to use these waste molecules ($B_{enforced}$) together with general chemicals that can be purchased ($B_{stock}$). One could conceive using TANGO to repurpose waste into de novo molecules.

In drug discovery, we often want to generate molecules that share a common scaffold and diversify from this scaffold. In Fig. 3 right panel, we show this capability explicitly where a 1-step reaction from an enforced scaffold forms a new molecule with improved properties. An important capability of TANGO here is that > 1 block can be considered simultaneously and it is not uncommon that one is interested in multiple scaffolds. There are existing works that can enforce a specific scaffold with pre-defined exit vectors to attach new chemical groups to but TANGO is much more general.

Where is constrained synthesizability useful beyond drug discovery?

In this paper, we focused on organic molecules for drug discovery. Organic molecules encompass many classes of matter, for example organocatalyst design [3] and functional materials design like semiconductors [4]. TANGO could be extended to tackle these problems.

Overclaim of first generative approach and no discussion of existing works

In section “Synthesizability-constrained Molecular Generation”, we cited many works that have indeed tackled synthesizability in generative models. However, our problem setting is constrained synthesizability for which the existing works do not tackle. More specifically, we generate molecules with an explicit synthesis pathway (step-by-step recipe to make the molecule) that also incorporates specific blocks. In Appendix F, we show examples of these synthesis pathways and put a box around the specific enforced block used. While we operate in the organic molecules space, we are also, to the best of our knowledge, not aware of crystal (inorganic) generation works with constrained synthesizability.

For detailed discussions of why existing works cannot easily be extended to constrained synthesizability, we kindly refer the reviewer to our response to Reviewer ogSL.

More recently, constrained retrosynthesis algorithms have been proposed. More details about this

[5] is a specific recently proposed constrained retrosynthesis algorithm. Retrosynthesis is a different problem setting to ours: retrosynthesis involves generating a synthesis pathway given a target molecule. We do not have a “target molecule” in the same sense, as TANGO is guiding the model to generate molecules that have a synthesis pathway containing the enforced blocks and containing optimized properties. Furthermore, in [5], the algorithm is trying to propose a synthesis pathway that incorporates a single pre-defined block. TANGO can generalize to a set of blocks, shown by the results when we enforced 5, 10, 100 blocks.

Thank you to the reviewer again and we would be eager to continue discussion!

[1] https://www.nature.com/articles/s41586-022-04503-9

[2] https://pubs.acs.org/doi/10.1021/acs.jcim.1c00469

[3] https://onlinelibrary.wiley.com/doi/full/10.1002/anie.202218565

[4] https://pubs.rsc.org/en/content/articlelanding/2020/nr/c9nr10687a

[5] https://openreview.net/forum?id=LJNqVIKSCr