Retrosynthesis Planning via Worst-path Policy Optimisation in Tree-structured MDPs

Thank you for your detailed feedback and for the time you spent in reviewing our manuscript. We have addressed every point you raised and will incorporate the corresponding changes into the final version.

W1. Thank you for bringing this to our attention. We have regenerated Figure 2 and Figure 4 in PDF format to ensure its readability in different devices. For Figure 2, we enlarged the size of each sub-figure, increased the font size, and ensured that all sub-figures scale cleanly on A4 paper. Because we cannot upload revised paper during rebuttal, these improved figures will appear in the final version.

W2. Table 1 shows the results for different combinations of single-step models and search algorithms across three benchmarks. Each row represents a specific combination (e.g., "InterRetro - Retro*" means InterRetro as the single-step policy with Retro* as the search method). The columns then show: (1) "DG" - direct generation without search, (2) "100", "200", "500" - results with different search budgets (maximum model calls). We will improve the caption and the table accordingly to make them easy to understand.

Q1. Contribution 1 and contribution 2 formulate the problem into a tree-structured MDP with the worst-path objective, and propose self-imitation to optimise this objective. However, it still has a distance from the theory to a practical algorithm - we need to decide how the agent interacts with the tree MDP, how the agent stores past successful experience for self-imitation, etc. Therefore, we have contribution 3, which extends the theory, adds more implementation details, and solves the retrosynthesis problem in practice. The engineering decisions and implementation details that enable this capability constitute a significant contribution beyond the theoretical framework.

Q2. The notation $2^{\mathcal{S}}$ denotes the power set of $\mathcal{S}$, which is the set of all possible subsets of $\mathcal{S}$. We use this because in retrosynthesis, applying a reaction to a molecule produces a set of reactant molecules (typically 1-3 reactants), not just a single molecule.

For example, if molecule $s$ undergoes reaction $a$, it might produce two reactants {$s_1, s_2$}. Since {$s_1, s_2$} is a subset of all possible molecules $\mathcal{S}$, we have {$s_1, s_2$} $\in 2^{\mathcal{S}}$. This is why our transition function is defined as $\mathcal{T}: \mathcal{S} \times \mathcal{A} \rightarrow 2^{\mathcal{S}}$ rather than $\mathcal{T}: \mathcal{S} \times \mathcal{A} \rightarrow \mathcal{S}$. We have added this clarification to Section 3.1 in the revised manuscript.

Q3. The Retro*-190 in our paper refers to the dataset from Retro*, as described in Section 5.1 by Chen et al. (2020). $190$ is the number of target molecules included in the dataset. The dataset can be downloaded in Retro*'s GitHub page. We refer this dataset as Retro*-190 rather than simply Retro* to keep the suffix indicating the number of the test molecules in the dataset, being consistent with ChEMBL-1000 and GDB17-1000.

Q4. Our impressive results stem from two key innovations: (1) the worst-path objective that directly optimises for complete route validity, and (2) the self-imitation learning that effectively leverages successful experiences. For the research community, these results demonstrate that search-free retrosynthesis is not only possible but can outperform search-based methods. This could redirect research efforts toward end-to-end learning approaches. For practical applications, the search-free nature offers immediate benefits: real-time route generation for large molecular libraries, integration into interactive design tools, and reduced computational infrastructure requirements.

However, we must acknowledge important limitations. Our approach, like all current methods, assumes predicted reactions are experimentally feasible — an assumption that doesn't always hold. Real-world deployment would require integration with reaction feasibility predictors and experimental validation workflows. Additionally, our method currently optimises for route depth rather than other practical considerations like cost and reaction conditions.

We see our work as a significant step toward practical automated synthesis planning, but achieving industry-ready systems will require addressing these additional challenges. The impressive metrics indicate that the core planning problem is becoming tractable, allowing the field to focus on these remaining practical considerations.

---

Finally, we thank you again for raising important questions about our submission. Have we adequately addressed the main concerns? Please feel free to let us know if there are additional concerns or questions. Have a nice day!

Reference

Chen, Binghong, et al. Retro*: Learning retrosynthetic planning with neural guided A* search. ICML. 2020.

Thank you sincerely for your professional and constructive feedback. We are pleased to receive your comments, which have helped us improve the manuscript significantly. We have analysed and addressed each point below, and we hope our responses satisfactorily resolve your concerns.

W1. In line 141, we define the transition function as a mapping from the state–action space to the power set of the state space, $\mathcal{T}: \mathcal{S} \times \mathcal{A} \rightarrow 2^{\mathcal{S}}$, in line with your suggestion. We have revised the presentation to improve clarity.

W2. We have revised the experimental section accordingly; please see our response to the Questions below.

W3. We agree that the quality of the synthetic route is essential in experimental settings. However, our emphasis on computational efficiency reflects the growing demand for scalable retrosynthesis planning, not a compromise on quality. In AI-guided drug discovery, retrosynthesis planning is increasingly deployed in batch settings to evaluate large numbers of candidate molecules generated by models such as LLMs. These pipelines often screen tens of thousands of compounds, where each must be filtered or ranked based on synthesizability. In such settings, latency per molecule quickly becomes a bottleneck. Furthermore, human-in-the-loop platforms (e.g., IBM RXN and ASKCOS) allow chemists to interactively sketch molecules and receive immediate feedback on the molecule's synthesizability. These real-time applications require short latency, enabling seamless user interaction and rapid design iteration. Such fast feedback is critical in practical workflows, where chemists often explore many variants of a lead compound in a single session. Therefore, in modern retrosynthesis applications, short inference times are not merely desirable but often necessary. When route quality is comparable, computational efficiency becomes the key to making retrosynthesis planning actionable at scale.

Q1. The heuristic search algorithms mentioned in the background paragraph require substantial model calls: Retro* typically exceeds 150, MCTS around 250, and Greedy DFS around 300. More details are provided in our response to Question 5. we agree that describing them as "thousands" is inaccurate; to clarify this, we have updated the text to: “These methods heavily rely on time-consuming real-time search, often requiring hundreds of model calls for each molecule...”

Q2. Thank you for highlighting this concern. The $\gamma^{T}$ term serves only to discourage unnecessarily deep decompositions: its effect is relative to the alternatives available for a given molecule. A short path that fails to reach purchasable building blocks receives a return of 0, whereas a longer but successful path still earns a positive return $\gamma^{T}>0$; consequently, our policy will always favour the viable route, even if it is longer. Among successful routes, $\gamma^{T}$ then biases the planner toward the shallower option, cutting out extra steps while keeping the route feasible. Moreover, because the worst‑path objective scores each synthesis tree by its weakest branch, any tree containing a dead‑end is automatically rejected, preventing the search from being “trapped” in short but invalid regions. Therefore, the depth penalty prunes gratuitous exploration while still selecting longer routes whenever they are actually required.

Q3. In InterRetro, we initialise a pretrained Graph2Edits model ($\pi_\theta$ in Algorithm 1), and finetune it using self-imitation learning until convergence. During the search process, this finetuned Graph2Edits model is used as the one-step expansion model to generate reactants. We do not combine or integrate it with any other trained policy. The retrosynthesis search process, like Retro* (Chen et al., 2020) and MCTS (Hong et al., 2023), typically consists of three steps: selection, expansion, and update. Our finetuned Graph2Edits model is used exclusively in the expansion step, where it predicts a sequence of edits to decompose a molecule into its reactants.

Q4. Thank you for pointing this out. We mistakenly reported the results for a model-call budget of 400 (Liu et al. 2023). We have now thoroughly reviewed all other reported results to ensure their accuracy.

Q5. Thank you for your suggestion. Below, we compare our method with the suggested RetroGraph and other relevant algorithms:

To compute the average model calls, we set the maximum model-call budget to 500 and measured the average number of calls required by each algorithm to successfully construct a synthesis route for molecules in the Retro*-190 dataset. We find that InterRetro and PDVN both achieve state-of-the-art performance in terms of model-call efficiency. These results will be included in the revised version of the paper.

Q6. We acknowledge the importance of a retrosynthesis planning algorithm to support various optimisation objectives. In this case, we could incorporate new quantities into the weighting function and encourage the policy to learn those low cost reactions. The cost of chemical reactions may come from the reaction conditions, the price of the reactants, etc. To the best of our knowledge, a comprehensive dataset capturing such information is not yet available. We would be happy to extend InterRetro in this direction once the necessary infrastructure and data become available.

Limitations. In the revised version, we have incorporated the limitations and the potential negative societal impacts in Appendix to the last section.

Paper Formatting Concerns. We have added the guidelines in the revised version.

---

We would like to thank you again for raising important questions about our submission. Due to this year's rebuttal policy, we cannot upload a revised PDF to reflect these changes. However, we believe that after incorporating the valuable feedback our current version becomes much stronger. Have we sufficiently addressed the main concerns? Please feel free to let us know if there are additional concerns or questions.

References

Chen, Binghong, et al. Retro*: Learning retrosynthetic planning with neural guided A* search. ICML, 2020.

Hong, Siqi, et al. Retrosynthetic planning with experience-guided Monte Carlo tree search. Communications Chemistry, 2023.

Liu, Guoqing, et al. Retrosynthetic planning with dual value networks. ICML, 2023.

Thank you for bringing up these concerns. We’re glad to have the chance to address them in this discussion.

Q1. During search, each step in an unsuccessful path cannot receive any informative reward, offering little guidance and making the search inefficient. This issue is common in retrosynthesis search algorithms such as MCTS and Retro*. For example, in PDVN-MCTS, each step in an unsuccessful path is assigned a reward of $-1$, which provides insufficient learning signal and tends to prioritise shorter paths. To address this inefficiency, we train a stronger one-step model to guide node expansion more effectively and narrow the search space. In our experiments, InterRetro expands each molecule with only $10$ plausible actions, compared to $50$ in template-based methods such as PDVN. As a result, the search tree generated by InterRetro is significantly smaller, which directly improves efficiency by avoiding unnecessary exploration of shallow but infeasible paths.

Q2. We provide our responses to the two questions as follows:

Human-in-the-loop platform. We would like to clarify that a human-in-the-loop platform allows chemists to interactively sketch molecules and receive immediate feedback on the molecule's synthetic routes. This workflow typically involves three steps: (1) drawing or editing a target (or intermediate) molecule on screen; (2) receiving a newly suggested full synthetic route; and (3) accepting, rejecting, or further modifying nodes in the synthetic tree. Our proposed method accelerates the second step — suggesting a new route almost instantly, thereby keeping the chemists’ creative flow uninterrupted.

For your question, we agree that laboratory experiments can take days to weeks, which is significantly longer than the time required by retrosynthesis planning algorithms. However, improving the efficiency of the overall design-and-experiment cycle is a systemic challenge that extends beyond retrosynthesis planning alone and is not the primary objective of our proposed algorithm.

Synthesizability score. We agree that synthesizability scoring models are valuable for fast inference. However, quantifying synthesizability by explicitly examining synthetic routes remains an important and complementary approach (Liu et al., 2017; Parrot et al., 2023). In particular, Skoraczyński et al. (2023) investigated whether synthetic accessibility scores can reliably predict the outcomes of retrosynthesis planning. The best-performing methods in their study achieved an AUC of 0.90 and an accuracy of 0.81, indicating that many ponential molecules could still be missed during screening. Given InterRetro’s efficient and realistic route generation, we believe it can contribute meaningfully to more accurate synthesizability evaluation.

---

Thank you for attending the discussion. We’d be grateful for any further thoughts or feedback you might share.

References

Liu, Bowen, et al. Retrosynthetic reaction prediction using neural sequence-to-sequence models. ACS Central Science, 2017.

Skoraczyński, Grzegorz, et al. Critical assessment of synthetic accessibility scores in computer-assisted synthesis planning. Journal of Cheminformatics, 2023.

Parrot, Maud, et al. Integrating synthetic accessibility with AI-based generative drug design. Journal of Cheminformatics, 2023.

Thank you for recognising the novelty of our idea and the quality of our presentation. We have carefully addressed your concerns point by point and revised the manuscript accordingly. We hope that our responses satisfactorily resolve the issues raised.

W1. The routes provided in the supplementary materials are the synthetic routes generated by InterRetro on the Retro*-190 dataset, as stated in Section 5.1. We have improved the documentation and updated the anonymous repository, including a requirements.txt file and a step-by-step usage example. We believe that incorporating your suggestions enhances the reproducibility of our work.

W2. Thank you for raising the important concerns regarding baselines. AiZynth (from AstraZeneca) and ASKCOS (from MIT) are software platforms that allow users to deploy their own retrosynthesis algorithms via a user-friendly interface; they do not introduce new algorithms themselves. By default, both platforms implement a template-based search algorithm, which corresponds to the Template-MCTS baseline included in our original manuscript (Table 1). InterRetro significantly outperforms this baseline (20.00% vs. 95.78% success rate in Direct Generation, and 62.63% vs. 100.00% success rate when the model call budget is 500).

To further address the concern and strengthen our SOTA claim, we have run more experiments comparing InterRetro against two recent compelling algorithms: DreamRetroer (Zhang et al., 2025) and RetroCaptioner (Liu et al., 2024). As shown in the following table, InterRetro demonstrates superior performance, particularly in the search-free Direct Generation (DG) setting. We believe this new comparison reinforces InterRetro's state-of-the-art status.

Due to the limited rebuttal period, we have focused on the Retro*-190 benchmark. Results for ChEMBL-1000 and GDB17-1000 will be included in the final version of the paper.

W3. A key advantage of our method is its computational efficiency at inference time. We benchmarked the wall-clock time on a single NVIDIA RTX A5000 GPU. Our results show that Direct Generation takes approximately 0.88 seconds per molecule, while a search with a 500-model-call budget requires 8.53 seconds. The total wall-clock time required to solve the 190 molecules in the Retro*-190 dataset is presented below, demonstrating that InterRetro solves target molecules much faster than recent algorithms.

W4. Our training and evaluation protocol follows current methods such as PDVN (Liu et al., 2023) and Retro* (Chen et al., 2020). The training set provides target molecules for the model to learn how to construct valid synthetic trees. The trained models are then evaluated on the test set based on their success rate in constructing valid trees and the length of those routes. The training set consists of the shortest routes synthesizable using USPTO-50k reactions and eMolecules building blocks. The test set is constructed by extending route extraction, ensuring no overlap with the training set, and applying additional filtering to make the test set more challenging. More details can be found in Section 5.1.2 of Chen et al. (2020).

Thank you for recommending the PaRoutes dataset. We have tested InterRetro on the set-n1 and set-n5. Due to the large size of each set (10,000 molecules), we evaluated only the first 200 molecules in each. As shown in the table below, InterRetro shows consistently strong performance, as it does on Retro*-190, ChEMBL-1000, and GDB17-1000. This supports the objectivity and robustness of our chosen benchmarks.

W5. We believe that testing on multiple molecule datasets is helpful to comprehensively evaluate each algorithm, thus we choose multiple distinct benchmarks - Retro*-190, ChEMBL-1000 and GDB17-1000. Retro*-190 dataset serves as the standard benchmark in the field, enabling direct comparison with prior work. ChEMBL-1000 from ChEMBL dataset (Zdrazil et al., 2023) is a manually curated collection of bioactive molecules exhibiting drug-like properties, making it an appropriate benchmark for retrosynthesis planning. Regarding GDB17-1000 (Ruddigkeit et al., 2012), it covers a size range common for typical drugs and lead compounds, featuring unusual ring systems and connectivity patterns that challenge generalisation capabilities. If a method performs well on both drug-like molecules (ChEMBL) and structurally diverse, synthetically challenging molecules (GDB17), it demonstrates strong generalisation beyond memorisation of common reaction patterns. Thus we believe that these three datasets are appropriate benchmarks for retrosynthesis planning. We have added this justification to Section 5.1 to clarify our evaluation strategy.

Q1. Our analysis on the Retro*-190 benchmark shows that InterRetro tends to find more direct routes than those in the dataset. Specifically, the valid routes from our search-free Direct Generation (DG) model have an average depth of $4.54$, which is shorter than the dataset's average of $6.96$. As shown in the table below, this trend of finding valid routes continues when a search budget is applied. Unfortunately, we cannot perform a similar analysis for the ChEMBL-1000 and GDB17-1000 benchmarks as they do not provide ground-truth synthesis routes.

Q2. While our primary objective is to discover valid synthetic routes, we agree that evaluating the model's ability to reproduce known routes is an important indicator of its feasibility. When planning with Retro* and maximal model call 500, 38.9% of the known routes appears in the search tree constructed by InterRetro, while 46.4% of the known routes appears in the search tree built by standard Graph2Edits. Although slightly lower than the standard Graph2Edits, our finetuned model is still effective at recovering the routes presented in the dataset. In our experiments, InterRetro can represent valid but genuinely different synthetic strategies, often shorter than the routes presented in the dataset (as shown in the supplementary materials). This suggests our model balances between learning from established chemistry and discovering potentially more efficient alternatives.

Q3. For the dataset usage experiments shown in Figure 4a, subsets of molecules were selected via random sampling from the USPTO-50k training data. To ensure robust results for each sampling ratio (1%, 2%, 5%, and 10%), we created three independent, randomly sampled subsets and trained a separate model on each. The performance reported in the figure is the average across these three runs. This process ensures our results are objective and robust to sampling variations.

Q4. We appreciate the reviewer pointing out this interesting result, which is indeed reasonable on closer inspection. The Direct Generation (DG) model is entirely focused: it makes a single, decisive prediction for the best reaction at each step. In contrast, a search algorithm like Retro* or MCTS expands a much wider tree by considering multiple (e.g., 10) possible reactions at each step. When the search budget (i.e., the number of model calls) is severely limited, the algorithm cannot explore these numerous branches deeply enough to reliably identify the optimal path, increasing the chance of selecting a suboptimal action compared to the focused DG approach.

---

We would like to thank you again for raising important questions about our submission. After incorporating this valuable feedback, we believe that our current version becomes much stronger. Have we sufficiently addressed the main concerns? Please feel free to let us know if there are additional concerns or questions.

References

Chen, Binghong, et al. Retro*: Learning retrosynthetic planning with neural guided A* search. ICML, 2020.

Liu, Guoqing, et al. Retrosynthetic planning with dual value networks. ICML, 2023.

Liu, Xiaoyi, et al. RetroCaptioner: Beyond attention in end-to-end retrosynthesis transformer via contrastively captioned learnable graph representation. Bioinformatics, 2024.

Ruddigkeit, Lars, et al. Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17. Journal of Chemical Information and Modeling, 2012.

Zdrazil, Barbara, et al. The ChEMBL database in 2023: A drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic Acids Research, 2023.

Zhang, Xuefeng, et al. A data-driven group retrosynthesis planning model inspired by neurosymbolic programming. Nature Communications, 2025.

Thank you for your detailed feedback and for the time you spent in reviewing our manuscript. We have addressed every point you raised and will incorporate the corresponding changes into the final version.

Weaknesses. Thank you for raising this important concern. We agree that isolating the source of performance improvements is crucial for properly attributing the effectiveness of our approach. To address this, we provide the following ablation results on the Retro*-190 dataset. The first two rows compare the performance of the original pretrained Graph2Edits model (used with Retro*) and the finetuned version (InterRetro). The substantial improvement in performance after finetuning demonstrates that the self-imitation learning process and the worst-path objective meaningfully enhance the base model’s effectiveness.

In the next two rows, we evaluate a template-based model (Template - Retro*) and its finetuned variant using the same training framework. Again, we observe a significant performance gain after fine-tuning, even though the underlying single-step model is not as strong as Graph2Edits. This indicates that our learning framework improves multi-step planning across different single-step models, not just Graph2Edits.

These results suggest that the performance gains stem from the proposed training methodology rather than solely relying on the strength of Graph2Edits.

Q1. We chose Graph2Edits as our single‑step model for three practical reasons:

• Better generalisation. Graph2Edits edits molecules at the bond‑ and atom‑level, so it can apply the same local transformation to many unseen but similar leaving groups. Template‑based methods, which store entire reaction patterns, often fail when an exact template match is missing.

• Fast and lightweight. The Graph2Edits model requires less than 10 MB of memory, whereas a template-based method, due to the storage of template information, can consume up to 1 GB, which hinders its applicability in parallel computation. At inference time Graph2Edits model is far cheaper than large generative models (e.g., transformer–decoder architectures), which speeds up both training and route generation.

• High‑quality support constraint. The Graph2Edits paper reports state-of-the-art single‑step accuracy and shows that its predicted edits follow well‑established reaction chemistry. By constraining our policy to this edit vocabulary, we preserve chemical plausibility while still allowing our advantage‑weighted updates to re‑rank the edits.

Q2. Thanks for your suggestions. We have reported the results in our response to the Weakness above. We have revised the submission to reflect the ablation results.

---

We would like to thank you again for raising important questions about our submission. Due to this year's rebuttal policy, we cannot upload a revised PDF to reflect these changes. However, we believe that after incorporating the valuable feedback our current version becomes much stronger. Have we sufficiently addressed the main concerns? Please feel free to let us know if there are additional concerns or questions.