We are extremely appreciative of the reviewer’s feedback and suggestions to improve our work. We are glad that the reviewer considers our work to “potentially introduce a new paradigm for future multi-step retrosynthesis planning”.

We provide additional experiment results in the following link: https://drive.google.com/file/d/1RIcdPo-uV5Fibbllk2COJl9niUhrVAFk/view?usp=sharing

fair comparison

We follow the Rootaligned paper to set the number of test-time augmentation to 20. In updated Table 1, we extend the search time for each molecule to 60 minutes. Only Rootaligned cannot finish all calls. Due to the time constraint for rebuttal, we will adjust the test-time augmentation in our future version. Note we do not intend to limit the solving time for comparison benefit, but only to rule out methods with extremely long solving times.

whether the synthesis path proposed by the model has been seen during pretraining. Experiments to show the routes for the provided molecules.

We conducted an ablation study for direct user queries and presented the results in Table 3. The results suggest that the models do not simply “remember” synthetic routes from their training data. We provide the routes for four suggested molecules discovered by our method in the PDF.

for synthesizable molecular design

In contrast to approaches like [1] that start with an unsynthesizable molecule and then “project” it to a synthesizable form—often compromising similarity and potentially degrading properties (For instance, Table 2 in [1] observes that the average similarity can be modest)—our method jointly optimizes molecular properties and ensures synthesizability from the outset. By directly sampling candidate molecules with high properties and verifying their synthetic accessibility through route sampling, we avoid an additional “projection” step and minimize property loss. We will discuss [1] in our related work as an alternative way to find optimized molecules with synthesis pathways.

The size of the molecule set will increase

While it is true that the set can increase with each additional step, our framework mitigates this issue in practice. Specifically, during mutation, we provide the LLM only with the partial route starting from the first invalid step, rather than resupplying the entire route. Furthermore, purchasable molecules in the set can be filtered out and do not need to be reintroduced to the LLM. Our experiments indicate that this approach remains manageable in real-world scenarios.

Replace the chemical reaction proposed by the LLM with a retrieved one. Label leakage.

This procedure does not introduce label leakage because we rely only on reactions known to exist (i.e., those in the database). In fact, the single-step specialized models are also trained on reactions extracted from USPTO-FULL. In the original Retro* paper, they even filter the test route dataset by only keeping the routes whose reactions are all covered by the top 50 predictions by the single-step model (Section 5.1.1).

The main role of retrieving is to use as a template and approximate information as a reference.

How to verify the feasibility of the synthesis pathway

Yes. Since the reaction templates used are from reliable databases, we can apply the reaction template using the chemical tools rdchiral to verify.

Practical advantages of the proposed method

Through detailed analysis of the failure cases, we observed that the main issue is not from the LLM itself but from the partial reward mechanism—namely, that optimal SCScore does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop.

To address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines. This indicates that LLMs offer a powerful framework for complex, multi-step decision-making in scientific domains, and we are the first to show that LLMs can achieve comparable or even better performance compared with specialized models. This also provides evidence for solving other science problems with a similar structure.

Regarding the reaction database, it is worth noting that specialized models also require such databases: for instance, localRetro formulates one-step retrosynthesis as a classification problem, necessitating a pre-defined set of reaction data.

At last, we would be happy to engage in further discussions with the reviewer regarding these topics or any additional concerns the reviewer has – please let us know! We thank the reviewer again for their time spent with our paper. If the reviewer finds that our new experiments, analyses, and discussions are useful for improving the paper, we would be grateful if the reviewer would consider a fresher evaluation of our paper.

[1] Luo S, et al. Projecting Molecules into Synthesizable Chemical Spaces, ICML2024

[2] Chen B, et al. Retro*..., ICML2020

We thank the reviewer for their positive feedback! We are thrilled the reviewer finds our LLM-based retrosynthesis framework “could be further extended to an autonomous laboratory framework or a molecular synthesis and generation pipeline using LLMs”.

Below, we address the reviewer’s questions about experiments as well as the technical contribution of the paper. We have provided additional experiment results in the following link: https://drive.google.com/file/d/1ANfpzJLJgJh83saOwBzK4AMgenb41Jnc/view?usp=sharing

Performance of LLM-Syn-Planner

First, we conducted error analysis for LLM-Syn-Planner and observed that the main shortcomings stem not from the LLM itself but from the partial reward mechanism—namely, that a strong SCScore does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop.

To address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines.

Other general LLMs

We have included new experimental results for the open-source models DeepSeek V3 and DeepSeek R1. The performance of DeepSeek V3 is shown in Table 1, while the results for DeepSeek R1 appear in Table 4. Interestingly, although DeepSeek R1 is designed as a reasoning model, it performs worse than DeepSeek V3. Moreover, because DeepSeek R1 includes its thinking process in the output, its overall cost is roughly three times higher than that of DeepSeek V3.

Technical contribution

Our primary goal in this work is to investigate how effectively LLMs can be leveraged for retrosynthesis tasks. The key contribution of this work is to apply the idea of an evolutionary algorithm to generate the entire decision trajectory to the problem of retrosynthesis (which is new). In addition, the workflow we propose such as how LLMs can serve as the proposal and how a proper proxy and retrieval can augment the process is new. We will rephrase the text to reflect this. The strongest contribution, in our humble opinions, of this work is the demonstration of the strong performance of the retrosynthesis planning problem with our LLM-based tree-sampler EA method. To the best of our knowledge, we are the first to show that LLMs can achieve comparable or even better performance compared with specialized models. This also provides evidence for solving other science problems with a similar structure.

A cost analysis is also needed

In Figure 1, we present a cost analysis showing the average cost of each discovered route across the four datasets. As the route length increases, the cost of finding that route also rises, but at an acceptable rate. Given the rapid advancements in LLMs, we anticipate that these costs will decrease over time. For example, Llama 3 outperforms Llama 2 significantly despite being released only one year later.

Ablation of the prompt

In Table 2, we present an ablation study examining how the <Explanation> component in the overall prompt and the ‘Rational’ tag at each step affect performance. Our findings show that both elements enhance the language model’s reasoning capabilities, underscoring the importance of including thinking/explanation steps in the prompt.

We are happy to address any additional comments or concerns the reviewer may have. Otherwise, we would appreciate it if the reviewer continues to view this work positively. Thank you once again for your time and thoughtful feedback.

We thank the reviewer for their positive feedback! We are glad that the reviewer considers our method to be “novel” and our paper to be “well-written”. We wholeheartedly agree with the reviewer that this method can “be easily adapted to molecular design problems, unifying LLM-augmented reaction decision programs”.

We have uploaded the results of the new experiments and the required algorithm box in https://drive.google.com/file/d/1IsuWkKzzLrNM0aYdbKx-3n2YnqkaECgg/view?usp=sharing

New LLM experiments:

We first did an error analysis for LLM-Syn-Planner and observed that the main errors in LLM-Syn-Planner mainly come from the partial reward mechanism—a strong SC Score does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop. To address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines.

Based on this, we conducted experiments on an open-source model DeepSeek-V3, and the results are shown in Table 1. Both LLMs achieve comparable or even surpass the baseline specialized models, which demonstrates the effectiveness of the LLM-Syn-Planner framework. To the best of our knowledge, we are the first to show that current LLMs can achieve decent performance on multi-step retrosynthesis planning tasks, paving the way for an autonomous LLM agent framework for molecule synthesis and optimization.

Moreover, we also tested DeepSeek R1. The results are shown in Table 4. Interestingly, although DeepSeek R1 is designed as a reasoning model, it performs worse than DeepSeek V3.

Evaluation metrics

We conducted a cost analysis shown in Figure 1. The figure shows the average cost of each discovered route across the four datasets. As the route length increases, the cost of finding that route also rises, but at an acceptable rate. As open-source LLMs improve, it is reasonable to believe that LLM-Syn-Planner will also improve, so computational cost will be mitigated in the near future. For the route length, due to the time constraint for rebuttal, we will add it in our future version.

Definition of the functions INITIALIZATION and MUTATION

We show the definition in Algorithm 1 and Algorithm 2.

INITIALIZATION queries the LLM to produce an initial route for the given target molecule. It uses a RAG strategy by first identifying molecules similar to target_molecule in retrieval_db, retrieving their known routes, and providing these references to the LLM to guide route generation.

MUTATION prompts the LLM to revise any invalid parts of current_route beginning at invalid_step_index. The LLM may directly fix the incorrect step, substitute it with a more suitable reaction, or even propose an alternative multi-step path if needed.

typo in the algorithm box

Yes in line 228, R_0 should be P_0. In line 231, t should start from 0. Thanks for pointing out. We will fix this typo in the next version.

solve rate with more model calls (N)?

When the number of model calls for LLM-Syn-Planner increases from 100 to 300, the solve rate shows a significant improvement. However, increasing the number of model calls from 300 to 500 yields only minimal additional gains. One factor is that our partial reward does not always correlate with final synthetic success (e.g., a molecule with an SC score of 1 might still be extremely difficult to synthesize in practice).

For reaction-level evaluation, how is the chemical feasibility of the proposed reaction assessed?

To assess the chemical feasibility at the reaction level, we verify whether the product molecule in this step could validly undergo the backward reaction. In practice, we use the rdchiralRun function from the rdchiral package to check if the product matches the reaction template such that the backward reaction is recognized. If rdchiralRun confirms that the product can serve as a valid reactant under the backward reaction, we regard the forward reaction as chemically feasible.

How does the LLM identify substructures, and in what form are these substructures represented and used to extract templates? (line 197)

We begin by prompting the LLM to identify relevant substructures and functional groups in the product molecule. In response, the LLM provides these substructures in SMILES format. Next, we verify each suggested substructure SMILES to ensure it is indeed present within the product structure. Once validated, we use these SMILES to compute Tanimoto similarities between the identified substructures and the product molecules in our reference reaction database to extract templates.

We hope that our responses were sufficient in clarifying all the great questions asked by the reviewer. We sincerely thank the reviewer for their time and consideration. We would be happy to engage in further discussions if the reviewer has any additional questions or concerns.

We thank the reviewer for their feedback and thoughtful questions. We are ecstatic that the reviewer finds our method “crucial for tackling the combinatorial complexity of retrosynthesis”. We have updated the new results here: https://drive.google.com/file/d/1Car72zPjMG9__41PfJAGbMkbNObYHjBS/view?usp=sharing

Error analysis

We first did an error analysis and observed that the main errors in LLM-Syn-Planner mainly come from the partial reward mechanism—a strong SC Score does not guarantee a molecule is in the purchasable set. Consequently, the algorithm can get stuck in an infinite loop. To address this, we simply added a penalty for repeated entries, which substantially improved performance and, as shown in Table 1, allowed LLM-Syn-Planner to outperform baselines.

Underperformance

After resolving the infinite loop issue in our error analysis, we present the updated results in Table 1. LLM-Syn-Planner now achieves superior performance compared to the baseline specialized models. Additionally, we include a new open-source LLM, DeepSeek-V3. Our findings show that both LLMs perform well within our framework, demonstrating the effectiveness of LLM-Syn-Former.

Additional ablation studies

We conducted several ablation studies to evaluate different design choices: route format, the use of molecule RAG, the reward signal, EA parameters, and prompt robustness.

In a separate ablation study for direct user queries (Table 3), we queried each LLM 100 times for a fair comparison. On the USPTO easy dataset, both LLMs solved fewer than 10 routes, and on the Pistachio Hard dataset, they failed in nearly all cases. These results suggest the models do not merely “remember” synthetic routes from their training data.

Additionally, Figure 1 provides a cost analysis, while Table 4 presents an ablation study using DeepSeek-R1, and Table 5 studies the statistical significance.

problematic claim & our technical contribution

We agree with the reviewer the search algorithm itself is not new. The key contribution of this work is to apply the idea of an EA to generate the entire decision trajectory to the problem of retrosynthesis (which is new). In addition, the workflow we propose such as how LLMs can serve as the proposal and how a proper proxy and retrieval can augment the process is new. We will rephrase the text to reflect this.

Our primary goal in this work is to investigate how effectively LLMs can be leveraged for retrosynthesis tasks. The main contribution is our demonstration that an LLM-based tree-sampler EA method can deliver strong performance in multi-step retrosynthesis planning—surpassing even specialized models. We believe this finding not only validates the utility of LLMs in retrosynthesis but also offers a valuable contribution to the broader research community.

acknowledge EA for molecule design, Partial reward in RL and RAG in NLP

Thank you for pointing out the literature. We cited Jensen 2019 and here we adopt a similar EA framework, but we design it for a sequential decision-making problem that is highly different (search for a single molecule vs a synthesis route). We use LLMs as the proposal with additional consideration of partial reward and retrieval-based augmentation. We will acknowledge them in the next version.

Compared with MolLEO

Our work and MolLEO share only the concept of combining EA with LLMs, but address fundamentally different domains. For task domains, MolLEO focuses on small molecule optimization (naturally represented in SMILES strings). But LLM-Syn-Planner is designed for a sequential decision-making problem. For task challenges, molecule optimization uses LLM to directly edit the molecule and can easily get ground truth feedback from an oracle function. However, retrosynthesis planning poses a more complex, sequential decision-making challenge within a constrained search space.

Scalability Concerns

We have shown that the open-source model DeepSeek-V3 can outperform specialized models, suggesting that as open-source LLMs continue to improve, so will LLM-Syn-Planner, thereby reducing computational costs. Currently, the primary efficiency bottleneck is the OpenAI API rate limit, which caps the number of tokens processed each minute. Another bottleneck stems from the raw reaction database; classifying or indexing these reactions would improve the efficiency of RAG.

question about backward reaction

In retrosynthesis planning, especially in top-down planning algorithms, the reaction is considered in reverse. We denote this by writing the reaction as Product >> Reactant1.Reactant2, is often referred to as a backward reaction.

We sincerely thank the reviewer for their valuable feedback. We hope our rebuttal has effectively addressed their questions and concerns. If the reviewer is satisfied with our responses, we kindly ask them to consider a reassessment of our paper. We remain more than happy to address any additional questions or concerns that may arise.