Active Retrosynthetic Planning Aware of Route Quality

We sincerely thank the reviewer for providing valuable feedback. We detail our response below point by point. Please feel free to let us know if you have any additional concerns or questions.

Real-life scenarios

We wholeheartedly agree with the reviewer that the quality metric required by our framework in a real-life scenario should be expensive but not prohibitively so. This consideration constitutes the primary motivation behind our active planning framework, aiming to query a minimum number of reaction quality annotations while still planing high-quality routes.

While our current implementation involves querying the surrogate model, our inspiration is drawn directly from real-life retrosynthesis planning scenarios, such as in online softwares like SYNTHIA, where chemists are pivotal end users. In this context, integrating chemists as valuable resources into the AI planning process will be invaluable for planning routes that are not only feasible but also of practical high-quality. We envision the successful deployment of our framework in this scenario for several reasons.

• Online annotation by chemists introduces minimal time delays and manageable labor costs, making it an ideal candidate for a route quality metric that is expensive but not prohibitively so.
• Our framework is intentionally designed to be compatible with various types of annotations, including a coarse-grained quality rating from 0 to 10. We believe such a rating is sufficient for the planner to make satisfactory decisions. Additionally, this rating can also be seamlessly integrated into our current framework by replacing the bucket index to which a quality value belongs (see details in Section 3.2 in the revision) with this discrete rating.
• Chemists contribute valuable insights beyond mere reaction yields, such as knowledge about preferred reactions in real-world synthesis contexts, which can include factors like toxicity, material costs and work-up difficulty (post-process, like purification or separation).

Related works

We thank the reviewer for pointing out these recent important related works [2-6], which we have cited and discussed in the revision version.

• Related works on multi-step retrosynthesis planning
  • [5] employs an RL-based method to improve single-step practicality, aiming to discover shorter routes with lower costs. However, [5] does not consider real-world reaction qualities, by treating not-dead reaction costs as 1 and evaluating routes based solely on their length.
  • As a complementary, our work contributes to finding high-quality routes by actively querying the real-world reaction qualities. We have proven the capability to reduce query costs while maintaining high route qualities. Importantly, our active framework is compatible with retrosynthetic planning methods like [5].
• Related works on single-step retrosynthesis prediction
  • [2] addresses the identification and completion processes by employing local template prediction combined with a global attention mechanism.
  • [3] tackles the concern that fixed parameters might be sub-optimal by introducing a robust non-parametric local reaction template retrieval approach.
  • [4] resolves the issue of SMILES neglecting the characteristics of molecular graph topology and reaction atom transformations. Our focus on retrosynthetic planning can readily integrate with the success achieved by recent state-of-the-art single-step retrosynthesis prediction methods.

Nitpicks

We appreciate your advice to help us improve this paper and revise the mentioned nitpicks in the revision.

[1] Binghong Chen et al. "Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search".

[2] Chen et al, "Deep Retrosynthetic Reaction Prediction using Local Reactivity and Global Attention"

[3] Xie et al, "Retrosynthesis Prediction with Local Template Retrieval"

[4] Zhong et al, "Root-aligned SMILES: a tight representation for chemical reaction prediction"

[5] Liu et al, "Retrosynthetic Planning with Dual Value Networks"

Q1: Could you expand on "critic takes both the current molecule and its preceding reaction cost as input"? Is the previous reaction cost literally concatenated into the molecule representation somewhere as a single floating point number?

The preceding reaction cost is concatenated in the format of its representation as $\mathcal{O}(a^r,a^q)$ in Eq.(2). Concretely,

• When $a^q=1$, we derive $\mathcal{O}(a^r,a^q)$ from (1) discretizing continuous reaction quality values into $N^M$ discrete buckets, (2) learning $N^M$ trainable embeddings in $d^M$ dimensions for all buckets within our critic, and (3) determining the bucket index that the queried quality value $u$ belongs to and thereby the associated bucket embedding.
• when $a^q=0$, $\mathcal{O}(a^r,a^q)=\mathbb{M}$ as a $d^M$-dimensional trainable embedding.
• In our implementation, we consider $d^M=512$ and $N^M=18$ buckets which are defined in Fig 6 in the revision. These buckets are obtained via (1) collecting ~28M reactions during planning by GRASP and Retro$^*$, (2) computing their reaction qualities by our surrogate model, and (3) defining the bucket boundaries to ensure that each bin covers a similar number of reactions.

Q2: Is Equation 6 fully correct? I am confused about the fact that it does not include the discount factor $\gamma$, and also the way $N(s)$ is used seems slightly off; I may be wrong though.

We apologize for any confusion arising from the equations with respect to $Q^*$ and we further clarify Eq.(5) and Eq.(6) in the revision version.

• The visit count number $N(s)$ is used for the reaction action selection: $a^r_t=argmax_{a^{r}\in ch(s_t|T)}Q^*(s_t,a^{r})$ where $Q(s_t, a^r_t)$ is a value initialized by our critic network $Q_\theta(s_{t+1},\mathcal{O}(a^r_t,\pi_\phi(a^r_t)))$ and updated afterwards. The $Q^*$ function, quantifying the molecular synthesizability, is computed using the following equation:

$$Q^*(s_t,a^r_t)=Q(s_t, a^r_t)+\beta\frac{\sqrt{N(s_{t-1},a^r_{t-1})}}{1+N(s_t,a^r_t)}$$

• The visited count number $N(s)$ is also employed in the update step. The $Q(s_t, a^r_t)$ values are updated in association with the visit counts of the state-action pair using the following equation where $Q'$ is the $Q$ values of newly added state-action pairs. This update process also incorporates the discounting factor $\gamma$ as mentioned in the main text.

$$Q'(s_t,a^r_t) = Q(s_t,a^r_t) + \frac{1}{N(s_t,a_t)}(\gamma Q'-Q(s_t,a^r_t))$$

Q3: Could you elaborate on how the quality metric is used by the baseline algorithms like Retro* to produce the results from Table 2? I understand that for the baselines it is assumed the quality metric is always queried, which leads to 100% query rate. Could you discuss how the resulting quality value is used by the algorithms themselves during optimization?

Retro* [1] leverages the A* search algorithm with the value of a molecule defined as

$$$$

V_t(m|T)=g_t(m|T)+h_t(m|T).

Hi Reviewer 23Kg,

We would like to follow up to see if our response addresses your concerns or if you have any further questions. We would really appreciate the opportunity to discuss this further if our response has not already addressed your concerns.

Thank you again!

Thank you sincerely for your thoughtful and positive feedback on our work. We are particularly grateful for your recognition of the various aspects of our research. Below, we have provided a detailed explanation for your remaining concern as follows. Please do not hesitate to let us know if you have any further questions.

Q1: In the ablation study, why doesn't the actor+critic consistently and significantly improve the success rate compared to the random+critic baseline, as observed with normalized route quality?

Q2: Why doesn't the success rate change monotonically with the query rate, unlike the normalized route quality?

We address Q1 and Q2 together because of their high correlation. We have added a discussion into Section 4.2 and a case illustration in Appendix D in the revision.

• (1) The success rate and route quality are two distinct and potentially conflicting objectives, as illustrated in the case study of Fig 8 in Appendix D. In the figure, the root molecule $M_0$ has three candidate reactions, and $R_0$ is identified as a high quality reaction. However, the child molecule $M_1$ of $R_0$ is an unexpandable dead node with no further reaction candidates.
  • If the planner selects based only on observable next-state molecular structures of $M_1$, $M_2$ and $M_3$, being agnostic of the reaction quality values of $R_0$, $R_1$ and $R_2$, it might select $R_2$ for its most synthesizable next-state molecule $M_3$.
  • Alternatively, given observable reaction quality values $R_0$, $R_1$ and $R_2$, the planner could be misled into selecting $R_0$ due to its highest route quality.
• (2) The "actor" in Table 3/4 is a policy network $\pi_\phi$ that decides whether to query the quality of reactions. By taking the reaction quality as rewards (see Eq.(1) for details), the actor is trained to prioritize querying those reactions that contribute to high-quality routes.
• Considering (1) the disparity between high-quality and high success rates and (2) the actor optimized towards high-quality only, it is reasonable that
  • actor+critic does not show consistent and significant improvement on success rates compared to random+critic;
  • the success rate does not change monotonically with the query rate.

Writing

Q7: The term “reaction cost” is misleading and inconsistent: “cost” usually refers to some undesired property that should be as low as possible, however, this paper defines cost as yield (i.e., higher “cost” indicates better quality). This is misleading and counterintuitive. Meanwhile, “observing in Fig, 2 that a molecule with a low preceding reaction cost should be prioritized to expand first”, is clearly inconsistent with Fig. 2 (i.e., low-cost=low-quality route should not be prioritized).

We apologize for any confusion arising from the term "reaction cost". For a more straightforward and intuitive understanding, we have revised all instances of the term "reaction cost" into "reaction quality" as well as Fig 2 in the updated version.

Q8: The description of Expansion in the inference stage is not clear. $Q^*$ sometimes refers to a value, or a function (i.e., in Eq. (5)), which is confusing. What is $Q^*(a)$ and $Q^*(s)$ exactly?

For a better understanding, we have replaced $Q^*(s_t)$ and $Q^*(a^r_t)$ with a unified form $Q^*(s_t, a^r_t)$ in Eq 5 and Eq 6. $Q^*$ is a function that quantifies the synthesizability of a state-action pair as outlined below. Inside the equation, $N(s, a^r)$ denotes the visit count number. $Q(s_t, a^r_t)$ is a value initialized by $Q_\theta(s_{t+1},\mathcal{O}(a^r_t,\pi_\phi(a^r_t)))$ and updated afterwards.

$$Q^*(s_t,a^r_t)=Q(s_t, a^r_t)+\beta\frac{\sqrt{N(s_{t-1},a^r_{t-1})}}{1+N(s_t,a^r_t)}$$

During a selection step, We recursively perform a reaction action selection $a^r_t=argmax_{a^{r}\in ch(s_t|T)}Q^*(s_t,a^{r})$ and obtain the next state $s_{t+1}$ by tranition function $\mathcal{P}$ until reaching a leaf molecule node. In a update step, we perform a $Q(s_t,a^r_t)$ value backward traversal as follow where $Q'$ is the $Q$ values of newly added state-action pairs.

$$Q'(s_t,a^r_t) = Q(s_t,a^r_t) + \frac{1}{N(s_t,a_t)}(\gamma Q'-Q(s_t,a^r_t))$$

Method

Q3: How is the state encoder $\mathcal{E}$ trained? It should also wrap $\mathcal{E}$ in Eq. (4), right?

We first apologize for any ambiguity. We have updated Eq.(4) along with its descriptions in response.

• If $\mathcal{E}$ is an offline estimator such as the value estimator in Retro*, it is trained before implementation in our framework. The parameters of $\mathcal{E}$ are frozen and not evolved in training of the critic.
• If $\mathcal{E}$ is an online estimator, it can be regarded as part of the critic parameters and is wrapped with the other critic parameters for training.

Q4: In Selection, is there any exploration or possibility to generate multiple routes?

We thank the reviewer's great comment, and have accordingly updated Section 3.3 to detail the definition of $Q^*$ which involves exploration.

Concretely, the function $Q^*$ is defined as

$$Q^*(s_t,a^r_t)=Q(s_t, a^r_t)+\beta\frac{\sqrt{N(s_{t-1},a^r_{t-1})}}{1+N(s_t,a^r_t)}$$

which follows the UCT(Upper Confidence Bound applied to Trees) function to balance between exploitation of the route with the maximum $Q$ and exploration of those less frequently visited routes.

Q5: How is the masked value M set?

• We set the mask as a $d^M$-dimensional trainable embedding.
• Meantime, it is crucial to emphasize the consistency in the representation of $\mathcal{O}(a^r,a^q)$:
  • When $a^q=1$, we derive $\mathcal{O}(a^r,a^q)$ from (1) discretizing continuous reaction quality values into $N^M$ discrete buckets, (2) learning $N^M$ trainable embeddings in $d^M$ dimensions for all buckets within our critic, and (3) determining the bucket index that the queried quality value $u$ belongs to and thereby the associated bucket embedding.
  • when $a^q=0$, as mentioned above, $\mathcal{O}(a^r,a^q)=\mathbb{M}$ as a $d^M$ dimensional embedding.
• In our implementation, we consider $d^M=512$ and $N^M=18$ buckets which are defined in Fig 6 in the revision. These buckets are obtained via (1) collecting ~28M reactions during planning by GRASP and Retro$^*$, (2) computing their reaction qualities by our surrogate model, and (3) defining the bucket boundaries to ensure that each bin covers a similar number of reactions.

Q6: During inference, does the method require the external surrogate model to obtain the reaction cost? How accurate/reliable is this surrogate model?

• Our major contributions revolve around exactly an active planning framework that trains a querying actor/critic during training and later applies it during inference to plan a high-quality route with minimal reaction quality annotation from an external surrogate model or a chemist expert.
• Still, ours is compatible with existing multi-step planning methods such as Retro* as backbone planners. In scenarios where an external surrogate model or a chemist is unavailable, ours reduces to the backbone planner.
• In the response to Q1, we have empirically validated the effectiveness of our surrogate model. Besides, our case study in Appendix E showcases that ours achieves much higher route quality ratings by chemists. This outcome provides strong evidence that the surrogate model faithfully reflects yields.

We sincerely appreciate your constructive comments on this paper. We detail our response below point by point. Please kindly let us know if our response addresses the issues you raised in this paper.

Related works

We thank the reviewer for pointing out this recent and important related work [1], which we have duly cited and discussed in the revision.

• [1] introduces a novel multi-step planning approach via in-context learning, departing from conventional search algorithms. However, its primary objective is still evaluating success rates of planned routes.
• Our major contributions lie in an active framework capable of synsthesizing routes with not only high success rates but also high quality. Thus, we have proved the compatibility and orthogonality of ours with other multi-step planning strategies like Retro*; it is also readily extendable to state-of-the-art planning methods like [1].

Unverified claims

Q1: In reaction cost annotation, the authors adopt a surrogate model to provide reaction cost annotation, and hope “the model prioritizes the identification of high-yield reactions over high-frequency reactions”. Can the authors empirically verify this claim?

• Our active planning framework utilizes the reaction quality annotated by our surrogate model as rewards. Consequently, as long as reaction qualities predicted by the surrogate model show high correlations with yields, the model learns a policy that prioritizes high-yield reactions over high-frequency ones.
• We proceed to validate that reaction qualities by our surrogate model indeed demonstrates a high correlation with yields.
  • The training of our surrogate model involves two steps: (1) pre-training on the USPTO-MIT dataset, and (2) finetuning on an in-house expert dataset of routes featuring high-yield reactions. It is important to note that we introduce step (2) precisely to ensure that high predictive probabilities from our surrogate model align with high yields.
  • To evaluate our surrogate model, we resort to a route-with-yield test set. Following the method described in [2], we extract synthesis routes with yields from the USPTO-milligram-scale reaction yield dataset [3]. For evaluation purposes, we randomly select 200 routes, encompassing approximately 1000 reactions.
  • We thereby calculate the Pearson correlation coefficient(PCC) between the reaction quality predicted by our surrogate model and the literature yield. In Fig 7 of the revised manuscript, (a) illustrates the 0.059 PCC of the pre-trained model while (b) shows the 0.611 PCC of the finetuned model, providing strong evidence that the surrogate model accurately predicts yields.

Q2: Are there real cases when high-yield reactions and high-frequency reactions are not overlapping? Intuitively, high-frequency reactions are usually high-yield ones, otherwise they cannot be frequently collected in a dataset, right?

Frequently collected reactions in a single-step dataset are not necessarily high-yield, which we substantiate based on an analysis from [3] that explores yields reported in the open-source USPTO dataset.

• The USPTO dataset with reaction yields in sub-gram scale [3] contains a large number of reactions and a broad range of superclasses, and a reaction distribution closely resembling that of the USPTO single-step dataset, such as USPTO-MIT.
• The actual reaction yield distribution of the above dataset, originally presented in [3], is depicted in Fig 5c. Notably, a significant proportion of reactions within the dataset exhibits relatively low yields, affirming that the USPTO single-step dataset is not inherently biased to high-yield reactions.
• High-frequency $\neq$ high-yield Fig 5a (originally presented in [3]) shows various superclasses of reactions, where each color corresponds to a superclass and the coverage area of each color roughly represents the frequency of that superclass of reactions in the dataset. Combining Fig 5a and 5b, we conclude that high-frequency superclasses do not show a significant correlation with high yields. For example, the superclasses annotated in purple and cyan demonstrate low yields, with only the green reaction superclass corresponding to high yields in Fig 5b.

[1] Liu, Songtao, et al. "FusionRetro: molecule representation fusion via in-context learning for retrosynthetic planning".

[2] Binghong Chen et al. "Retro*: Learning Retrosynthetic Planning with Neural Guided A* Search".

[3] Philippe Schwaller et al. "Prediction of chemical reaction yields using deep learning"

Hi Reviewer BDYr,

We would like to follow up to see if our response addresses your concerns or if you have any further questions. We would really appreciate the opportunity to discuss this further if our response has not already addressed your concerns.

Thank you again!

I appreciate the authors providing detailed evidence and explanations to my questions. I have also read the updated version with improved clarity. Since my concerns were all addressed, I raised my score from 5 to 6.

W4: Prior work is not correctly attributed, regarding Segler (2018) [1], Coley (2019) [2], Schwaller (2020) [4] and Liu et al 2023 [3].

As clarified in the aforementioned ``Clarification on Our Major Contributions'', our primary contributions stand distinct from the prior works [1-4]. Furthermore, our proposed framework further is compatible with and enhances the real-world practicality of there existing planners.

• The planning methods in [1]-[4] only accommodate reaction cost metrics that are efficient to have yet inaccurate to reflect real-world reaction quality such as yields.
  • [1] utilzes a fixed heuristic based on starting material prices and reaction labor costs, unrelated to yields.
  • [2][3] consider the cost for each reaction as a uniform 1 and optimize towards shortest routes. However, a short route might have a lower yield than a long route.
  • [4] relies on a forward prediction likelihood biased to the high-frequency reactions and the SCScore. SCScore, which gauges molecule synthesizability (a.k.a. feasibility) but overlooks the real-world reaction performance quality.
• The methodologies above require the annotation of costs for every reaction along a route. In the pursuit of incorporating reliable reaction qualities derived from chemists or lab experimentation, which entail significant costs, these methods tend to be economically impractical for real-world deployment.
• Therefore, our essential contribution lies in introducing a framework designed to (1) actively decide when to query human evaluation (surrogate model in our current implementation) and (2) select a minimum number of reactions that are most informative to observe their quality annotations. Our framework not only reduces query costs but also renders eminently practical in real-world scenarios.
• By specifically addressing the challenge of annotation costs, a facet often overlooked by previous works, our framework not only accommodates but synergistically integrates with existing models. This compatibility facilitates the seamless implementation of their models within our framework, thereby extending their active query capabilities.

Questions

Q1: I would suggest to order the related work section, in particular the Multi-step planning section, by the order in which the works appeared, because subsequent works influenced each other.

We arrange the order of the related work section by the search algorithm previous studies used. We highlight the search algorithm categories, e.g. $A^*$, $MCTS$，$RL$, in the revision version updated.

[1] Segler, M., Preuss, M. & Waller, M. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 604–610 (2018).

[2] John S. Schreck, Connor W. Coley, and Kyle J. M. Bishop Schreck. Learning Retrosynthetic Planning through Simulated Experience. ACS Central Science 2019

[3] Guoqing Liu, Di Xue, Shufang Xie, Yingce Xia, Austin Tripp, Krzysztof Maziarz, Marwin Segler, Tao Qin, Zongzhang Zhang, and Tie-Yan Liu. 2023. Retrosynthetic planning with dual value networks. In Proceedings of the 40th International Conference on Machine Learning (ICML'23)

[4] Schwaller P, Petraglia R, Zullo V, Nair VH, Haeuselmann RA, Pisoni R, et al. Predicting Retrosynthetic Pathways Using a Combined Linguistic Model and Hyper-Graph Exploration Strategy. ChemRxiv. Cambridge: Cambridge Open Engage; 2019.

We thank the reviewer for providing feedback. Below, we detail our response point by point.

Clarification on Our Major Contributions

In response to the reviewer's summary characterizing our work as ``an RL based search algorithm that incorporates some definition of route quality for retrosynthesis is proposed'', we express concerns about potential misinterpretations of the major contributions of our work. To address this, we elaborate as follows.

• Our contribution lies in not first incorporation of route quality related metrics into retrosynthesis.
• Our contribution lies in being the first in acknowledging the cost of annotating reaction quality, and thereby enabling the practical incorporation of costly-yet-accurate route quality evaluation into retrosynthesis through an active and interactive manner. Our motivations and contributions can be encapsulated in the following points:
  • (1) There has been a lack of efficient-to-evaluate and meantime qualified route quality metrics; for example,
    • (a) Predicted yield by a prediction model [efficient but inaccurate]: constructing accurate and robust yield prediction models has historically been challenging, explaining existing models being confined to specific reaction types.
    • (b) Route length [efficient but unfaithful]: The illustration in Fig. 1 of the main text demonstrates that a short route might yield less than a longer one.
    • (c ) Single-step probabilities [efficient but inaccurate]: As noted in our Introduction, single-step probabilities primarily reflect feasibility rather than reaction quality.
    • (d) SCScore [efficient but inaccurate]: While SCScore quantifies synthesizability (feasibility), it neglects actual reaction performance quality.
    • (e) Chemist rating [more accurate but less inefficient]: Chemists offer an overall evaluation of reaction yield, but this process is less efficient than a prediction model.
    • (f) Lab experiment verification [accurate but inefficient]: While reflective of groundtruth reaction quality, including yields, this approach is exceedingly time-consuming.
  • (2) The preceding works, including [1]-[4] the reviewer mentioned, operate under the assumption that annotating route quality is labor-free and trivial. Consequently, they accommodate metrics (a)-(d), as incorporation of highly accurate evaluation (e)-(f) for every reaction along the planned route is deemed prohibitively costly.
  • (3) Thus, we are motivated to propose this "active" retrosynthetic planning framework, strategically minimizing the reactions where we query metrics such as (e)-(f) while sufficiently improve the route quality.
    • Example: In the case of a molecule requiring 25 synthesis steps, our framework endeavors to pinpoint only 2 key reactions (10% querying rate) for the evaluation of either (e) or (f).
    • This framework maintains compatibility with existing retrosynthesis planning methods, a validation substantiated in our experiments on Retro* and GRASP. Take Liu et al. 2023 [3] as an example. If training the PDVN with ARP online, the synthesizability value Network $R$ and the cost value network $Q$ in PDVN are incorporated into the estimator $\mathcal{E(s_t)}$ in the ARP framework by $\mathcal{E(s_t)}=R(s_{t-1},a^r_{t-1})*Q(s_{t-1},a^r_{t-1})+(1-R(s_{t-1},a^r_{t-1}))*c_{dead}$.
  • (4) In our current implementation, we adopt the surrogate model as a surrogate for (e)-(f), which however does not compromise the effectiveness of our active planner.

[1] Segler, M., Preuss, M. & Waller, M. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 604–610 (2018).

[2] John S. Schreck, Connor W. Coley, and Kyle J. M. Bishop Schreck. Learning Retrosynthetic Planning through Simulated Experience. ACS Central Science 2019

[3] Guoqing Liu, Di Xue, Shufang Xie, Yingce Xia, Austin Tripp, Krzysztof Maziarz, Marwin Segler, Tao Qin, Zongzhang Zhang, and Tie-Yan Liu. 2023. Retrosynthetic planning with dual value networks. In Proceedings of the 40th International Conference on Machine Learning (ICML'23)

[4] Schwaller P, Petraglia R, Zullo V, Nair VH, Haeuselmann RA, Pisoni R, et al. Predicting Retrosynthetic Pathways Using a Combined Linguistic Model and Hyper-Graph Exploration Strategy. ChemRxiv. Cambridge: Cambridge Open Engage; 2019.

Weakness W1: Somewhat disconnected from previous evaluation

• To begin, we seek to draw a comparison between the metrics employed in our framework and those utilized in prior studies.

  • Our metrics: (c1) success rate, (o2) query rate, and (o3) normalized route quality.
  • Metrics in prior studies: (c1) success rate, (p2) average route length, and (p3) number of single-step model calls.

• We have already substantiated the superior performance of our framework in terms of (c1) success rate compared to previous planning methods.

• Concerning (p2) average route length, its application in previous works aimed to quantify a specific aspect of route quality. However, it is worth noting that this metric is independent of real-world route quality, as illustrated in Figure 1 of the main text. This motivates our adoption of (o3) normalized route quality for a more faithful evaluation.

• we extend our assessment to (p3) the number of single-step model calls. The performances of various baselines and our framework are outlined below. The GRASP approach achieves the fewest iterations among all baselines. Our approach, GRASP with ARP, surpasses GRASP by 1.2 in the benchmark dataset and by 3.9 in the expert dataset. In conclusion, our framework enhances the planning efficiency of the implemented planners."

Model	Retro*-0	Retro*	Retro*-0+	Retro*+	EG-MCTS-0	EG-MCTS	GRASP	Retro* with ARP	GRASP with ARP
Benchmark test set	69.2	53.5	52.6	48.7	45.9	43.1	41.3	46.0	40.1
Expert test set	36.7	33.2	28.9	25.7	33.4	28.9	22.6	25.6	18.7

W2: The model to assign quality is not described in enough detail

• We had elucidated the training methodology for acquiring the surrogate model that assigns quality in the paragraph of Reaction quality annotation in Section 4.1.
  • "Initially, the method in Guo et al. (2020) is employed to pre-train a model utilizing the USPTO-MIT dataset, followed by the fine-tuning of the model in reactions derived from the high-quality, expert-annotated dataset"
  • We introduced this fine-tuning step on the expert dataset consisting of high yield reactions precisely to ensure that high predictive probabilities from our surrogate model align with high yields.
• We had included additional information on the implementation details in Appendix C and presented a thorough evaluation of the surrogate model itself in Appendix D.

W3: It is not convincingly defined what reaction quality is

• We had defined the labor-intensive reaction quality in the Introduction section.
  • Line 4 Section 1: "The ideal reaction qualities that meet real-world chemical practicability, e.g., yield of a reaction, can be either annotated experimentally in a laboratory or by experienced chemists."
• We had also detailed how to implement the reaction qualities so far for our framework in the paragraph of Reaction quality annotation in Section 4.1:
  • "Unfortunately, there is no established large-scale available reaction data set with respective reaction qualities or promising reaction performance, e.g. yield, prediction model(Jiang et al. (2022)), and both lab verification and expert annotations are expensive and time-consuming. We adopt a surrogate model to provide reaction quality annotations. ... During deployment, it is practical to replace the surrogate model with a chemist to provide online annotations, e.g. a coarse-grained quality rating from 0 to 10."