Domain-Agnostic Molecular Generation with Chemical Feedback

We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

1. More baselines

Our selection of baseline models included notable ones like LIMO, Chemformer, and RetMol from post-2022, chosen for their well-recognized effectiveness in this field. It's important to note that finding universally recognized and effective baseline models in this rapidly evolving field can be challenging, especially with some newer models lacking publicly available code or comprehensive documentation. According to your recommendation, we have now incorporated a new baseline model, RT[1], into our analysis. (Highlighted in Table 4 Page 7, and Appendix E Page 18)

2. More ablation studies

In response to your suggestions, we have added the results of these ablation studies to Appendix H.5 (Highlighted in Page 23).

• Effect of label smoothing

We investigate the impact of label smoothing on the diversity of molecules generated by the model, employing IntDiv as the metric. IntDiv assesses the chemical diversity of generated molecules by calculating the average Tanimoto coefficient within the generated set. As shown in the following table, the model with label smoothing does not overly rely on singular, frequently occurring patterns learned from the training data. Consequently, this enhances the diversity and creativity of the molecules generated.

• Effect of $\alpha$ parameter

We also explore the impact of the hyperparameter $\alpha$. As illustrated in the following tables, an $\alpha$ value of 0 indicates no self-feedback. When $\alpha$ is increased to 3, the model demonstrates a superior ability to optimize molecules compared to an $\alpha$ of 1. However, increasing $\alpha$ to 5 does not necessarily lead to better performance than at an $\alpha$ of 3. During actual operation, it is necessary to adjust parameters according to specific requirements. Based on experience, setting $\alpha$ to either 3 or 5 is recommended.

5. Related paper

Thank you for bringing this to our attention. While it is true that this study [6] also uses molecule's SELFIES representations for pretraining, the focus and methodology differ significantly from our work. In the referenced paper, the approach reinterprets regression as a sequence modeling task, where training combines property tokens with text tokens. Their model is engineered to generate molecules by inputting expected molecular property values along with a given molecular scaffold (with the generated molecules incorporating this scaffold), or to predict molecular property values based on an input molecule.

In contrast, our model is specifically designed to generate entirely new molecules based on existing ones, without being confined to a predetermined scaffold. This key difference in our objectives and techniques distinctly sets our research apart from the approach outlined in the cited study.

Following your suggestion, we have now included the referenced paper as a new baseline in Table 4 (Highlighted in Page 7) and a discussion of this study in Appendix E (Highlighted in Page 18) of our revised manuscripts.

6. Other revisions

We are grateful for your attention to these details and pointing out these oversights. In response to your valuable suggestions, we have made the following revisions:

• Task-specific tuning: We changed the phrase "heavy reliance on task-specific tuning" to "hard to train due to the high variance of RL". This alteration underscores our method's aim to alleviate the instability and difficulty associated with training reinforcement learning models, thereby offering a more stable and efficient process for downstream applications. (Highlighted in Section 1, Page 1)

• Focus on synthetic molecules: We included additional references to support our claim about previous studies. (Highlighted in Section 1, Page 2)

• Examples of molecular hallucination: To avoid any ambiguity, we have rephrased the description in the scenario where the model generates a molecule with an additional side chain. The revised text emphasizes that the perceived structural robustness is illusory and does not guarantee desirable properties, exemplifying "molecular hallucination". (Highlighted in Section 2.2, Page 4)

• Brittleness of SMILES: We added necessary references to support the statement about SMILES potentially leading to meaningless sequences. (Highlighted in Section 1, Page 2)

Thank you again for your constructive suggestions! Your insights have significantly contributed to enhancing the completeness and persuasiveness of our paper. We hope that the revisions and clarifications provided in our responses adequately address your concerns.

[1] Survey of Hallucination in Natural Language Generation. In ACM Computing Surveys 2023.
[2] A Survey of Hallucination in “Large” Foundation Models. 2023.
[3] Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models. 2023.
[4] Cognitive Mirage: A Review of Hallucinations in Large Language Models. 2023.
[5] Contrastive Post-training Large Language Models on Data Curriculum. 2023.
[6] Regression Transformer enables concurrent sequence regression and generation for molecular language modelling. In NMI 2023.

We sincerely thank you for your insightful feedback. We have addressed your concerns below and hope our responses provide clarity:

1. Concept of molecular hallucination

We apologize for any ambiguity in our expression, but we wish to clarify that our concept of "molecular hallucination" is indeed aligned with the NLP concept of hallucination. In NLP, hallucination refers to generating text or responses that, while grammatically correct, fluent, and natural, deviate from the provided source inputs (faithfulness) or lack factual accuracy (factualness) [1] [2] [3] [4].

Therefore, in our study, "molecular hallucination" is used to describe the phenomenon where molecules created by language models, although chemically valid and adhering to basic structural rules (similar to producing grammatically correct sentences), fail to possess the anticipated chemical properties (akin to producing text that deviates from the intended message or content of the source input). This concept is consistent with the understanding of hallucination in NLP, and as you may have noticed, other reviewers have also concurred with this viewpoint.

To enhance clarity, we have included related works about hallucination and further clarified the concept of "molecular hallucination" (Highlighted in Appendix D.3).

2. Concept of self-feedback

We chose the term "self-feedback" to underscore the model's self-driven learning and optimization process. In this process, the model guides its own learning and optimization based on the evaluation results of molecules it autonomously generates, though these evaluations are aided by an external scoring function. This term is specifically chosen to differentiate our work from studies that use responses from other models as their feedback signal sources [5]. In contrast, our feedback mechanism is based on the responses generated by our own model.

In response to your feedback, we have revised our wording to better convey this concept to readers, emphasizing the model's self-reflective learning cycle, despite incorporating external scoring (Highlighted in Section 2.2, Page 5).

3. Additional ablation study

We apologize for any confusion that may have arisen from our presentation and would like to clarify that the domain-agnostic molecular prefix-tuning in the second phase is distinct from parameter-efficient tuning methods. As mentioned in Page 3, the penultimate paragraph, this approach impacts the entire model's parameters, boosting its ability to understand and adapt across various molecular domains.

In response to your valuable suggestion, we have now incorporated an ablation study in our manuscript (Highlighted in Appendix H.5, Page 24). To investigate the impact of prefix tuning on the model, we present the mean (and standard deviation) of penalized logP improvement for molecules generated compared to inputs under varying similarity constraints, as detailed in the following table. This data demonstrates that prefix tuning enhances the model's performance in optimizing molecules.

Additionally, Figure 8 (Page 9) provides a comparison of the model's attention to key substructures, both with and without prefix tuning. For enhanced clarity, we have also included a tabular representation of the data visualized in this figure, offering a more direct interpretation of the impact of prefix tuning.

As indicated in the aforementioned tables, the implementation of domain-agnostic prefix tuning not only enables the model to more effectively adapt to downstream tasks but also improves its interpretability.

4. Modifications to the figures

We have revised these figures to better facilitate reader understanding.

• For Figure 7 (Highlighted in Page 8), the x-axis represents the QED values, and the y-axis shows their distribution.
• For Figure 8 (Highlighted in Page 9), the first subfigure has the x-axis representing the Substructure Attention Level (SAL) values, while the second subfigure's x-axis also represents SAL values and the y-axis shows their distribution.
• For Appendix Figure 2 (Highlighted in Page 21), the x-axis represents the property values and the y-axis shows their distribution.

Thanks for the authors' efforts on rebuttal. I appreciate them. The response has resolved my concern on the molecule hallucination problem, and I have raised my rating accordingly. I still have the following questions about the submission.

• Regarding the concept of self-feedback. As the authors agree, the objective function is to align the generation probability to an external model. In [1], the external model is a RLHF reward model; in this submission, the external model is the QED or plop score function. In concept, it is still external feedback but not self-feedback, because the training signal does not come from MolGen itself.

• Regarding the prefix tuning method. this approach impacts the entire model's parameters. Does it mean that the MolGen's other parameters (not the prefix) are also tuned in fine-tuning?

Reference:

[1] CONTRASTIVE POST-TRAINING LARGE LANGUAGE MODELS ON DATA CURRICULUM. 2023

Thank you very much for your insightful comments. Below are our responses:

• Limitations about self-feedback

Currently, the self-feedback mechanism in our study is tailored for optimizing a single target objective. As we mentioned in Appendix B, this approach exhibits reduced efficacy when applied to scenarios involving multiple objectives. This is primarily due to the generation of divergent preference rankings, which leads to uncertainty in the model's decision-making process about which optimization directions to prioritize. Consequently, an area for future research could be the development of self-feedback techniques that are more adept at handling and optimizing multiple objectives simultaneously.

Thank you again for your constructive suggestions! We hope our responses address your concerns.

We sincerely thank you for your insightful feedback. Below are our detailed responses to your concerns:

1. Effectiveness of SELFIES

We apologize for any confusion that may have arisen from our presentation.

• To further clarify, as detailed in Appendix F, we have conducted a comparative analysis using both SMILES and SELFIES formats. In Table 1, Figure 6, and Figure 8, Chemformer (S-PLM), which is pre-trained on SMILES and employs a BART architecture, serves as our baseline for comparison. This setup, with MolGen not utilizing the self-feedback mechanism, ensures an unbiased evaluation. The experimental results demonstrate that using SELFIES as the molecular language enables the generation of 100% valid molecules, while also facilitating a more focused attention mechanism in the model.
• The decision to use SELFIES is based on its standardized and more simplified syntax, which we find to be particularly well-suited for processing by language models. This choice is reflected in our results, where SELFIES ensured the generation of 100% valid molecules, aligning well with our study's objectives. We wish to underscore that SMILES remains a vital and widely adopted format for molecular representation, and our study does not intend to diminish its importance. Instead, our utilization of SELFIES represents a novel exploration, driven by the specific objectives of our research.

2. Synthetic molecule v.s. natural product generation

The division of the molecular dataset into "synthetic" and "natural product" domains is to effectively explore and understand molecules of varying complexities and origins. The "synthetic" domain encompasses artificially synthesized chemical molecules tailored for specific needs, e.g., drug development. On the other hand, the "natural product" domain covers molecules naturally occurring, which are pivotal in biological activities and often provide insights for drug development. Natural product molecules generally exhibit greater structural complexity and diversity (further data statistics are available in Appendix C), often resulting from the myriad of unique chemical structures produced through natural biological processes. This classification helps us better understand the unique challenges and features of each domain.

In our research, we follow the methodologies of prior works[1] for distribution learning, where the baselines focus on synthetic molecule generation. Building upon this foundation, we have extended our scope by including the generation of natural products as a new and more challenging task. This expansion not only enhances the complexity of the tasks we address but also broadens the applicability of our model to a wider range of molecular structures encountered in various scientific domains. Given that both synthetic and natural molecules represent a significant portion of molecules in daily life and research, we posit that our model’s proficiency in these domains indicates its capability to handle various molecular tasks effectively. In response to your suggestion, we have elaborated on this aspect in Appendix C (Page 16).

3. Effectiveness of prefix tuning

We performed an additional experiment to investigate the impact of prefix tuning (Highlighted in Appendix H.5, Page 24). Here, we present the mean (and standard deviation) of penalized logP improvement for molecules generated compared to inputs under varying similarity constraints, as detailed in the following table. This data demonstrates that prefix tuning enhances the model's performance in optimizing molecules.

Additionally, Figure 8 (Page 9) provides a comparison of the model's attention to key substructures, both with and without prefix tuning. For enhanced clarity, we have also included a tabular representation of the data visualized in this figure, offering a more direct interpretation of the impact of prefix tuning.

As indicated in the aforementioned tables, the implementation of domain-agnostic prefix tuning not only enables the model to more effectively adapt to downstream tasks but also improves its interpretability.