MADGEN: Mass-Spec attends to De Novo Molecular generation

Question 1: The authors have not compared with previous de novo molecular elucidation methods from MS such as MSNovelist [1], Spec2Mol [2], MIST [3], etc.

Response: Thank you for the suggestion! We have included MSNovelist and Spec2Mol in our experiments. We report Top-1/Top-10 accuracy, similarity, and MCES in Table 2 in the updated draft. Note that for MSNovelist, only the accuracy is available.

We did not include MIST in our comparisons, as it predicts molecular fingerprints rather than performing de novo molecular structure generation, which is not the focus of our method, of MSNovelist, and of Spec2Mol.

Question 2: manuscript’s presentation requires significant improvement for publication readiness

Response: Thank you for pointing out these issues. We have fixed them in the paper and optimized the presentation of the paper.

• Line 242 - Line 243: Why the word 'Following' is underlined

This is a rendering problem in the OpenReview when using \citet. We found that Following is underlined when using google scholar pdf viewer but not google chrome viewer.

• Why there is T in Eq. (6)

The T means the endpoint of the bridge process - we have defined $\mathcal{E}_T:=\mathcal{E}$ in line 239, and $e_T$ is an element in $\mathcal{E}_T$/

• \citep is used for parenthetical citations while \citet is used for textual citations, where the citation is part of the sentence. The authors are suggested to use these two commands appropriately to ensure clarity in their references and maintain consistency in citation formatting.

We have double-checked the \citep and \citet in our draft and fixed all incorrect usage the issues.

• Line 102: The “F” in “Generative Frameworks for molecular generation” should be lowercase.

Fixed

Question 3: The code is not provided for reproducing the results.

Response: Thanks for the comment. We'd provided the anonymous repository in here: https://anonymous.4open.science/r/abc-482F. Note that our code is built upon the repository https://github.com/igashov/RetroBridge.

[1]. MSNovelist: de novo structure generation from mass spectra (Nature Methods)

[2]. An end-to-end deep learning framework for translating mass spectra to de-novo molecules (Communications Chemistry)

[3]. Annotating metabolite mass spectra with domain-inspired chemical formula transformers (Nature Machine Intelligence)

Thank you for the response and additional experiments. My concerns have been addressed, and I have adjusted my score to 6 to support this work. I believe that de novo molecule generation from mass spectra is a highly important problem in practice, yet it has not received sufficient attention from the AI research community. The authors’ idea of using retrieved molecular scaffolds to guide molecule structure elucidation is also interesting.

Question 1: In Section 3.2.1, the authors state, “Since the atom set V can be directly inferred from the chemical formula.” However, it is unclear where the chemical formula is derived from. Could the authors clarify this, as in real-world scenarios, the input typically does not contain the chemical formula?

Response: Chemical formulas are derived based on the MS1 peak, which specifies the molecular weight of the ionized molecule that becomes fragmented and measured as a spectrum. From the weight of the ionized molecule, one could generate candidate formulas. Additional information such as the spectrum can be used to refine the list of candidate formulas.

For example, in the recent MIST-CF[1], a dynamic programming algorithm generates exhaustive chemical formula candidates for the MS1 peak within a small mass tolerance, often filtering implausible options using chemical rules like ring double bond equivalents (RDBE). Subsequently, peaks within the spectrum are annotated with subformulas using a Formula Transformer, a neural network, based on the candidate formulas. MIST-CF scores each candidate formula based on its alignment with the observed fragmentation spectrum, outputting a ranked list of likely formulas. SIRIUS[2] is another de noo tool that assigns a chemical formula to a spectrum. Like, MIST-CF, SIRIUS generates candidate formulas, and assigns potential subformulas to peaks. These subformula annotations are then organized into a fragmentation tree using maximum a posteriori (MAP) optimization. Finally, SIRIUS calculates the likelihood of each chemical formula based on the constructed tree. Another technique, BUDDY[3], each molecular weight associated with each peak within the spectrum is searched against a curated molecular formula database. Similarly, the neutral loss (what was lost during fragmentation and not measured) is also searched in the formula database. BUDDY prioritizes explainable candidate formulas and filters implausible formulas. Currently, SIRIUS and BUDDY are the two most common tools used by practitioners to annotate their spectra.

Question 2: For the MassSpecGym dataset, what is the “best published state-of-the-art performance” referred to? Are there baseline performance metrics reported for the other two datasets as well? & Weakness 4: The discussion of baselines is unclear.

Response: The manuscript describing the MassSpecGym dataset was just accepted at NeurIPs 2024 [4]. For de novo generation with a known chemical formula, performance is reported for “random chemical generation”, “SMILES transformer”, and a “SELFIES Transformer”. The accuracy was zero for all three techniques. We also report on running additional tools, namely Spec2Mol[5] and MSNovelist[6], as summarized in Table 2.

Performance metrics include Top-1 accuracy, similarity, and MCES (Maximum Common Edge Subgraph). The results indicate that Top-1 accuracy was zero across all datasets, but similarity and MCES scores varied. For example, Spec2Mol achieved higher similarity and MCES values on MassSpecGym compared to NIST23 and Canopus, with similarity scores of 0.19 (MassSpecGym), 0.16 (NIST23), and 0.18 (Canopus). Corresponding MCES values were 45.89 (MassSpecGym), 20.88 (NIST23), and 38.97 (Canopus).

Weakness 3: The scaffold retrieval performance, especially when using a predictive retriever, remains relatively low (e.g., NIST23).

Response: We have double-checked the implementation for NIST23 and noticed there is a numerical issue when computing the cosine similarity score. We fix the bug and re-do the experiment for NIST23. Correctness of other datasets are also double-checked, below is the updated result for NIST23:

We hope the updated result address your concern.

Reference

[1] Goldman, Samuel, et al. "MIST-CF: Chemical formula inference from tandem mass spectra." Journal of Chemical Information and Modeling 64.7 (2023): 2421-2431.

[2] Dührkop, Kai, et al. "SIRIUS 4: a rapid tool for turning tandem mass spectra into metabolite structure information." Nature methods 16.4 (2019): 299-302.

[3] Xing, Shipei, et al. "BUDDY: molecular formula discovery via bottom-up MS/MS interrogation." Nature Methods 20.6 (2023): 881-890.

[4] Bushuiev, Roman, et al. "MassSpecGym: A benchmark for the discovery and identification of molecules." arXiv preprint arXiv:2410.23326 (2024).

[5] Litsa, Eleni, et al. "Spec2Mol: An end-to-end deep learning framework for translating MS/MS Spectra to de-novo molecules." (2021).

[6] Stravs, Michael A., et al. "MSNovelist: de novo structure generation from mass spectra." Nature Methods 19.7 (2022): 865-870.

Thanks for the suggestion, we have added baselines (Spec2Mol [1], MSNovelist [2], Random Chemical Generator [3], SMILES Transformer [3], SELFIES Transformer [3]) in the paper. For your convenience, we have attached the table below, which can also be found in the updated draft.

We hope this will address your concern.

References

[1] Litsa, Eleni, et al. "Spec2Mol: An end-to-end deep learning framework for translating MS/MS Spectra to de-novo molecules." (2021).

[2] Stravs, Michael A., et al. "MSNovelist: de novo structure generation from mass spectra." Nature Methods 19.7 (2022): 865-870.

[3] Bushuiev, Roman, et al. "MassSpecGym: A benchmark for the discovery and identification of molecules." arXiv preprint arXiv:2410.23326 (2024).

We thank the reviewer for the insightful questions, below we addressed the raised concerns and comments.

Question 1: How is the candidate scaffold pool determined for predictive retrieval? There can be a huge number of scaffold candidates.

Response: For NIST23 and CANOPUS, we selected all candidates from PubChem by providing the chemical formula. MassSpecGym provided 256 candidates per test molecule. We are not aiming to find the correct candidate, but the correct scaffold, which could reduce the number of candidates to retrieval. We will update the draft to provide more background knowledge about the candidate pool selection.

Question 2: How does the contrast learning work for aligning the embeddings of mass spectra with their corresponding molecule? Some explanations are needed.

Response: We consider a contrastive learning framework similar to CLIP [1], which aligns the embeddings from two modalities. Here we treat spectrum as one modality and scaffold as the other. The CLIP-based framework has been popularized for information retrieval framework [2-7], where one can use the embedding similarity to determine what’s the most likely paired item (scaffold) based on the query (spectrum). We will clarify the background and previous paradigm in our updated draft.

Question 3: Regarding molecular generation in Phase 2, the absorbing transition matrix only applies to isolated atoms, strictly enforcing scaffold structure. Will such a hard constraint be problematic especially when the chosen scaffold is incorrect?

Response: Thank you for the insightful question! Indeed, a typical absorbing diffusion transition does not support modifying what's generated. However, compared to uniform transition matrix, absorbing diffusion significantly reduces the modeling complexity. It is commonly shown by previous research [8] that absorbing diffusion outperforms uniform diffusion. Moreover, the mentioned "hard constraint" is amendable by further introducing solver such as predictor-corrector during sampling [9-11]. As we do not innovate the fundamental framework of diffusion models, we do not include such augmentation in our experiment or method.

Question 4: Oracle retrieval requires both the MS/MS spectrum and the chemical formula (molecular graphs). If a chemical formula is known, what are the remaining challenges in solving the molecular structures?

Response: A chemical formula (not molecular graph) can have multiple corresponding molecular structures due to many possible molecular arrangements. For example, there are 44,374 known molecular structures (and hence graphs) in the PubChem database associated with C12H18N2O2. There are also likely many more structures undocumented in any databases. So the challenge is realizing the exact molecular structure that gave rise to the MS/MS spectrum.

Weakness 5: The SPA of the predictive retrieval is very low.

Response: After submitting the draft, we notices that there is an unexpected issue in the implementation of stage 1 for NIST23 dataset. We fix the implementation to have a better SPA result for this dataset. We also check the correctness of other implementations. Specifically, the new result yields a SPA of 57.8% for NIST23, 37.9% for CANOPUS, 34.8% for MSGym. We kindly refer the reviewer to the revised draft for the more details.

Weakness 6: The predictive retrieval approach yields poor molecule generation in Phase 2, where the generated structures fail to align with target properties, underscoring a critical limitation.

Response: We have introduced various baseline result in our updated draft. Specifically, our retrieval accuracy outperforms all baselines that are reproducible. We'd like again emphasize the challenging of MS/MS spectrum annotation task. To the best of our knowledge, our method is currently the SOTA in this venue.

Weakness 7: The conditioning of molecular generation on mass spectrometry data is largely based on classifier-free guidance, a well-established technique. The novelty is not well articulated.

Response: Thanks for the comment. While we employ CFG in our methodology for guided generation, we do not claim such a technique as our major contribution. Specifically, our major contribution is to propose a two-stage generative retrieval framework for mass spectra annotation(area). Under the proposed framework, we explore various implementations of CFG that give better performance.The implementation, which involves the spectrum embedding module and spectrum-molecule-interaction module, are novel to the community.

We sincerely appreciate you taking the time to reconsider our work and increasing your score. Your effort and support in evaluating our submission mean a great deal to us.

Best,

The Authors