MS-BART: Unified Modeling of Mass Spectra and Molecules for Structure Elucidation

Dear Reviewer n2SS,

Thank you for your insightful comments, suggestions, and time spent reviewing our work. We are glad that you find our work overall solid and technically sound, offering a unified and scalable framework that extends current techniques. Regarding your concerns, we will address them in detail one by one as follows:

Major Concern 1: Potential data leakage on MassSpecGym

Thank you for pointing this out. For the two benchmarks (MassSpecGym and NPLIB1), we used the official MIST weights pretrained on CANOPUS, downloaded from the official repository (mist_fp_canopus_pretrain.ckpt). To verify potential data leakage, we downloaded the CANOPUS training set and computed the structural overlap with the MassSpecGym test fold, finding an 18% exact match (exact matches / total test cases). We agree with your assessment that the originally reported MassSpecGym results are invalid.

We appreciate your suggestion and the provided checkpoints. However, we find that the provided checkpoints may not align with the official weights, as the model sizes differ (103.4 MB vs. 60.8 MB). We then trained our own MIST model under three settings:

1. CANOPUS_filter. We filtered the CANOPUS data (train/val/test) and the augmentation data (simulated spectra from a forward model) provided by the MIST repository by removing structures with high similarity (Tanimoto similarity > 0.75, as suggested below) to the test and validation sets in MassSpecGym. A total of 703 structures were excluded from the CANOPUS data and 559 structures from the augmentation data. The model was trained on the CANOPUS training data for 0.69 hours on an RTX 4090 GPU, achieving the best checkpoint with a test loss of 0.298 at epoch 138.
2. CANOPUS_filter_Contrastive. Using the same CANOPUS training data as in CANOPUS_filter and the cleaned augmentation data, we trained the model for 2.81 hours on an RTX 4090 GPU, obtaining the best checkpoint with a test loss of 1.328 at epoch 46.
3. MassSpecGym. We trained MIST on MassSpecGym training fold using one RTX 4090 GPU. After 1.8 hours of training, we obtained the best checkpoint with a test loss of 0.487 at epoch 22. The test loss was significantly higher than that of CANOPUS_filter, and since there are many hyperparameters in MIST training, this may indicate insufficient optimization and a suboptimal fingerprint model.

After obtaining the uncontaminated MIST model, we followed MS-BART's three-stage training pipeline and evaluated its performance on MassSpecGym. The results are presented as follows:

It is obvious that MS-BART suffers a performance degradation after data cleaning. However, the performance of our model is still acceptable, achieving sub-optimal or even SOTA (Top1 MCES) in similarity metrics while being significantly faster than DiffMS. The average inference time for a spectrum on one RTX4090 is around 3s for MS-BART, which is 53x faster than DiffMS's approximately 160s. It should also be noted that the performance degradation is attributed to the MIST model's bad performance. The fact that MS-BART (CANOPUS_filter_Contrastive) slightly outperforms MS-BART (CANOPUS_filter) demonstrates this, and the poor performance of MS-BART (MassSpecGym) further supports this observation. Finally, if the fingerprints are correctly predicted (given the gold fingerprint), MS-BART shows excellent performance, demonstrating great potential for structure elucidation with the rapid evolution of fingerprint prediction (such as DreaMS, as mentioned in Minor comments 1)

Moreover, MS-BART represents the pioneering work in applying the NLP paradigm to structure elucidation. It can be readily scaled up with additional pretraining data and larger model parameters (such as at the Qwen/LLaMa level) to further boost performance.

Major Concern 2: Potential data leakage on NPLIB1

We agree with you and thank you for your suggestion. We checked the structural similarity between the NPLIB1 test set and the pretraining dataset. The exact structure match is 0.85% (exact matches / total test cases, 7 cases). We also calculated the maximum Tanimoto similarity (with the NPLIB1 test set) distribution of the 4M pretraining data as follows:

There is indeed some possible data leakage, but most (>98%) of the pretraining data are not similar (Tanimoto Similarity < 0.7) to the test set. We also appreciate your suggestion and pretrained again on two clean pretraining sets. The first set filters out similar structures with a Tanimoto Similarity > 0.5 and is denoted as 4M_leq5. The second set filters out similar structures with a Tanimoto Similarity > 0.75 and is denoted as 4M_leq75. After fine-tuning and alignment, we obtained the new results as follows.

The results demonstrate that MS-BART still maintains competitive performance, outperforming baseline models in all similarity metrics at a strictly-separated pretraining dataset, although the exact match accuracy suffers performance degradation. However, we are wondering if this is really a data leakage issue, as this is inevitable with data and parameter scaling. We only pretrain on the structure with computed fingerprints, not the MIST-generated fingerprints. In contrast, we think this is a strong evidence showing the pretraining of MS-BART is effective, as MS-BART can learn meaningful molecule structures in pretraining and generalize to MIST-predicted fingerprints through finetuning and alignment. We hope to discuss this further with you during the discussion period.

Major Concern 3: Top-kaccuracy definition

We generate the predicted molecule in SELFIES. Since SELFIES representations are not unique, we first use the selfies library to transform the SELFIES into SMILES and then use RDKit to obtain the corresponding structure, followed by generating the canonical SMILES using Chem.MolToSmiles(mol, canonical=True). We appreciate your suggestion and have checked DiffMS, where they use Chem.MolToInchi(mol) for structure matching. Both methods correspond to a unique structure, ensuring a fair comparison. We have also validated this through experiments and found that the accuracy remains consistent.

---

Responses to minor comments are omitted due to the character limit. We will share them during the discussion period if requested and permitted.

---

Q1: Performance.

MS-BART still achieves the SOTA performance on NPLIB1 in terms of MCES and Tanimoto under corrected evaluation protocols, while on MassSpecGym, we also achieve SOTA on Top1 MCES metric. However, there is indeed a performance degradation, especially on MassSpecGym. We identify two main reasons for this:

1. MassSpecGym presents greater complexity, as each structure has multiple associated spectra (average > 5 per structure), which introduces additional noise for our model. Additionally, the use of a different pretraining dataset (DiffMS, which trains on self-collected data and removes exact-match molecules) may also lead to an unfair comparison.
2. The most important reason may be that the MIST model performs poorly on MassSpecGym. Training a robust MIST model on this specific dataset requires extensive hyperparameter tuning.

Q2: Low-quality fingerprints.

MS-BART exhibits a degree of robustness in handling low-quality predicted fingerprints, and the overall performance would get worse if the quality of the predicted fingerprints gets lower, as demonstrated in Major Concern 1.

Q3: Adding chemical formula.

Thank you for your suggestions. It should be noted that all recent diffusion-based methods use chemical formulas to determine the atom set and achieve promising results. Additional formulas definitely include important information and are very likely to boost the model's performance. We plan to explore effective ways to incorporate these informational hints into MS-BART in the future work, such as extending the vocabulary with numbers and specific atoms.

---

Finally, we sincerely appreciate your time and valuable suggestions, which have helped enhance and refine our work. We welcome further discussion during the rebuttal period.

Identifier: MassSpecGymID0228341
2Dinchikey (14): LSHVYAFMTMFKBA
Inchikey: LSHVYAFMTMFKBA-CTNGQTDRSA-N
SMILES: C1[C@H]([C@@H](OC2=CC(=CC(=C21)O)O)C3=CC(=C(C=C3)O)O)OC(=O)C4=CC(=C(C(=C4)O)O)O 
Canonical SMILES: O=C(O[C@@H]1Cc2c(O)cc(O)cc2O[C@H]1c1ccc(O)c(O)c1)c1cc(O)c(O)c(O)c1
canonical SELFIES: [O][=C][Branch2][Ring2][Ring2][O][C@@H1][C][C][=C][Branch1][C][O][C][=C][Branch1][C][O][C][=C][Ring1][Branch2][O][C@H1][Ring1][N][C][=C][C][=C][Branch1][C][O][C][Branch1][C][O][=C][Ring1][Branch2][C][=C][C][Branch1][C][O][=C][Branch1][C][O][C][Branch1][C][O][=C][Ring1][=Branch2]

Top1 Pred SELFIES: [O][=C][Branch2][Ring2][Ring2][O][C][C][C][=C][Branch1][C][O][C][=C][Branch1][C][O][C][=C][Ring1][Branch2][O][C][Ring1][N][C][=C][C][=C][Branch1][C][O][C][Branch1][C][O][=C][Ring1][Branch2][C][=C][C][Branch1][C][O][=C][Branch1][C][O][C][Branch1][C][O][=C][Ring1][=Branch2]
Top1 Pred Inchikey: LSHVYAFMTMFKBA-UHFFFAOYSA-N

Dear Reviewer LqRY,

Thank you for your insightful comments and time spent reviewing our paper. We are glad you found our paper well-written and generally of good quality. Regarding the novelty, as you mentioned, while every component is not a new method, incorporating them together and being the first work to apply the pretrain-finetune-alignment paradigm from NLP to mass spectrometry data modeling in an end-to-end way is indeed novel. We will respond to your questions one by one as follows.

Q1: Ablation of threshold epsilon choice.

We first choose the threshold epsilon=0.2 empirically, as we have checked the probability distribution and found most probabilities are smaller than 0.3. A big threshold may cause information loss, while a small threshold would introduce noise, so we choose epsilon=0.2 for a balance empirically and achieve excellent performance, surpassing the SOTA.

However, as Reviewer n2SS pointed out the potential data leakage, we replicate the experiment with clean data and choose epsilon by evaluating on the val set, selecting the best threshold based on the highest Tanimoto similarity.

We chose epsilon=0.11 for MassSpecGym. To evaluate the sensitivity of the hyperparameter choice to the final result, we also selected epsilon=0.15 (intermediate validation Tanimoto) and epsilon=0.20 (the worst validation Tanimoto) for comparison and presented the results as follows.

The results in parentheses indicate results that have not undergone re-rank (Line 215-218), and will also be reported in the later table. It is obvious that the re-rank by formula match and generative probability significantly boosts the final performance. As for the threshold sensitivity, there is no significant difference between the epsilons with the same rank post-processing.

Q2: Could the author provide some error analysis by visualizing how MS-BART in failure case for each of the three stages generated molecule structure compared with the ground truth?

Due to the NeurIPS rebuttal format restrictions, we cannot provide visualized error analysis in the rebuttal. We appreciate your suggestions and will add error analysis in the future version.

Q3: Could the author explain why in Figure 3 middle figure Top-1 accuracy decrease a bit (3.50 --> 3.46) from stage 2 to stage 3?

Thank you for pointing this out. A possible explanation is that the alignment is tuned with Tanimoto similarity rather than being based on an exact match. The rank loss aims to align the predicted structure's Tanimoto similarity with the gold structure to the model's generative probability, so it is difficult to control the accuracy. As we have replicated the experiments, we updated the figure data in the table below.

We can observe obvious benefits between each stage with or without re-ranking and demonstrate the effectiveness of MS-BART. It is more convincing to observe the top-1 similarity improvement, which can directly show the payoff of the contrastive alignment since we only generate three candidate structures for alignment in this stage.

Q4: Potential overfit or data leakage.

The results in Table 4 are the evaluation results after re-ranking (described in Line 215-218). Re-ranking can improve the evaluation stability. As for data leakage, we use the 4M pretraining dataset provided by the MassSpecGym official data, which has been refined by excluding any molecules with an MCES distance of less than two from any molecule in the test fold of MassSpecGym by the authors, ensuring there is no data leakage. Since we have replicated the experiments, we reevaluate the results in Table 4 with and without re-ranking as follows.

Upon re-evaluation, it can also be observed that our model is not sensitive to temperature changes, even for the top1 result without re-ranking. The main reason can be attributed to the beam-search multinomial sampling (refer to the Hugging Face text generation documentation), which combines beam search (beam width=100) and stochastic sampling, and the SELFIES vocabulary size of 185 (Line 139), resulting in this high confidence.

Q5: In addition to Q4, suppose the author is doing a clean train validation and test split, they are encouraged to show the structure similarity (e.g. edit distance) between the simulated pre-training dataset, MassSpecGym train, validation and test and NPLIB1 train, validation and test set.

In response to Q4, there is no data leakage between the pretraining dataset and the MassSpecGym test set, so further structure similarity analysis is not necessary. As for NPLIB1, there is some overlap (7 cases) between the pretraining dataset and the test set. Since MCES is time-consuming, we demonstrate the structure similarity using Tanimoto similarity. More details are provided in the response to Reviewer n2SS (Major concern 2).

Q6: Table 1 for baseline performance, have the author finetuned MIST on MassSpecGym? It is a bit concerning to see MIST+Neuraldeciper/MSNovelist has non-zero accuracy for NPLIB1 but zero accuracy on MassSpecGym.

The baseline results are reproduced from MassSpecGym and DiffMS as we state in the Table 1 caption. After a careful review of the DiffMS paper, MIST+NeuralDecoder/MSNovelist undergoes the same pretraining and finetuning, aligning with DiffMS.

Q7: It would be great if the author could analyze the model sensitivity of temperature before stage 3 Chemical Feedback to see how stage 3 influence model's confidence.

Thank you for your suggestion. We evaluated the MS-BART after stage 2 fine-tuning and showed the results as follows.

It is not surprising to find our model also remains stable before stage 3, mainly due to the decoding strategy in response to Q4. This is also strong evidence demonstrating the effectiveness of MS-BART, as we can observe the similarity metric improving between stage 2 and stage 3 (evaluation results shown in Q4) under different temperatures.

Q8: How sensitive is MS-BART to hyperparameter alpha in equation (5)?

Thank you for your suggestion. Due to the large size of MassSpecGym, we evaluated the sensitivity using NPLIB1. We conducted experiments on the clean pretraining data (4M_leq50, details are provided in our response to Reviewer n2SS, Major Concern 2). We tested alpha values in {1, 3, 5} and present the results below.

The final result we report is alpha=3, as it performs the best in the top1 Tanimoto on the validation set. Although there are some fluctuations between different alpha settings, it is clear that MS-BART is not very sensitive to the alpha selection.

---

Finally, thank you again for your insightful and constructive comments. The idea of active fingerprints as tokens and the unified modeling of mass spectra and molecules provides a new paradigm for structure elucidation, as most recent methods are based on diffusion. This paper provides strong insights on how to incorporate MS spectra into LLMs rather than using the directly noisy number sequence, opening a new strategy to align the spectra modality with language models. This would inspire many innovative ideas, such as reasoning models for explainable strcuture elucidation.

Dear Reviewer dSeN,

Thank you for recognizing our work and the time spent reviewing our paper. We are glad you find the use of active fingerprints as tokens interesting and intuitive, and that further alignment is a nice idea. Regarding your concerns, we will address them one by one as follows.

W1&Q2: The results sections lack comparisons to library matching methods discussed in the related works.

This paper aims to discover novel molecules from MS/MS, where library matching methods are not applicable. As we responded to Reviewers LqRY and n2SS, the pretraining and training sets are strictly separate from the test set, so it is obvious that library matching will achieve zero accuracy.

W2&Q3: The comparisons in Table 1, Table 2, and Figure 3 do not have error bars so it is tricky to determine if the ablations show statistically significant results.

As we respond to Reviewer LqRY, we adopt re-ranking (described in Lines 215–218) and beam-search multinomial sampling (refer to the Hugging Face text generation documentation), which makes MS-BART demonstrate high confidence in the generated structure. As shown in the table responding to Reviewer LqRY (Q4), MS-BART is not sensitive to temperature, and multiple experiment results are consistent. This high confidence ensures the performance gain mainly comes from different stage training rather than randomness.

W3: L283 typo "similar"

Thank you for pointing this out. We have fixed it.

Q1: How well does the model perform on the variant of the MassSpecGym task that provides chemical formulas as input? Is the additional chemical formula information complementary to what the model learns?

Thank you for your suggestions. It should be noted that all recent diffusion-based methods use chemical formulas to determine the atom set and achieve promising results. Additional formulas definitely include important information and are very likely to boost the model's performance. We plan to explore effective ways to incorporate these informational hints into MS-BART in the future work, such as extending the vocabulary with numbers and specific atoms.

Dear Reviewer gPth,

Thank you for recognizing our work and for your insightful comments. We are glad you find that MS-BART enables end-to-end learning by bringing two disparate modalities into a shared vocabulary and can learn robust spectral–structure correspondences before seeing scarce experimental data. Regarding your concerns, we will respond to them one by one as follows.

W1&Q1: Finetuning relies on MIST-predicted fingerprints; errors or biases in MIST could propagate through the model, but the paper does not quantify how fingerprint noise impacts downstream accuracy.

In response to Reviewer LqRY (Q1), we have verified that the threshold controlling the activation of the fingerprint is robust and the performance fluctuation is not significant. Regarding the impact of MIST performance on downstream accuracy, in the table responding to Reviewer n2SS (Major concern 1), we found that poor fingerprint prediction reduces the exact match accuracy but does not affect the similarity metric as much. These quantified results demonstrate that MS-BART is robust to fingerprint noise and is promising for real-world applications, as similar but not exactly matching structures are still useful for domain experts to narrow down the chemical space [1].

[1] DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra. ICML2025.

W2: The unified Transformer and beam search may be resource-intensive; the paper reports relative decoding latency trends but lacks concrete runtime or memory benchmarks for real-time or large-scale deployments.

Thank you for pointing this out. We have reported that the average decoding latency on one A800 with 80G memory in a computer cluster is around 1.45s. To facilitate large-scale and real-world deployment, we also tested the model on a consumer-grade RTX 4090 with 24G memory. The average inference time is around 3s, and the memory consumption is around 10GB when processing one spectrum with a beam width set to 100. Although MS-BART uses beam search, the vocabulary size (185) is small, and the total number of parameters is around 104M, so the inference latency and memory consumption is acceptable. At the same time, current LLM inference and serving packages, such as VLLM, also support encoder-decoder models, which would further improve the serving throughput.

W3: The ranking loss favors candidates with higher global Tanimoto scores, but that metric can be dominated by common substructures rather than spectral features. This may encourage the model to pick “average” molecules that share generic motifs with the ground truth rather than truly spectrally justified structures.

We appreciate this observation. While the Tanimoto scores can assess the similarity between predicted molecules and real molecules, we acknowledge that the potential limitation you mentioned may exist. However, there are no existing metrics more suitable for rewards, as the edit distance is time-consuming. Future work can focus on finding more reasonable rewards to reflect truly spectrally justified structures

W4: Since the MADGEN has uploaded updated benchmark results, incorporating those into the evaluation would help readers understand MS-BART’s gains relative to the other model.

Thank you for the reminder, we have updated the results.

Q2: Does MS-BART generalize to spectra acquired under different collision energies, adduct types, or instrument platforms?

Yes. MS-BART does not process the spectra directly but relies on MIST to predict the fingerprint. After a careful review of MIST [2], we found it highly dependent on proper chemical formula assignments from MS1 precursor masses, which are independent of collision energies, adduct types, and instrument platforms. It should also be noted that MassSpecGym presents this complexity, as each structure has multiple associated spectra (averaging more than 5 per structure) with different collision energies, adduct types, and instrument platforms. MS-BART can handle this complexity well and achieves sub-optimal or even SOTA (Top1 MCES) performance in similarity metrics.

[2] Annotating metabolite mass spectra with domain-inspired chemical formula transformers. Nat. Mach. Intell. 2023.

Dear Reviewer huv7,

Thank you for your thoughtful comments and the time you have taken to review our paper. We are pleased that you find the shared vocabulary conceptually elegant and the paper well-organized and readable. In response to your concerns, we will address each point below.

W1&Q1: More fine-grained ablation would help validate the design choices.

Thank you for your suggestion. Following your feedback, we conducted a more detailed ablation study on the NPIB1 dataset using clean pretraining data (4M_leq50, as detailed in our response to Reviewer n2SS, Major Concern 2). The results are as follows.

The table shows that pretraining with translation or hybrid denoising improves performance compared to no pretraining. While SELFIES denoising pretraining shows slightly lower performance, this may stem from insufficient pretraining (all experiments use the same hyperparameters as the final MS-BART). However, this does not contradict our proposed pretraining-finetuning-alignment paradigm, as pretraining still enhances cross-modal learning. In future work, we plan to explore why the denoising task negatively affects final performance and how to optimize it so that all pretraining tasks contribute positively.

W2: Similar method proposed in paper: "Conditional Molecular Generation Net Enables Automated Structure Elucidation Based on 13C NMR Spectra and Prior Knowledge"

Thank you for identifying the related paper. However, our approach has clear distinctions from the referenced work.

First, mass spectrometry is much more complicated than NMR because it suffers from great complexity and heterogeneity, making it hard to model. Moreover, annotated experimental mass spectra are scarce, and the predicted mass spectra vary significantly from the experimental ones, which are not available for direct large-scale pretraining.

Second, our model transfers these two disparate modalities into a shared Transformer vocabulary, enabling end-to-end learning. This is definitely novel, as the reference paper only uses number sequences to represent the NMR shift.

Third, the alignment stage is clearly different from fine-tuning because it further aligns the model to generate more accurate predictions using the same training set, whereas the reference paper uses more data, which is scarce in the mass spectrometry domain.

Finally, thank you for pointing this out. We will also discuss this paper in the related work section of our future version.

W3&Q3: Despite improved performance over prior models, the absolute accuracy remains low (e.g., top-1 accuracy < 4% on MassSpecGym), suggesting the method remains far from real-world application. This limits immediate practical applicability and calls for more discussion about failure cases, limitations, or downstream usage scenarios.

Thank you for your insightful feedback. Our work primarily aims to discover novel molecules from the dark chemical space, where the spectra are not in the annotated spectra library or even the corresponding molecules are not in the molecule library, making traditional library matching definitely invalid and resulting in zero accuracy performance. Although the exact accuracy is low, the retrain experiments on the clean dataset reveal that MS-BART can indeed discover truly novel molecules. Moreover, as discussed in paper [1], generating similar but not exactly matching structures is still useful for domain experts to narrow down the chemical space.

Previous experiments conducted on the contaminated data, along with the strong performance observed on the golden fingerprint benchmark (detailed in response to Reviewer n2SS, Major concern 1), suggest two clear strategies to improve overall results. First, future work can be devoted to improving the performance of fingerprint prediction, such as DreaMS [2]. Second, for the known molecules and targeted spectra annotation, it would significantly boost the performance if similar molecules or the true molecule are in the training data.

Finally, the unified vocabulary also provides strong insights on how to incorporate MS spectra into LLMs rather than using the directly noisy number sequence, opening a new strategy to align the spectra modality with language models. This would inspire many innovative ideas, such as reasoning models for explainable strcuture elucidation.

[1] DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra. ICML2025.

[2] Self-supervised learning of molecular representations from millions of tandem mass spectra using DreaMS. Nat. Biotechnol. 2025.

W4: Minor typographical errors (e.g., “hallucaination”, “simliair”, “slected”).

Thank you for pointing this out. We have fixed it.

Q2&Limitations: Impact of MIST Fingerprint Predictions

Thank you for your insightful comments. Since MS-BART uses MIST to process the MS spectra into fingerprints, how MIST performance affects downstream tasks is a common question. In our response to Reviewer LqRY (Q1), we have verified that the threshold controlling the activation of the fingerprint is robust and the performance fluctuation is not significant. Regarding the impact of MIST performance on downstream accuracy, in the table responding to Reviewer n2SS (Major Concern 1), we found that poor fingerprint prediction reduces the exact match accuracy but does not affect the similarity metric as much. These quantified results demonstrate that MS-BART is robust to fingerprint noise and is promising for real-world applications, as similar but not exactly matching structures are still useful for domain experts to narrow down the chemical space [1]. Future work can focus on improving MIST with more high-quality data, projecting MIST embeddings into LLM space for joint pre-training and training, or directly modeling formula peaks with LLM following MIST.

[1] DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra. ICML2025.