Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing

Thank you for your detailed comments. We address your concerns as follows:

A significance analysis is needed to strengthen the reliability of performance improvement.

Thank you for your suggestion. We agree that statistical significance analysis is essential for validating the reported improvements. In our main results, we presented only the mean statistical improvements without assessing their significance. To address this limitation, we conducted pairwise t-tests at the peptide level to evaluate metric improvements on the 9-species-v1 dataset. The t-values for the six base models are 82.4, 42.1, 38.6, 88.2, 43.2, and 50.7, respectively. For all models, the p-values are less than 1e-12. The results show that RankNovo's performance improvements over all base models are statistically significant:

• All comparisons yielded t-statistics of at least 38.6943 (based on total 90K samples)
• All p-values were below 1e-12, substantially lower than the conventional 0.05 threshold These findings conclusively demonstrate that RankNovo achieves statistically significant improvements in both peptide recall and amino acid precision, providing stronger support for the conclusions presented in Section 4.2 (Main Results).

key parameters such as λ (and potentially others) should be analyzed through dedicated experiments to assess their impact on model performance.

Thank you for your valuable suggestion. While we provided λ evaluations in Section 4.4 and Appendix D.3, we agree more analysis strengthens our work. We conducted additional experiments testing different λ values:

Our findings show:

• λ impacts performance, with neither extreme values being optimal, validating our choice of λ=0.5 which balances PMD and RMD contributions.
• Even with sub-optimal λ values, RankNovo maintains peptide recall ≥0.650, still outperforming both ByNovo (0.623) and ContraNovo (0.618), demonstrating the framework's inherent robustness. While computational constraints prevented exhaustive hyperparameter analysis, our experiments across five λ values provide strong evidence for the effectiveness of our selected configuration and the overall robustness of RankNovo's approach.

a discussion on computational complexity and runtime is essential to provide a more comprehensive understanding of its feasibility in real-world applications.

Thank you for this important question. Appendix E.5 has already discussed RankNovo's inference speeds. As expected, RankNovo is slower than single-model approaches—a natural trade-off for reranking frameworks. The main bottleneck isn't reranking itself but gathering peptide candidates from base models, which require sequential autoregression and beam search, while RankNovo's inference needs just one attention forward pass.

RankNovo offers a flexible speed-performance trade-off addressing real-world concerns. Users can: 1.Use fewer base models/candidates when prioritizing speed 2.Scale up when maximum accuracy is critical These findings are documented in Appendix D.2 Table 12. This adaptability is valuable in proteomics research and clinical settings where resource constraints and accuracy requirements vary significantly, enabling researchers to make informed decisions based on their specific constraints and objectives.

The zero-shot experiment lacks clear justification. How does it relate to the challenge of heterogeneous mass spectrometry data (Lines 80-82)? Is it testing generalization ability? Should it be framed as an OOD test?

Thank you for this important question about our zero-shot experimental setup. Our zero-shot experiments evaluate RankNovo's generalization to unseen base models, which indeed constitutes an OOD test as you correctly suggested. We train RankNovo using outputs from only a subset of base models, then test its ability to rerank predictions from all models, including previously unseen ones. This capability is crucial for practical applications. As new sequencing models emerge, users can apply our released checkpoint directly without retraining, ensuring RankNovo's long-term utility.

Regarding the heterogeneous MS data challenge, this is primarily addressed by our reranking approach itself. We deliberately employed six heterogeneous base models, each capable of performing well on different segments of the heterogeneous data (Fig. 1B). RankNovo functions as a meta-reranker that leverages collective knowledge while minimizing individual biases, directly addressing heterogeneous data distribution challenges, which is the advantage of ensemble learning proven in previous studies [1-3].

[1]End-to-end training of CNN ensembles for person re-identification [2]Don't take the easy way out: Ensemble based methods for avoiding known dataset biases [3]Exploring model learning heterogeneity for boosting ensemble robustness

Thank you for your detailed comments. We address your concerns as follows:

Is the choice of base models making benchmark comparison favorably for RankNovo? Should performance improvement be attributed to base models' capabilities or the reranking model?

Thank you for this important question about the contribution of base models versus our reranking approach.

Our benchmark comparison is methodologically sound because: 1) We included all available baselines in our evaluation, including the previous SOTA ContraNovo. 2) Our methodology follows established practices in ensemble research [1][2][3], where comparing ensemble methods against constituent base models is standard.

To address whether RankNovo's performance stems from stronger base models or the reranking approach itself, our ablation studies in Appendix D.2 and Table 9 are informative. When training and evaluating RankNovo using only five weaker models, our reranking approach still achieved 0.651 peptide recall - significantly outperforming the excluded strongest ByNovo model (0.623). This confirms that while high-performing base models contribute to results, the reranking methodology itself provides substantial independent value.

[1] Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models

[2] Llm-blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion

[3] End-to-end Training of CNN Ensembles for Person Re-identification

After Eq. 3, when forming the MSA embedding, it is not clear if the stacking is row-wise or column-wise.

Thank you for this important clarification question.

The stacking is performed row-wise. For a single peptide sequence embedding with dimensions (L, D)—where L represents the number of amino acids (after padding) and D represents the embedding dimension—the MSA embedding is constructed by row-wise stacking the embeddings of N peptide candidates to form a tensor of shape (N, L, D).

This MSA embedding construction follows established practices in the field, consistent with works such as MSA Transformer [1], AlphaFold2 [2], and AlphaFold3 [3].

[1] MSA Transformer

[2] Highly Accurate Protein Structure Prediction with AlphaFold

[3] Accurate Structure Prediction of Biomolecular Interactions with AlphaFold 3

The description of the backbone of RankNovo in Section 3.4 would be easier to understand if it were presented in the form of a diagram.

Thank you for this constructive suggestion. While we included a model backbone illustration in Figure 2(B), we acknowledge it doesn't fully capture all technical details of RankNovo's architecture.

To address this, we've created a more detailed diagram focusing on tensor shapes and attention mechanisms, available at https://anonymous.4open.science/r/RankNovo-F2FB/NewFig.png.

I think it is worth reiterating or clarifying that RankNovo is the first DL-based reranking framework since some of the base models included already feature deep learning architectures.

Thank you for this important point. You're right - some base models do incorporate deep learning architectures.

But To clarify: RankNovo is novel as the first deep learning-based reranking framework specifically for de novo peptide sequencing. While existing models (including the base models) directly decode peptides through regression, RankNovo introduces a fundamentally different approach by evaluating multiple peptide candidates and selecting the optimal prediction.

This reranking mechanism represents a methodological shift, allowing RankNovo to function as a universal enhancement module that integrates with various de novo sequencing models regardless of their architecture.

Just after Eq. 1, it is stated that “Intensity signals are projected to d dimension with a linear layer because of its relatively lower accuracy”. What does “its” refer to here – the linear layer or intensity signal?

Thank you for highlighting this ambiguity. "Its" refers to the intensity signal, not the linear layer.

Intensity signals in MS/MS spectra inherently have lower accuracy compared to m/z signals due to measurement errors, as documented in Chang C, et al. [1]. Given this limitation, we follow Casanovo and ContraNovo by embedding intensity signals using a simple linear transformation, which is less sensitive to tiny changes than the sinusoidal encoding used for m/z signals.

[1] Quantitative and In-Depth Survey of the Isotopic AbundanceDistribution Errors in Shotgun Proteomics

Q6: What does ri refer to in Eq. 5?

A6: In Equation 5: $g = E_{i \neq j} [P(r_i, r_j)] = \frac{1}{n(n-1)} \sum_{i=1}^{n} \sum_{j=1, j \neq i}^{n} |M(r_i) - M(r_j)|$ The term $r_i$ refers to the i-th residue (amino acid) in our vocabulary. This equation defines the gap penalty $g$ as the average mass difference between all possible pairs of non-identical amino acids.

Thank you for your detailed comments. We address your concerns as follows:

It's better to provide benchmark performance of RankNovo on NovoBench.

Thank you for your suggestion. Following your recommendation, we expanded our evaluation to include all three NovoBench data sources: Seven-Species (from DeepNovo), Nine-Species-V1, and HC-PT (from InstaNovo). We construct Seven-Species and HC-PT as instructed in NovoBench.

The results on NovoBench show that RankNovo is consistently superior on Nine-Species and HC-PT datasets. On the other hand, zero-shot performance of models trained on MassiveKB (ByNovo and RankNovo) don't perform normally on Seven-Species. Further analysis shows that the reason lies in the distribution misalignment between MassiveKB and Seven-Species. The former is collected by high-resolution MS equipment, while the latter is composed of low-resolution data, which makes the phenomenon explainable.

We greatly appreciate your valuable suggestion and will incorporate these additional benchmark results and the discussion about data composition into our camera-ready manuscript.

More baselines should be included for a more comprehensive comparison.

Thank you for this valuable suggestion. We have expanded our evaluation to include InstaNovo, AdaNovo, π-PrimeNovo, and π-HelixNovo. Results show RankNovo outperforms all these baselines across all metrics on Nine-Species-V1, a brief summary is provided below.

Additionally, as mentioned in our response to Q1, we've included comprehensive comparisons with InstaNovo, AdaNovo, and π-HelixNovo on NovoBench.

These expanded baselines and comparisons will be included in the camera-ready version.

It's recommended to discuss works combining database search and de novo sequencing, such as SearchNovo.

Thank you for this valuable recommendation. We have expanded Section 2.1 to include SearchNovo, a concurrent development that combines database search with de novo sequencing. SearchNovo employs a retrieval mechanism to identify similar peptide-spectrum matches and integrates prefix peptide sequence information through a fusion layer. This addition provides information on the broader research landscape. We will include this expanded discussion in the camera-ready version.

RankNovo requires the target peptide predicted by one of the base models and this may harm flexibility. The advantages of reranking methods compared to SearchNovo should be explained.

Thank you for this important question about flexibility limitations and advantages.

We view SearchNovo and RankNovo as complementary approaches with distinct advantages:

• Inference-time vs. Training-time Improvement While SearchNovo enhances itself through novel architectures and training techniques, RankNovo explores extracting additional value from existing model outputs at inference time. As models approach optimal performance on available training data, the collective information in prediction data represents an untapped resource that RankNovo specifically leverages.

• Training-free Scaling and Adaptation SearchNovo primarily uses information from the single most similar PSM, limiting its capabilities post-training. In contrast, RankNovo's axial-attention modules integrate features from multiple base models and demonstrate training-free performance scaling with more candidates (Section 4.3, Fig 3(A), Table 12). This allows practitioners to balance efficiency and performance as needed.

• Controlled Prediction Space with Enhanced Reliability While RankNovo requires at least one base model to correctly sequence the peptide, this constraint brings practical advantages. By limiting modifications to existing predictions rather than generating novel sequences, RankNovo minimizes the risk of introducing new errors, providing heightened reliability and interpretability for real-world applications. Despite this constrained prediction space, RankNovo achieves state-of-the-art performance across all benchmarks.

In conclusion, these approaches address different aspects of sequencing challenge and can be integrated - SearchNovo could serve as a base model within RankNovo's framework, potentially combining respective advantages.

Fix typos.

Sorry for the confusion, the typos will be fixed in the final script.

Thank you for your detailed comments. We address your concerns as follows:

The paper lacks validation on more challenging datasets with diverse post-translational modifications (PTMs).

We sincerely appreciate your insightful observation regarding the need for validating RankNovo on datasets with more diverse post-translational modifications. You've highlighted a crucial aspect of peptide de novo sequencing that directly impacts the practical utility of our approach in comprehensive proteomics studies.

In our current Nine-Species-V1 benchmark, we included three classes of PTMs: Oxidation-M, Deamidation-N, and Deamidation-Q. The promising results of these modifications demonstrated that RankNovo can effectively enhance performance on PTM-containing spectra when such modifications are incorporated during training.

To address your valuable suggestion, we conducted additional experiments on a more diverse set of biologically significant PTMs from the dataset compiled by Zolg et al. [1]. Specifically, we selected three functionally important modifications: Acetylation (K), Dimethylation (K), and Phosphorylation (Y). Each PTM included ~62.5K spectra split 8:1:1 for training/validation/testing.

Given that these new PTMs were not in the original vocabulary of our models, we performed necessary fine-tuning procedures. We combined the training and validation datasets across all three PTMs, reinitialized the embedding and final linear layers to accommodate the expanded vocabulary, and fine-tuned both the six base models and RankNovo accordingly.

Our results consistently demonstrated RankNovo's superiority:

• Acetylation (K): 5.6% improvement over the best base model (0.889 vs 0.833)
• Dimethylation (K): 2.8% improvement (0.487 vs 0.457)
• Phosphorylation (Y): 6.7% improvement (0.589 vs 0.522)

These compelling results further validate that our deep learning reranking framework maintains its potential across a more diverse spectrum of PTMs, underscoring the robustness and broader applicability of our approach for advanced proteomics research.

[1]ProteomeTools: Systematic characterization of 21 post-translational protein modifications by liquid chromatography tandem mass spectrometry (LC-MS/MS) using synthetic peptides

Testing on data from a broader range of mass spectrometry instruments would better demonstrate cross-platform generalization.

Thank you for highlighting this limitation. While our original benchmarks (Nine-species-v1/v2) used only Q Exactive instruments, our training dataset (MassiveKB) includes PSMs from various instruments, potentially enabling cross-platform generalization.

To address your concern, we note that the PTM datasets mentioned in our response to Q1 were collected using Orbitrap Fusion Lumos instruments, different from our benchmark's Q Exactive instruments. RankNovo consistently outperformed all base models on these Orbitrap Fusion Lumos datasets across all three PTMs (Acetylation-K, Dimethylation-K, and Phosphorylation (Y)).

These results provide strong evidence of RankNovo's effectiveness beyond Q Exactive instruments, demonstrating its cross-platform generalization capabilities. We appreciate this suggestion which helped strengthen our evaluation.

The manuscript would benefit from addressing additional research directions in the literature review to better contextualize scientific contributions.

Thank you for this valuable suggestion. We decided to incorporate the recommended research directions into our final manuscript as follows:

• Introduction (Section 1):

Added ensemble approaches for multiple search engines (PepExplorer, iProphet)

• Related Work - De Novo Peptide Sequencing (Section 2.1):

Incorporated PTM identification works (Open-pFind, pNovo+)

• Related Work - Candidate Reranking (Section 2.2):

Expanded with list-wise reranking algorithms (ListNet, ListMLE)

• Related Work - Axial Attention (Section 2.3):

Enhanced with efficient attention mechanisms (Linformer, Performer)

These additions better contextualize our contributions within existing literature and help readers understand how our work bridges deep learning techniques and proteomics applications. We appreciate your feedback which has improved the manuscript's depth and quality.