We appreciate your valuable feedback. Below are our detailed responses. We hope they address your concerns and welcome your favorable reconsideration.

W1 & L: All-atom generation

• Thank you for your question. Regarding all-atom generation, we recognize this limitation (Appendix C.1) and consider it a future research direction.
• We would like to clarify that models like AF3 can directly determine the atoms contained in the side chain because the amino acid type is explicitly known at input time. However, in our model's input, the amino acid types of the CDRs part are unknown, so it is non-trivial to directly design an all-atom result when design sequence and structure when co-designing the structure and sequence.
• Furthermore, many works, including AbX [1], also generate only backbone atoms and use Rosetta for side-chain packing and relaxation.

W2 & Q3: Novelty

• Thanks for the critical question regarding the novelty of our work. This allows us to clarify our core contribution, which lies not in an innovation in a single technical component, but in proposing and validating a new and, we believe, more efficient paradigm for protein design.
• We would like to clarify that our purpose for using AF3-like models is not merely to leverage their ability to generate protein backbones. Our core motivation is to "repurpose" large-scale folding models, which have been pre-trained on vast amounts of general protein data, as "Foundation Models" to solve the problem of data scarcity in specific downstream tasks, such as antibody design.
• This idea is analogous to the paradigm in the field of Large Language Models (LLMs), where foundation models like GPT are fine-tuned for various downstream tasks. We posit that AF3-like models are, in effect, the foundation models for the "AI for Science" domain, and our work serves as a successful application of this "pre-train and fine-tune" paradigm in the field of antibody design. This is fundamentally different in principle from previous methods which train models from scratch on limited antibody data. We believe that this approach of "repurposing" protein folding foundation models holds great potential for the broader field of protein design in the future.

Q1: Discrete vs. Continuous Diffusion Trade-off

• Thank you very much for the question and attention to this interesting finding in our work. We completely agree that this is a valuable observation and are happy to share our understanding and speculation on it.
• First, we would like to clarify a premise: the "discrete" and "continuous" methods we compare are applied only to the generation of the sequence. For structure generation, we consistently use the native, continuous diffusion-based EDM method from the AF3-like model. Based on this, our explanation for the trade-off phenomenon you observed is as follows:
  • Why is discrete diffusion superior for sequence (AAR)? We believe this primarily stems from data modality matching. Amino acid sequences are fundamentally discrete data (a choice from 20 categories). Discrete diffusion models are specifically designed to handle such data, allowing them to more accurately capture the underlying distribution and thus achieve better results in sequence recovery accuracy (AAR).
  • Why does using continuous diffusion (for sequence) lead to superior structure (RMSD)? We speculate this is due to training schedule alignment, not a direct benefit to the structure.
    • When we use continuous diffusion for the sequence, we can perfectly reuse the training schedule from the base model's (Boltz-1) pre-training, where the noise level is sampled from a continuous distribution. This maximally preserves the original model's structure prediction capability, hence resulting in better RMSD performance.
    • When we use discrete diffusion for the sequence, we must "adapt" the structure module's continuous training schedule to discrete time steps for synchronization. This means the sampling of the noise level σ degenerates from a continuous space to a discrete set. This "non-native" training approach has a slight mismatch with the base model's pre-training, which we speculate leads to a minor degradation in structure prediction capability, manifesting as a slightly higher RMSD.
• Based on the analysis above, an ideal "hybrid method" is not a simple combination of the two models, but rather one that resolves the training schedule mismatch while maintaining the advantage of discrete diffusion for sequence modeling.
• A promising direction is to adopt a discrete diffusion model that can operate in continuous time/schedule. This would allow sequence and structure generation to be perfectly synchronized within a unified, continuous training framework, with each component operating in its optimal modality. We acknowledge that this is an advanced and challenging research direction with few available references at present. We have included the exploration of such methods as an important part of our future work and will state this in the paper. Thank you again for your insightful question.

Q2: Impact of MSA Filtering

• Thank you for your insightful suggestion. We have conducted the corresponding test according to your feedback, and the results have led us to a more nuanced understanding, which we will elaborate on below.
• Our methodology employs a 0.2 similarity threshold to filter the MSA. We would like to humbly clarify our rationale for this protocol. Our primary goal was to establish an academically rigorous and conservative benchmark from the outset. This strict filtering serves two purposes: 1) to preemptively prevent any potential for data leakage to interfere with our results, and 2) to ensure a consistent data distribution is used for both training and testing, which is fundamental for a reliable evaluation. In fact, we believe that even without filtering, the impact and risk of data leakage are minimal for the following reasons:
  • Our statistical analysis shows that sequences with high similarity to the ground truth are extremely rare. For instance, sequences with >0.6 similarity constitute only about 7% of the raw MSA, and those with >0.8 similarity are a mere 2%.
  • The MSAModule in our architecture internally subsamples the vast MSA for each computation. This mechanism further diminishes the already low probability of the model actually utilizing these few highly similar sequences.
  • This concern is primarily relevant to the controlled setting of academic benchmarking to ensure fair comparison. In practical, real-world antibody design applications, leveraging all available homologous information is standard practice.
• Therefore, we hypothesized that data leakage would not have a significant impact and the model would have difficulty learning from such a minuscule amount of "leaked" information. Consequently, we predicted that using unfiltered sequences as input would likely not lead to a substantial improvement in results.
• We must be transparent that this is a theoretical conclusion based on the model's mechanics, as experimentally verifying it across various thresholds would be prohibitively expensive due to high training costs. After establishing our perspective, we performed the test as you suggested. The AAR(%) results are as follows:

• As shown, the overall performance is indeed far worse than reported in our paper. We believe the primary and most plausible reason for this decline is distribution shift. To illustrate, the total unfiltered MSA is, on average, nearly twice as large as the one filtered with our strict 0.2 threshold. This dramatic increase in sequence quantity does not merely add more data; it also alters the information density of the input. It fundamentally alters the statistical properties of the input distribution our model was trained on, consequently impacting the distribution of key latent variables within the model.
• However, it is crucial to note that the overall AAR is still above our 0.2 threshold, and the performance on the key CDR-H3 region remains at a respectable 43.65%. This strongly suggests that our model learns from complex patterns and multiple data (e.g., antigen's information), rather than simply "cheating" via leakage.
• This drop demonstrates that violating distribution consistency hurts performance. This is precisely where the effectiveness and importance of our filtering method lie: it is the mechanism that establishes and guarantees this crucial consistency. While the 0.2 threshold may not be theoretically optimal—as a comprehensive search is too costly—our results in our paper empirically prove that it creates a sufficiently effective distribution, while also preventing potential leakage at source.
• Tackling this sensitivity to distribution shift is a valuable direction for future work. We plan to explore potential solutions, such as training the model with filtered MSAs using random threshold to improve its robustness. This requires substantial experimentation. Due to time constraints, we are committed to pursuing this in our future research.
• On the other hand, we do not view this distribution shift as entirely negative. The fact that respectable performance is maintained on key metrics suggests that this shifted distribution contains rich, relevant information, which could potentially be harnessed to generate more diverse protein designs, although this too would require further experimental validation.

Reference:

• [1] Zhu, Ren, et al. “Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary, Physical and Geometric Constraints.” ICML (2024).

We hope this response could answer your questions and address your concerns, looking forward to receive your further feedback soon.

Thank you for dedicating your time and providing valuable feedback. In the following, we have carefully crafted detailed responses to address your comments.

W4 & Q19: Statistical Confidence

• Thank you for your suggestion. To ensure the credibility of our results, we have added the standard error of the mean (SEM) to our main results as you suggested. Here are the updated results for Tables 1-3:

• Typical antibody results:

• Nanobody results:

• The standard errors (SEM) are small across all metrics, and our performance gains significantly exceed the SEM, confirming the statistical significance of our improvements.

W5 & 16: Another benchmark

• We thank you very much for your valuable feedback regarding benchmark diversity. We completely agree that validation on multiple benchmarks greatly enhances the reliability and generalizability of research findings.

• Our initial decision to create our own data split was driven by a strict consideration for experimental fairness. As you understood, since our method is based on a model (Boltz-1) with a specific pre-training data cut-off date, data from many public benchmarks (like RAbD) have already been "seen" by it. A direct test would lead to an unfair evaluation due to data leakage.

• Nevertheless, we recognize the limitations of a single benchmark. Therefore, in response to your suggestion, we have supplemented our work with a full experiment on the RAbD benchmark. For fairness, all baselines were retrained on our training set, ensuring a comparable level of potential data leakage for all models.

• The new experimental results on the RAbD benchmark are as follows:

• As shown in the table above, our model remains comprehensively and significantly superior to all baseline methods. This added result further demonstrates the robustness and superiority of our method. We will include this experiment and its discussion in the final version of the paper. Thank you again for your suggestion, which has made our work more complete.

W6: Another baseline

• Thank you for your recognition. As we pointed out in the paper, among the mentioned papers:

  • AbDiffuser has no open-source code.
  • IgGM and AbX only released inference code and model weights, without training code.
  • RFAntibody adopts a two-stage method: first generating the structure using RFDiffusion, and then generating the sequence using ProteinMPNN. This is fundamentally different from our co-design settings.

• Therefore, we did not include them in the main results in the main text. However, to provide results that are as rich as possible, we still provided a comparison of the results with IgGM and RFAntibody in Tables S2, S3, and S4 in appendix, which you can refer to. Our model still has an advantage in the results.

W7: Filter out complexes that exceed 2k residues

• We thank you for your meticulous review and valuable suggestion. Regarding the severity of this limitation, we believe its impact is minor. In our final test set, only a single antibody-antigen complex (PDB ID: 8sak) was filtered out due to this limit. We consider its impact negligible for the following key reasons:

  • Data Rarity: Antibody-antigen complexes larger than 2,000 residues are very rare in the entire PDB database. In practical applications, one could also significantly reduce the input length by using only the antigen's epitope region.

  • Hardware Limitation: The limit stems from GPU memory constraints, not an intrinsic flaw in our algorithm. On hardware with larger GPU memory, our model could process these very long complexes without any modification.

  • Existing Solutions: This limitation is inherited from our base model Boltz-1. It is worth noting that subsequent updates to Boltz have effectively mitigated this issue by introducing techniques like chunking. Migrating our method to these updated versions would likely support longer sequences even on existing hardware.

• We agree and will move this disclosure to the main text in the final version for better visibility.

W15 & L: Limitation of in-silico experiments and novel/non-standard residues

• Thank you for your suggestion. Due to cost issues, we are indeed unable to conduct wet-lab experiments to verify the actual results and can only measure the results through computer simulation experiments and related metrics. We want to state frankly that this limitation is common in the field.

• Regarding the limitation of AF3 on novel and non-standard residues, we sincerely accept it. Our current discrete diffusion only supports the 20 standard types of amino acids. However, to our knowledge, Boltz has added support for Modified Residues in subsequent updates, thus being able to predict the structure of non-standard residues. Therefore, this limitation can be solved by migrating our method to the latest Boltz model and solving the sequence generation problem for non-standard residues. We will list it in the "Future work" section of our paper.

W17: Performance of general protein–protein-interface data.

• Thank you for your suggestion. As our work primarily focuses on the sequence-structure co-design of antibodies, it cannot support the design of general proteins. We would like to clarify that our method can indeed maintain a certain folding accuracy. To test the structure prediction capability of MF-Design on general proteins, we took a total of 128 pure protein complexes from the Boltz-1 test set for testing and compared them with Boltz-1 itself. We chose MFDesign-C because it has better structure prediction ability. The following are the comparison results:

• It can be found that on TM-Score, RMSD, and DockQ, our model still maintains folding accuracy comparable to Boltz-1.

W20: Estimate of computational effort

• Thank you for your suggestion, we will add this result in the final version of paper. Below is an estimation of our running time, which is the average time to generate a single prediction by running 10 different sample inputs in single NVIDIA H100 GPU. The suffixes -1 and -5 represent predicting 1 result or 5 results per sample, respectively.

• It can be found that the difference in running time between our model and Boltz-1 is almost negligible. Furthermore, when predicting multiple results per sample, it supports parallel prediction, which greatly speeds up the average running time for a single prediction result.

• Compared to other baseline methods for antibody codesign, our method does not have an advantage in computational efficiency due to the larger number of model parameters and a more complex architecture. However, the advantages in parameter count and architecture also bring us better results, which we believe is a reasonable performance-efficiency tradeoff.

Possibly relevant missing literature references

• Thank you for your additions. We will add these two related articles to our citations.

We hope this response could answer your questions and address your concerns, looking forward to receive your further feedback soon.

We sincerely thank you for taking the time to review and thoughtfully engage with our rebuttal. We are pleased that the additional analyses address your concerns and further strengthen the work. We greatly appreciate your positive evaluation and your revised recommendation to accept. As noted, we will incorporate the above clarifications into the final version of the paper. Your constructive feedback and support are truly valuable and mean a great deal to us.

I thank the authors for their very detailed and helpful rebuttal and the added experiments.

In my opinion, this rebuttal strengthens the paper (added uncertainty estimates (SEMs), a second benchmark on RAbD with retrained baselines, computational cost via H100 timings, and clearer discussion of the >2k-residue limit). Assuming the above clarifications are incorporated into the final paper version, I change my initial "Borderline Accept" to an "Accept". The work is practically relevant, technically sound, and with the changes from the rebuttal also empirically stronger.

Thank you for acknowledging our work and the valuable feedbacks. In the following, we have carefully crafted detailed responses to address your comments.

W & Q1: Heavy methodology writing and clearer contribution

• We sincerely appreciate your suggestion. We apologize for the issues with the writing. We promise to thoroughly revise the main text in future version, move non-essential parts to the appendix, and highlight our key contributions in the main paper.

• We would like to humbly clarify that our contribution lies in transforming a well pre-trained AF3-like model (leveraging its learned general protein interaction knowledge) into an antibody sequence-structure co-design model, which effectively addresses the scarcity of antibody-antigen complex structure data while achieving state-of-the-art performance with original folding accuracy maintained, resulting a multi-purpose method. In addition, such paradigm may offer insights for other protein-related fields where similar specific data limitations exist beyond antibodies.

Q2: Impact of fine-tuning strategies

• Thank you for this insightful question. Our training strategy is guided by the principle of Curriculum Learning, which is a well-established and effective methodology for training large-scale models. We were inspired by the multi-stage training paradigm of models like AlphaFold3, where the core idea is to present data from easy to hard, allowing the model to first learn local, simpler features before progressing to global, more complex structures.

• Unlike a general-purpose model, we have tailored this curriculum specifically for the task of antibody co-design. Since our primary goal is to design the CDRs that bind to an antigen, we have designed the training to expand its focus outwards from a CDR-centric perspective. Specifically:

  • Stages 1-3: The cropping strategy we have designed for the first three stages always includes the complete CDR regions of the antibody. We first let the model learn the antibody structure in isolation (Stage 1), and then gradually introduce the context of the antigen epitope and its surrounding areas (Stages 2 & 3), while progressively increasing the maximum number of input tokens. This design forces the model to continuously focus on the antibody-antigen interface, which is most critical to our task.

  • Stage 4 & Robustness: We fully anticipated the risk you astutely pointed out: an excessive focus on the CDR-proximal regions might degrade the model's ability to predict the structure of antigen parts that are distant from the binding interface. To mitigate this, we have specifically designed a fourth training stage. In this stage, we have implemented a mixed-strategy cropper: there is a 50% probability of using the aforementioned CDR-centric cropping, and a 50% probability of performing completely random sub-structure sampling across the entire antibody-antigen complex. This ensures that while the model masters local design, it also revisits and maintains its pre-trained capability for global complex structure prediction.

• The necessity of this strategy is empirically validated. As shown in our ablation study in Appendix G.2 (Table S1), without the fourth training stage, the model's prediction performance on the antigen structure worsens, as measured by both RMSD and TM-score. Therefore, our carefully designed four-stage strategy is crucial for the model's robustness.

• Finally, given the substantial computational cost of fine-tuning models of this scale, an exhaustive search of all possible strategies was not feasible. However, we believe our proposed and empirically-validated curriculum learning strategy strikes an effective balance between task-specific adaptation and the preservation of the model's general capabilities.

Q3: Ablation study with other discrete token diffusion method.

• Thanks for the valuable suggestion. First, we will explain why we chose D3PM-absorb as the version for sequence diffusion.

  • D3PM [2], as a typical method for discrete data diffusion, has several different types, including uniform, absorb, gauss, etc. D3PM-absorb is the most suitable one for amino acid sequence design. Specifically, D3PM-absorb can be seen as a Masked Language Model, suitable for handling the generation of text sequence modality. Since amino acid sequences can be regarded as text modality information, they are applicable to D3PM-absorb.

  • In addition, in the original D3PM paper, Table 1 also compared the performance of different types on text generation, and the effect of absorbing was significantly better than other types of discrete diffusion. This proves the effectiveness of D3PM-absorb in terms of performance.

  • Furthermore, many previous protein sequence design works have adopted D3PM-absorb. For example, LaMBO-2 [1] from NeurIPS 2023 used D3PM-absorb for antibody sequence generation. Our specific implementation also referenced the implementation of LaMBO-2.

• Next, we follow your valuable suggestions and choose another discrete token diffusion model D3PM-uniform to conduct further ablation studies. We have supplemented the results on the test set after retraining with D3PM-uniform for sequence diffusion, as shown below:

• Typical antibody results:

• Nanobody results:

• The results show that D3PM-uniform is significantly weaker than D3PM-absorb in terms of AAR. On the other hand, its RMSD results do not show a clear advantage over continuous diffusion methods. This ablation study demonstrates the effectiveness of our chosen method.

Reference:

• [1] Gruver, Nate, et al. "Protein design with guided discrete diffusion." NeurIPS (2023).

• [2] Jacob Austin, Daniel D Johnson, et al. "Structured denoising diffusion models in discrete state-spaces." NeurIPS (2021).

We hope this response could answer your questions and address your concerns, looking forward to receive your further feedback soon.

Thank you for acknowledging our work. Your rating has provided us with great encouragement, and your detailed feedback and suggestions have been immensely helpful. Below is our detailed response.

W1: Why call it AF3-like

• We sincerely appreciate the reviewer for this valuable feedback on our terminology. We fully agree that Boltz-1 is an excellent model with its own distinguishing features, and our decision to use the term "AF3-like" is based on two primary considerations:

  • As mentioned in our introduction, a series of models including Chai-1, Boltz-1, and Protenix have recently emerged. They share the core diffusion-based architecture pioneered by AlphaFold3. We use "AF3-like" to refer to this class of models that share a common technical foundation.
  • Our core contribution — the integration of sequence-structure co-diffusion — is an architectural modification that applies to this general framework, not to any feature specific to Boltz-1. We chose Boltz-1 as it is an excellent open-source implementation with available training scripts. Therefore, the term "AF3-like" more accurately reflects the general applicability and potential impact of our work, rather than confining it to a single model.

W2 & Q3: Clearer details in main article

• We sincerely appreciate the reviewer for this invaluable suggestion. Due to space limitations, we had to place some key details in the appendix. Following NeurIPS' standard practice, accepted papers are permitted one additional page - we would use this to incorporate the highlighted content. If accepted, we will add a clear introduction to more details, improving accessibility. Thank you again for your constructive feedback, which will significantly improve the quality of our paper.

Q1: All-atom generation

• Thank you for this insightful question, which touches upon a core consideration of our methodology. We would like to clarify this with the following points:

  • We have acknowledged this limitation in Appendix B.5 and consider it as one of our future research directions.
  • Regarding the premise that sequence can be derived from predicted side chains, this inverts the actual operational mechanism of models like AF3. These models are conditioned on a known amino acid sequence to predict an all-atom structure, including the side chains. When the sequence is provided, the model knows which side-chain atoms to generate. However, in our co-design task, the CDR sequences are initially unknown and are represented by the special <X> token. In this context, the model cannot determine which side chains to build for these unknown residues, making direct end-to-end all-atom design a non-trivial problem.
  • Our backbone-first approach is an established paradigm. Many existing works, including the recent AbX[1] (ICML 2024), also generate backbone atom coordinates first, followed by side-chain packing and relaxation using tools like PyRosetta.

• We hope this addresses your concerns. In summary, while Boltz-1 is capable of all-atom prediction, our specific task setting justifies the rationality of our backbone-first strategy.

Q2: Pretrain a diffusion model using all PDB and finetune on antibody dataset

• Your insight is exactly right – this is precisely what we do in our work! In fact, the paradigm used in this paper exactly follows a pretraining-finetuning approach as you proposed and we have already used a pretrained diffusion model. Boltz-1 is actually a pretrained AF3-like diffusion model, trained on the entire PDB, a large dataset containing all available complexes, and thus possesses rich pre-trained information. Our work involves fine-tuning these types of models on a subsequent antibody dataset. It is precisely because pre-training on the entire PDB dataset is very costly that "repurposing" already pre-trained models is much more convenient and cheap.

Q4: Include an AF3-like baseline and necessary of sequence codesign modification

• Thank you for your question. We would like to clarify that sequence co-design is the main work of this paper, which transforms a structure prediction model into a generative model, rather than just fine-tuning a structure model on antibody data. Therefore, the sequence codesign modification is necessary.

• On the other hand, AF3 itself, as a structure prediction model, does not have the capability to generate sequences in co-design. So we can only compare it on the structure prediction task. In fact, in Table 4, we have already listed the comparison between our fine-tuned model and an AF3-like structure prediction model like Boltz-1 itself, where the model with "Fine-tuned?" as '✗' is Boltz-1. It can be found that our model achieves better results on CDR-masked input and maintains comparable accuracy on CDR-unmasked input.

• Moreover, to demonstrate that our fine-tuned model still maintains good performance on general protein structure prediction, we conducted an additional evaluation. For this test, we selected a set of 128 protein-only complexes from the original Boltz-1 test set. We present the comparison of the relevant performance metrics below. This comparison shows that our fine-tuned model's capability is well-preserved.

Reference:

• [1] Zhu, Ren, et al. "Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary, Physical and Geometric Constraints." ICML (2024).

We hope this response could answer your questions and address your concerns, looking forward to receive your further feedback soon.

I guess my Q2 was a little different, AF3 is a sturcture prediction module trained on the PDB that happens to have a conditional (sequence conditioned) diffusion model trained as part of the process (an addition to AF2, which probably improved a lot of diversity related issues in comparisons), I understand you repurpose the conditional diffusion module within AF3-like models and adapt it to do codesign, and that this model has been trained on the PDB, however my question was more, why not use a pretrained diffusion model thats trained to generate designs (not a structure prediction approach with codesign adaptation) and then finetune this on antibodies?

Dear reviewers,

Please don't forget to give necessary feedback to the rebuttal, and also don't foreget to make acknowledgement if you haven't done.

AC.