An All-Atom Generative Model for Designing Protein Complexes

Thank you for your thorough review and insightful comments that have helped us enhance the clarity and quality of our manuscript. We have carefully addressed your concerns below.

Q1: Claims And Evidence on Complex Design

A1: Thank you for your questions. We address your concerns as follows:

• A1.1 & A1.2: We indeed condition on the target protein in tasks such as antibody design and peptide design. Additionally, the unconditional multimer generation in a "chain-by-chain" manner (in appendix), is also part of protein complex design.
• A1.3: Thank you for your suggestion. We agree that explicitly defining notation for multi-chain proteins will enhance the clarity. In the revision, we will modify the notation system to include multi-chain notations as follows: For a protein $\mathcal{P}$ consisting of $C$ chains, the $c$-th chain has length $N^{(c)}$, and each modality is represented as the combination of individual chains.
• A1.4: RMSD is sensitive to local structural variations and can be affected by symmetry considerations during alignment, leading to high RMSD despite overall structural similarity and high TMscores. The image (https://anonymous.4open.science/r/Rebuttal-1784/multimer.pdf) illustrates this scenario, indicating that while RMSD is high, the backbone alignment is nearly perfect. We calculated RMSD for each chain in the multimer folding task, as shown in the table. The results indicate that APM achieves accurate predictions for each chain, outperforming Boltz_woMSA.

The aim of our work is to model protein complexes. To achieve this, we integrate single-chain data to learn general protein generation and multi-chain data for complex modeling. This approach makes sure that APM is able to handle both single chain and complex tasks.

Q2: Methods And Evaluation Criteria

Thank you for your comments. We address your concerns as follows:

• A2.1: Thank you for this suggestion. We will improve the clarity by reorganizing the figures in the main text.
• A2.2: We would like to clarify the effectiveness of Sidechain Module. During the first stage of pretraining, we trained the Sidechain Module independently, allowing it to learn torsion angle information. In the second stage, we maintained a 50% probability of continuing to train the Sidechain Module, ensuring the network's parametrization captures sidechain information. The effectiveness of Refine Module is demonstrated in Table 4, which serves as an ablation study. The results show that APM with the Refine Module significantly outperforms the version using only the Backbone Module in terms of both complex generation quality and binding affinity.
• A2.3: Thank you for your observation. We actually selected high-quality samples rather than dropping them, and we will correct this in the revision. We have checked and found only a small number of duplicate samples, which have been removed in training.
• A2.4: We used AlphaFold2's crop function, which ensures spatial continuity, preventing situations where residues are far apart in space.

Q3: Experimental Designs Or Analyses

A3.1: Thanks for your suggestion! We use two longest targets from [2]: 3di3 (binder length 193) and 6m0j (binder length 194). For comparison, we use RFDiffusion to design binders by first generating structures and then designing sequences using ProteinMPNN.

The table demonstrates that APM outperforms RFDiffusion in terms of dG, with a higher percentage of dG < 0. Additionally, APM shows better foldability, as indicated by higher ipTM, and successfully designs binders for both targets.

A3.2: We use 80 as the threshold. APM generates the most number of sequences with pLDDT scores above 80. We regret that we are unable to report on PAE at this time, as Boltz does not provide PAE metrics by default.

A3.3: We calculate dG without relax. The results demonstrate that APM maintains better performance compared to others, and successfully generating samples with negative dG.

A3.4: Thank you for pointing this out. We will revise our statement accordingly. Due to policy restrictions, we are currently unable to use AF3, which is why we opted for Boltz.

Q4: Other issues:

A7: Thanks for your careful review. We will correct mentioned issues.

We appreciate your helpful feedback. We have responded to your concerns below and look forward to any additional comments.

Q1: Essential References Not Discussed

A1: Thank you for highlighting the need to include additional related works. We will enhance our Related Work section by incorporating references on sidechain prediction, such as DiffPack and AttnPacker. For motif-scaffolding, we will discuss both structure-based and sequence-based methods, including RFDiffusion, Frameflow, EvoDiff, DPLM, and ESM3. Regarding protein structure refinement, we hypothesize this refers to methods like Rosetta relax and OpenMM minimization, as well as other works like DeepAccNet (NC2021, https://www.nature.com/articles/s41467-021-21511-x). We welcome your feedback to ensure comprehensive coverage of these important areas in our Related Work section.

Q2: Other Weaknesses

A2.1: Thank you for your question. The results presented in Table 4 is indeed an ablation study. We provide two types of dG scores: one with sidechain-only relax and another with both backbone and sidechain relax. The lower dG scores from sidechain-only relax indicate high-quality backbone generation without structural conflicts. The small RMSD scores between generated and relaxed backbone demonstrate that the initial structures are already near optimal conformations, requiring minimal adjustment to reach energy minima. Notably, APM with the Refine Module significantly outperforms APM using only the Backbone Module on these metrics, validating the effectiveness of the Refine Module in complex design.

A2.2 & A2.3: Thank you for your valuable feedback. We acknowledge that our current system combines widely accepted architectures to model protein sequences and structures. Exploring more efficient frameworks is indeed an important direction for the field, and we will discuss potential approaches to this in our response A4.2.

Q3: Other Comments Or Suggestions

A3.1: This is a great question! We believe that APM has the potential to generalize to other protein-related tasks in the future, such as protein docking or directly utilizing the pre-trained encoder for affinity prediction tasks. However, this remains an open question, and we leave this for future exploration.

A3.2: Thank you for pointing this out! We acknowledge that APM currently does not achieve SOTA performance across all metrics in the task of unconditional single-chain generation. To address this, we have adjusted the temperature strategy for sequence sampling, resulting in significant improvements over previous methods for this task. Please refer to the table below for updated results.

A3.3: Thank you for this question. APM's current implementation and experiments focus on CDR-H3 design in SFT and zero-shot manner, leveraging the conserved nature of antibody framework regions.

Q4: Questions For Authors

A4.1: Thank you for this question. To assess the effectiveness of the consistency loss, we conducted an experiment by removing the consistency loss and retraining the model to the same number of steps. When evaluated on unconditional single-chain generation tasks, the version with consistency loss showed improvements in both scTM and scRMSD compared to the ablated version, as detailed in the table below.

A4.2: This is a great question, and we fully agree that it represents a crucial challenge in scaling of protein models. A recent work Proteina (https://openreview.net/forum?id=TVQLu34bdw) may serve as an important exploration of model and data scaling. Proteina employs a scalable, non-equivariant AF3-like diffusion transformer, focusing on alpha carbon coordinates without frames, which allows for scalability to many parameters and long protein generation. These considerations are crucial for advancing the field and addressing the challenges of scaling protein generative models. In the future, we plan to explore Proteina-like scalable framework to further enhance the scalability and efficiency of our protein generative models.

A4.3: We achieve conditional generation by employing masking to ensure that the loss is not computed for the condition region. Additionally, we set the time step t=1 to ensure that the ground truth condition is provided.

Thank you for your valuable comments. We have addressed your concerns below and welcome any further feedback.

Q1: Relation To Broader Scientific Literature and Essential References Not Discussed.

A1: We appreciate the reviewer for pointing out these references. In our revision:

1. We will include FoldFlow1 (Bose et al.) in the Related Work section to clarify the concurrent development of SE(3) flow matching approaches to FrameFlow.
2. Although FoldFlow2 (Huguet et al.) showcases the integration of protein language models, the folding code for FoldFlow2 is currently unavailable. We will incorporate a discussion in the Related Work section: APM differs from FoldFlow2 in several aspects: our model is capable of generating all-atom structures and sequences, supports multi-chain generation, and is designed for complex design tasks.

Q2: Questions For Authors: Did you try a binder design task on the RFdiffusion benchmark? I would be very curious to know how it performs.

A2: We thank the reviewer for this valuable suggestion. As the Reviewer ZXig also mentioned the design of longer binder. Here, we design binders to the two targets showcased in RFDiffusion: SARS-CoV spike protein RBD (PDB id 6m0j) and IL-7RA (PDB id 3di3). For comparison, we first generate structures using RFDiffusion, then design sequences with ProteinMPNN. As APM currently does not support hot-spot residue, we only evaluate functionality and foldability.

The table demonstrates that APM in zero-shot mode significantly outperforms RFDiffusion in terms of dG, with a higher percentage of designs achieving dG < 0. Additionally, APM shows better foldability, as indicated by higher ipTM scores, and successfully designs binders for both targets. This indicates that APM is able to design high-quality binders.

We appreciate your thorough review and helpful comments. We have carefully addressed your concerns below and welcome any additional feedback.

Q1: Claims And Evidence

A1: Thank you for highlighting the need for statistical validation. We conducted folding with APM and ESM3 using 20 seeds. For RMSD, ESM3 shows a marginally better mean (4.708±0.094 vs 4.828±0.077) with statistical significance (p < 0.05). For TM-score, APM achieves better performance (0.856±0.002 vs 0.828±0.002) with statistical significance (p < 0.05). We appreciate your reminder and will revise our statement to reflect that APM demonstrates "competitive" or "comparable" performance to ESM3.

Q2: Weaknesses on Perplexity

A2: Thank you for your valuable suggestion. To evaluate sequence quality, we compute perplexity using ProGen2-base across various inverse folding methods, including average, median, and std of perplexity. The results indicate that the sequences generated by APM achieve comparable perplexity to ground truth sequences.

Q3: Experimental Designs Or Analyses and Weaknesses

A3.1: Thanks for your suggestion. We agree that a larger test set could enhance evaluation. Here, we focus on maximizing training data for design tasks, using only "samples missing cluster IDs" as the test set. This allowed us to retain more training data while ensuring reliable performance evaluation.

A3.2: Thanks for this suggestion. Due to policy restrictions, we are unable to access AF3's weights, which is why we opted for Boltz. Regarding Chroma, we added the results on unconditional multimer generation. The results demonstrate that APM can generate reasonable multi-chain structures with high binding affinity, significantly outperforming Chroma.

A3.3: Thank you for your question. The results presented in Table 4 is indeed an ablation study. We provide two types of dG scores: one with sidechain-only relax and another with both backbone and sidechain relax. The lower dG scores from sidechain-only relax indicate high-quality backbone generation without structural conflicts. The lower RMSD scores between generated and relaxed backbone demonstrate that the initial structures are already near optimal conformations, requiring minimal adjustment to reach energy minima. Notably, APM with the Refine Module significantly outperforms APM using only the Backbone Module on these metrics, validating the effectiveness of the Refine Module in complex design.

A3.4: Thank you for this suggestion. While we acknowledge the importance of wet lab experimental validation, we used pyRosetta's energy function as it is a widely accepted metric for evaluating binding affinity.

A3.5: For the folding task, we have already reported the std in A1. For antibody design and peptide design tasks, the metrics are based on multiple sampling. By adjusting the temperature strategy for sequence decoding in the unconditional single-chain task, we achieve improved results and additionally provide the std for RMSD and TMscore to demonstrate the stability of results(see table below).

A3.6: Thanks for your suggestion. APM currently supports 20 standard amino acids. This impacts our ability to evaluate certain enzyme-substrate complexes, as substrates can be diverse molecules beyond proteins. While enzymes are typically proteins, substrates can vary widely, including carbohydrates and lipids.

Within the current scope of the submission, we focus on protein-protein interaction tasks. We are working on extending our framework to support a broader range of biomolecules.

Q4: Essential References Not Discussed

A4: Thank you for your kind reminder. We have mentioned AF3 in the Related Work (line 77). We will revise manuscript to make it clearer. Regarding Chroma, we will include it in the revision.