Multi-objective antibody design with constrained preference optimization

Weakness 3: While the manuscript goes in great theoretical detail, intuition is often lacking. E.g. Equation 3 is introduced but an intuition, “first term maximizes rewards, while second term keeps the model close to the reference model.”, which could facilitate understanding for the reader is missing.

This suggestion is great to enhance readability of our manuscript. We have provided intuitive explanations for the key formulas in the article. For example,

1. We provided intuitive explanations for concepts such as the primal-dual algorithm and the dual gradient estimation (Equation 4,5). This equation can be interpreted as appending a penalty term ${J}^{({C})}$ to the original objective ${J}^{({R})}$. The penalty term, which depends on how much the antibody generated by the current model violates constraints, can be adjusted dynamically through the Lagrange multipliers. (Line 256-258).
2. We also provided intuitive explanations for the gradient of Lagrange multipliers (Equation 11). In this equation, the gradient of dual can be calculated by the expected degree of constraint violation in the sampled antibodies under the current policy model. (Line 332-334)

Intuitive description for more formulas can be found in Line 244-246 and 300-302.

Weakness 4: Many things necessary for fully understanding the paper are moved to the appendix, resulting in decreased readability. Further, this also applies to some of the most interesting results, e.g. Table 9 and especially Figure 4.

Following your suggestions, we have moved some of the important results (Table 9) to the main text. However, due to the strict 10-page limit for the main text at the ICLR conference, we were unable to present more results in the main paper. Therefore, some results were included in the supplementary materials, with a clear link provided in the main text. We placed the important experimental results at the beginning of the Appendix.

Weakness 5: Some tables are hard to read, as their caption and corresponding text do not exactly describe what is in the table.

Following your suggestions, we have revised the captions and provided detailed explanations for better understanding. Specifically:

In Table 1, "reference" represents the native antibody structure and sequence in the testing set. (Line 420-421)

For Table 3, we have explicitly clarified the meaning of “ESM-2 based” and “Multi-objective.” Below is the updated caption:

Ablation studies for AbNovo on the RAbD dataset. The ablation experiment settings include: without using a language model (w.o. LM), replacing the structure-aware language model with ESM2 (ESM-2 based), using supervised fine-tuning instead of preference optimization (SFT), and incorporating all constraints into the optimization objectives (Multi-objective).

Similarly, we revised other unclear parts of the article to improve clarity (Line 153-156, 461-466 , 486-488, and 518-519).

Weakness 6: In the abstract and introduction, a focus is put on “alleviate overfitting issues due to the scarcity of antibody-antigen training data”, but no analysis supporting such a claim is included.

To alleviate the scarcity of antibody-antigen training data, we utilized a structure-aware language models pre-trained on massive structures beyond antibody proteins. We included two ablation experiments to demonstrate the relative contribution of the structure-aware language model. (Line 447-456)

First, we trained an ablation model where we exclude the embeddings of the language model as input features. We observed significant drops across nearly all metrics, indicating the importance of the language model.

Second, we also trained an ablation model where we replace this structure-aware language model with a sequence-only language model (ESM2). It shows that the structure-aware model yielded better results than the purely sequence-based model.

Weakness 7: Question about case studies

We have now utilized another more reasonable case where the DiffAb fulfill all constraints. We note that though the antibody generated by DiffAb fulfilled all constraints, AbNovo outperformed DiffAb in terms of Rosetta Binding Energy and Evolutionary Plausibility.

We also presented more details in the updated figure, such as the CDR H3 sequences annotated with biophysical properties.

Thank you for your valuable suggestions, which have significantly improved the quality of our manuscript. We have provided point-by-point responses to your comments below. We hope that our revisions and additional experiments address your concerns satisfactorily.

Weakness 1: I would suggest introducing a background section, as there are many things in this manuscript that would benefit from a proper introduction.

We added more background introduction on both preference optimization and the diffusion-based generative model. For preference optimization, we introduce concepts on the beginning of our method (Line 159-163). We introduce more details on Equation 1 (Line 202-204), the definition of CTMC (Line 186), and $T^{(0:1)}$ (Line 191). Additionally, we explicitly describe that the time $t$ in diffusion processes is a uniform distribution of [0, 1] (Line 191-192).

To further enhance the readability of our manuscript, we have provided a detailed notation table for all symbols used throughout the paper, which can be found in Appendix A.1.

Weakness 2: The evaluation metrics remain unclear even after reading the appendix A.3. This holds especially for “Evolutionary Plausibility”, “Stability”, “Self-association”, “Non-specific Binding”.

We have revised the this part to make these terms more clear (Line 1427-1497) and show how the different metrics are calculated and what they represent. We also briefly explain these terms below.

Evolutionary Plausibility:

Evolutionary Plausibility measures how likely a designed sequence is evolutionarily plausible in nature, reflecting adherence to general evolutionary rules of natural proteins. Recent studies show that large-scale protein language models, trained on millions of natural protein sequences, effectively capture these evolutionary rules [1, 2]. Specifically, we calculate this metric as the log-likelihood of the designed sequence under a pre-trained protein language model (Line 1457). Importantly, this approach has proven useful for guiding antibody maturation in wet-lab experiments [1], and it is thus widely used as an evaluation metric in recent generative models for antibody design [3, 4, 5].

Stability:

Stability measures the stability of the conformation of designed antibody in isolation, without the antigen structure involved. This metric differs from Binding Energy, which evaluates the interaction between the antibody and the antigen. Specifically, we calculate this metric using established protocols based on the Rosetta software, as employed in previous methods [6].

Self-association:

Self-Association refers to the tendency of antibody molecules to aggregate with each other. Self-association can negatively impact the efficacy of antibodies, so low self-association is desired in practical antibody development [7, 8]. Previous studies have shown that larger negatively charged patches area in the CDRs corresponds to a higher risk of self-association [7]. Specifically, we calculate this metric using established protocols from previous methods [7].

Non-specific Binding:

Non-specific Binding, or Binding Specificity, refers to the undesirable interaction of antibodies with cellular proteins other than the intended target, particularly membrane proteins of the cell. Previous studies [7] have observed a strong correlation between non-specific binding and the hydrophobic patches in CDRs. Therefore, we use the hydrophobic patch area in CDRs as a proxy for evaluating the risk of non-specific binding, utilizing their established pipeline to calculate this metric [7].

[1] Hie, et al. Efficient evolution of human antibodies from general protein language models. Nature Biotechnology 2024.

[2] Shuai, et al. IgLM: Infilling language modeling for antibody sequence design. Cell System 2023.

[3] Zhu, et al. Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary. ICML 2024.

[4] Kong, et al. End-to-End Full-Atom Antibody Design. ICML 2023.

[5] Zhou, et al. Antigen-Specific Antibody Design via Direct Energy-based Preference Optimization. NeurIPS 2024.

[6] Li, et al. Full-atom peptide design based on multi-modal flow matching. ICML 2024.

[7] Makowski, et al. Optimization of therapeutic antibodies for reduced self-association and non-specific binding via interpretable machine learning. Nature Biomedical Engineering 2023.

[8] Yadav, et al. The influence of charge distribution on self-association and viscosity behavior of monoclonal antibody solutions. Molecular pharmaceutics 2012.

Question 1: The announcement of "The first deep generative model for multi-objective antibody design" in summarized contributions, AbDPO also supports multi-objective optimization.

Thank you for your suggestion. We have removed the word "first" from that sentence (Line 71).

Question 2: Question about energy minimization.

In our initial manuscript, following the post-processing procedures for designed antibodies in previous method [1, 2], we conducted the energy minimization for both backbone and side-chain structures.

In response to your concern, we have added a new experiment where we relaxed only side-chain atoms while keeping all backbone atoms fixed. As demonstrated in the table below, AbNovo continues to outperform other methods in this setting (row “w.o. fixed backbone” and row “fixed backbone”).

We can also observe that the energy regarding to all methods increased significantly. This may suggest that the designed backbone atoms have steric clashes that cannot be resolved without optimizing the backbone conformation. To validate this, we recalculated the Rosetta energy by removing the atomic repulsion energy term related to steric clashes. The energy scores drop substantially, suggesting that steric clashes in the fixed backbone contribute to the high energies observed (the third row named fixed backbone w.o. rep).

Regarding your question about whether these experiments demonstrate that AbNovo simply provides a better initial structure for Rosetta relaxation, we also computed the backbone structural deviations between pre- and post-relaxation in the setting where both backbone and side chain are relaxed. As demonstrated in the table below, since the backbone structures change very little during relaxation (less than 0.1 Å), this indicates that AbNovo inherently generates high-quality backbone conformations that do not heavily rely on relaxation for improvement.

[1] Luo, et al. Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures. NeurIPS 2022.

[2] Zhu, et al. Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary. ICML 2024.

Question 3: Does the optimization of these physical properties contribute to some chemical validity? For example, does the peptide bond length get closer to the actual length?

We found that through optimization of these physical properties, although steric clashes cannot be completely avoided, they can be further minimized to improve chemical validity. Here, we evaluate the chemical validate with the mean absolute error (MAE) of peptide bond length between native antibodies and designed antibodies. When comparing our base model and the preference-optimized model, we found that after preference optimization, the peptide bond length became closer to the actual length. Moreover, AbNovo outperformed other comparative methods in terms of peptide bond length.

Question 4: The standard deviation of the physical energy needs to be presented.

Following you suggestion, we have now computed the standard deviation of the physical energy. Specifically, for each designed antibody, we ran Rosetta 10 times and report the standard deviations in the table below. The low standard deviation across all methods indicates that this benchmarking metric is robust.

Question 5: The AAR performance is excessively high, and it's necessary to check whether the training data of the protein language model contains samples similar to the test set.

The structure-aware language model used by AbNovo is trained on PDB and AFDB structural data. When training this language model, we strictly filtered out all antibody structures and sequences from the training dataset.

Question 6: I am curious about how many amino acids have mutated in those designed antibodies that outperform natural ones (at least in binding energy).

Following your suggestion, we analyzed the designed antibodies that outperform the natural ones in terms of binding energy. On average, 44.9% (standard deviation of 5.8%) of amino acids were mutated in the CDR H3 region, and 19.8% (standard deviation of 2.5%) of amino acids were mutated across all CDR regions.

We sincerely appreciate your valuable suggestions, which have helped us further improve the quality of our article. We have provided point-to-point responses to your comments as follows.

Weakness 1: Seems to be an updated version of AbDPO, somewhat heavier but showing better performance.

Our method differs with AbDPO in both the optimization framework and the underlying motivation.

AbDPO employs direct preference optimization:

\max \ \ \ \ \sum_{i} R_i-\beta \mathrm{KL}(p_{\theta} |p_{\rm ref}) \ \ \ \ \Equation 1.

In contrast, AbNovo utilizes constrained preference optimization:

$\max \ \ \ \ \ \ \sum_{i} R_i-\beta \mathrm{KL}(p_{\theta} |p_{\rm ref}) \ \ \ \$Equation 2.

{\rm s.t.} \ \ \ \ C_j<C_{{\rm limit},j} \ \ \ \ \ for $j$ in all constraint sets

Our framework is novel in diffusion-based generative models and is supported by rigorous theoretical derivations and proofs.

The key motivation is that crucial biochemical properties in practical antibody development often present inherent trade-offs—for example, improving binding affinity may increase the risk of non-specific binding to non-target proteins [1, 2]. Simply combining multiple objectives, as in Equation 1, struggles to balance these trade-offs effectively, often allowing one objective to dominate at the expense of others. In contrast, the constrained preference optimization framework allows us to set thresholds for certain properties (e.g., specificity) while optimizing others (e.g., affinity). It enables dynamic adjustment of the relationship between 'objectives' and 'constraints' through the Lagrangian method, thereby mitigating trade-offs to some extent.

In our initial submission, we included an ablation study comparing the two optimization frameworks (see the "Multi-objective" row in Table 2, Line 474). In this study, we incorporated all constraints into the objective function as shown in Equation 1. The results demonstrate that AbNovo achieves better performance in Rosetta Binding Energy, Evolutionary Plausibility, RMSD, and AAR. In contrast, the ablation model tends to focus solely on satisfying the constraints, resulting in slight benefits in the metric of fulfilling all constraints but substantially sacrificing performance in other metrics.

Similar trade-offs are observed in language model preference optimization, balancing user-helpful responses against safety concerns (e.g., avoiding offensive content) [3, 4]. One of our contributions is extending the constrained optimization framework for language models to multimodal diffusion models, providing theoretical support for this extension.

[1] Makowski, et al. Optimization of therapeutic antibodies for reduced self-association and non-specific binding via interpretable machine learning. Nature Biomedical Engineering 2023.

[2] Makowski, et al. Co-optimization of therapeutic antibody affinity and specificity using machine learning models that generalize to novel mutational space. Nature Communications, 2023.

[3] Bianchi,et al. Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions. ICLR 2023.

[4] Liu, et al. Enhancing LLM Safety via Constrained Direct Preference Optimization. ICLR WorkShop 2024.

Weakness 2: The task setting is overly simplistic. Although the structure of the antibody's FR region is relatively conserved and can be considered known, the binding pose between the antibody and antigen is typically unknown. However, given that the main goal of this work is to propose a new method for antibody optimization, this limitation is understandable.

Thank you for your insightful comment regarding the limitation of our method. When the binding pose between the antibody and antigen is unknown, recent approaches in antibody design [1] utilize protein structure prediction and docking software to determine the pose before designing antibodies. In response to your concern, we conducted a new experiment where we first establish the binding pose following the strategy used in these methods, and then design the CDR regions with AbNovo.

As demonstrated in the table below, AbNovo continues to outperform other methods in this application scenario. We have included this table in Appendix (Table 9, Line 886).

[1] Luo, et al. Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures. NeurIPS 2022.

Thank you for valuable suggestions, which have helped us significantly improve the quality of our manuscript. We have provided detailed responses point-to-point to your comments as follows. We hope that our responses and additional experiments address your concerns.

Weakness 1 : Rosetta energy is used as an alignment metric. It is well-known that forcefield energies have a weak correlation with measured binding affinity, typically around 0.3 [1,2]. This may lead to the totally wrong direction.

Thank you for raising this important concern. We agree that forcefield energies are limited in measuring binding affinities. We would like to clarify our methods and the rationale for using forcefield energies as follows.

First, our method does not rely solely on forcefield energies for optimization. Instead, we employ a unified framework of constrained optimization that integrates multiple objectives and constraints to guide the optimization process and prevent it from diverging in an incorrect direction. Specifically,

1. We include the likelihood under a large-scale protein language model as an optimization objective, which has proven effective in improving antibody screening success rates in wet-lab experiments [1, 2].
2. We incorporate constraints for other biophysical properties such as specificity and low self-association, which have also proven useful in guiding antibody design in experimental settings [3, 4].
3. During preference optimization, we introduce a regularization term in the training loss function (Equation 3, Line 236, Page 5) to ensure the fine-tuned model remains close to the base model, preventing arbitrary divergence.

Second, despite its limitations, Rosetta binding energy has been utilized as a screening metric, and several proteins designed using it have been experimentally validated in wet-lab experiments [5, 6]. Thus, binding energy is widely used as a metric for benchmarking recent generative models for antibody design, such as DiffAb [7], dyMEAN [8], AbX [9], and AbDPO [10].

Third, the AbNovo framework is flexible and allows the inclusion of other constraints or optimization objectives important for antibody design. One of our key methodological contributions lies in building the framework of constrained preference optimization for antibody design, providing rigorous theoretical derivations and proofs.

Finally, we acknowledge that, although forcefield energy is widely used as a metric in recent work, there remains a gap in its correlation with experimental results. We have included this point in the Discussion section (Line 533) to raise awareness in the community and inspire the development of improved methods.

[1] Hie, et al. Efficient evolution of human antibodies from general protein language models. Nature Biotechnology 2024.

[2] Shuai, et al. IgLM: Infilling language modeling for antibody sequence design. Cell System 2023.

[3] Makowski, et al. Optimization of therapeutic antibodies for reduced self-association and non-specific binding via interpretable machine learning. Nature Biomedical Engineering 2023.

[4] Makowski, et al. Co-optimization of therapeutic antibody affinity and specificity using machine learning models that generalize to novel mutational space. Nature Communications, 2023.

[5] Cao, et al. Design of protein-binding proteins from the target structure alone. Nature 2022.

[6] Sun, et al. Accurate de novo design of heterochiral protein–protein interactions. Cell Research 2024.

[7] Luo, et al. Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures. NeurIPS 2022.

[8] Kong, et al. End-to-End Full-Atom Antibody Design. ICML 2023.

[9] Zhu, et al. Antibody Design Using a Score-based Diffusion Model Guided by Evolutionary. ICML 2024.

[10] Zhou, et al. Antigen-Specific Antibody Design via Direct Energy-based Preference Optimization. NeurIPS 2024.