Protein Inverse Folding From Structure Feedback

We sincerely thank the reviewer for the detailed and constructive feedback. We greatly appreciate the recognition of the strengths of our work, including its robustness, extensibility, and clarity of presentation. At the same time, we acknowledge the concerns raised regarding certain methodological and evaluative aspects. Below, we provide point-by-point responses to the weaknesses and questions.

Weakness 1, Question 1: We thank the reviewer for the valuable suggestion regarding statistical rigor and visualization. As the reviewer noted, Fig. 4 is presented as a boxplot, which includes the 25th percentile, median, and 75th percentile, providing a summary of the distribution and capturing potential skewness and variability in performance. To further address the reviewer’s concern, we perform t-test on the original data of Fig. 4:

We also have conducted an additional evaluation using AlphaFold 3 to assess the robustness of our improvements across different structure predictors. Specifically, for each test structure where multi-round DPO was applied, we folded 10 designed sequences using AlphaFold 3, resulting in 200 sequences in total (100 from the baseline and 100 from the fine-tuned model). We report the average TM-Score and RMSD per structure, along with standard deviations, in the table below:

We believe these results clearly reflect the significant performance improvements of our methods.

Weakness 2: We conducted additional experiments with smaller increments in the number of contrastive samples (0, 10, 20, 30, 40, 80). The results show that performance improves up to around 20 contrastive samples, after which it begins to plateau or slightly degrade. Specifically, both TM-score and RMSD indicate that using 20 samples yields the best balance between structural accuracy and stability, confirming the trend observed in our original submission. These findings support our original choice and reinforce the importance of moderate contrastive sampling in DPO training.

Weakness 3: We appreciate the reviewer’s concern regarding the biological relevance of the optimization target. We fully agree that incorporating functional and energetic criteria is crucial for real-world protein design. However, our primary goal in this work is to demonstrate that protein design algorithms can be effectively optimized entirely in silico, without relying on wild-type sequences or wet-lab supervision. To do so, we intentionally focus on TM-score as a well-established structural similarity metric, which provides a consistent and objective signal that can be computed automatically at scale. We view our approach as a proof of concept showing that feedback from structure prediction models alone can significantly improve inverse folding performance. We hope that our work can inspire future research to extend this framework by incorporating more biologically grounded objectives and stimulating the development of more accurate biological tools in the community.

Weakness 4: We thank the reviewer for raising this important concern about potential risk amplification in multi-round DPO training. Indeed, recent work such as Ren et al. (2024) [1] has highlighted how iterative optimization can reinforce early-stage biases or drift from desired distributions. We will explicitly discuss this risk in the Limitations section of our revised manuscript. In our experiments, we report TM-Scores and Recovery throughout the multi-round training process to ensure that the model continues to learn meaningful structural features; please refer to Fig. 6 for details. However, we would also like to emphasize that the primary contribution of our work is to demonstrate that fully in silico fine-tuning of protein language models using no external annotations or experimental data is feasible and can lead to significant performance gains. Our focus is thus on exploring and validating the possibility of DPO in protein design, rather than fully solving the challenge of safe or robust reinforcement-based learning in this domain. We leave the exploration of more robust and effective training frameworks for future work.

Weakness 5, Question 3: We thank the reviewer for the thoughtful question regarding the novelty of our method compared to existing DPO-based approaches in protein design. First, we would like to emphasize that all our results are obtained solely based on the model’s own predictions, without relying on any external supervision such as wild-type sequences, evolutionary constraints, or wet-lab annotations. This contrasts with existing methods such as ProteinDPO[2] and ProtGPT2 (DPO)[3], which leverage annotated or curated datasets during training. Our framework instead demonstrates that protein language models can be improved entirely in silico by leveraging structure predictors as feedback, making the fine-tuning pipeline significantly more scalable and self-sufficient. Second, while there are a few concurrent works that also incorporate predicted structural feedback (e.g., using ESMFold) into the DPO process, these studies are typically limited in scope and scale. For instance, Stocco et al. focus only on a single enzyme [4], and Park et al. apply DPO to peptide sequences[5]. In contrast, our work performs DPO on over 1.8M predicted sequences and evaluates the fine-tuned model across diverse structural folds, demonstrating generalizable improvement beyond a narrow domain or sequence type. Third, our study also provides initial insights and foundations for designing larger-scale, fully in silico protein design frameworks by studying the scaling behavior of DPO. We hope this work can serve as a stepping stone toward more robust and autonomous pipelines in future research.

We will further clarify these points in the revised version and highlight them in the Introduction and Related Work sections.

Weakness 6: We thank the reviewer for pointing out these presentation issues; we will clarify in the revised version. However, in terms of the Table. 1, since our framework is model-agnostic, we aimed to emphasize the performance gains achieved by our fine-tuned models relative to their respective baselines, rather than compare across architectures.

Question 2: We chose CATH 4.2 primarily for two reasons. First, our baseline models were trained on CATH 4.2, and to avoid any potential data leakage, we did not include CATH 4.3 structures in the training set. Using the same version ensures a fair and controlled comparison. Second, to better evaluate the generalization ability of our method, we conducted experiments on an out-of-distribution (OOD) set constructed from the CATH 4.3 release. Specifically, we selected protein domains from newly added Classes in CATH 4.3 that are not present in CATH 4.2, ensuring no overlap with the training data not just at the Fold level, but at the broader Class level. After filtering out sequences longer than 512 residues, we obtained a test set comprising 750 diverse and challenging structures. Compared to the original CATH 4.2 test set, which differs from training data at the Topology (Fold) level, this new dataset represents a more stringent test of generalization. The results show that our fine-tuned model consistently outperforms the baseline across all metrics. This performance gap is especially notable given the increased structural diversity and novelty in the CATH 4.3 set.

[1]: Ren, Yi, et al. "Bias amplification in language model evolution: An iterated learning perspective." Advances in Neural Information Processing Systems 37 (2024): 38629-38664.

[2]: Widatalla, Talal, Rafael Rafailov, and Brian Hie. "Aligning protein generative models with experimental fitness via direct preference optimization." bioRxiv (2024): 2024-05.

[3]: Mistani, Pouria, and Venkatesh Mysore. "Preference optimization of protein language models as a multi-objective binder design paradigm." arXiv preprint arXiv:2403.04187 (2024).

[4]: Stocco, Filippo, et al. "Guiding generative protein language models with reinforcement learning." arXiv preprint arXiv:2412.12979 (2024).

[5]: Park, Ryan, et al. "Improving inverse folding for peptide design with diversity-regularized direct preference optimization." arXiv preprint arXiv:2410.19471 (2024).

We thank the reviewer for the thoughtful comments and for highlighting the strengths of our work. We agree that recovery rate is not the most critical metric for de novo protein design, and we appreciate the recognition that our method improves structural similarity while maintaining reasonable recovery and perplexity levels. We also appreciate the suggested references. These works are indeed highly relevant to our study:

[1] presents a concurrent approach applying DPO fine-tuning to protein language models, which aligns closely with our motivation and methodology.

[2] emphasizes evaluating PLMs beyond recovery rate, which resonates with our view on the limitations of recovery as a sole metric.

[3] explores parameter-efficient fine-tuning of PLMs across diverse tasks.

We will include and discuss these works in the revised manuscript to better situate our contributions within the context of recent advances. Below, we address the reviewer’s questions and suggestions in detail:

Question 1: We appreciate the reviewer’s interest in understanding how our method performs across different fold types. To this end, we conducted an additional experiment where we grouped the test structures in CATH 4.2 according to their Class annotations (Mainly Alpha, Mainly Beta, Alpha-Beta, and Few Secondary Structures). The average TM-scores (mean and standard deviation) before and after DPO fine-tuning are summarized as follows:

As shown, DPO fine-tuning brings consistent improvements across all fold types, with especially notable gains in the Mainly Alpha and Mainly Beta categories. This demonstrates that our method not only improves structural quality overall, but does so robustly across a diverse range of protein topologies.

Question 2: All the results in Fig. 3 are obtained from the CATH 4.2 test set, which contains 1,120 test structures that do not share the same Fold with the training set. For each structure, we sampled 10 sequences using both the baseline and fine-tuned models, and then computed the mean value of each metric (RMSD, TM-score, and recovery rate.) across the 10 sequences. Finally, the plotted points in Fig. 3 represent the averages over all 1,120 structures. We would also like to clarify that the primary purpose of Fig. 3 is to analyze the scaling behavior of DPO with respect to different training configurations, including the number of training steps and training pairs. Our experiments reveal an interesting trend: while moderate amounts of preference pairs improve structural quality, excessive numbers of training pairs can actually degrade performance. This suggests that DPO benefits from high-quality, but not necessarily large-scale, feedback data, and highlights the importance of balancing signal strength with data quantity in preference-based optimization. For design sequences with high structure similarity but with low sequence recovery rate, we refer the reviewer to check our results on multi-round DPO in Fig. 6, which shows our methods achieve a higher TM-Score (0.7) but maintain low recovery (0.25).

Question 3: The majority of the computational cost in our pipeline is concentrated in the data construction phase, particularly during sequence generation and structure prediction (folding). In contrast, the actual DPO fine-tuning stage is relatively lightweight. In our setting, a complete DPO training run takes approximately 16 hours on a server with 8×A100 GPUs. To promote future research, we will also release our full training dataset upon publication, which includes over 1.8 million sequences and their predicted structures generated by ESMFold.

We thank the reviewer for their thoughtful and encouraging feedback. We are glad to hear that the paper is found to be well-written and the analyses insightful. We also appreciate the constructive suggestions for improving the clarity and completeness of our experiments. Below, we address the reviewer’s comments and concerns in detail.

Weakness 1: To train the ProteinMPNN model, we slightly changed our dataset curation process and constructed a simpler task. Specifically, we set a new threshold t_r = 0.8 to control the process of classifying samples into chosen and rejected. Instead of training on all predictions like InstructPLM, we only trained ProteinMPNN on the predictions with a TM-Score lower than t_r . Meanwhile, in order to increase the diversity of the chosen sequences, the wild-type sequence has also been introduced as a chosen sample during training. Details can be found in the Appendix A.1.4. This result highlights an important insight: applying DPO effectively may require more powerful and expressive sequence design models. ProteinMPNN, while efficient and widely used, may lack sufficient capacity to fully benefit from DPO-style fine-tuning. We hope our findings can inspire the development of stronger structure-conditioned design algorithms that are better suited for preference-based optimization techniques like DPO.

Weakness 2: We thank the reviewer for pointing out the importance of hyperparameter sensitivity analysis. We have conducted additional experiments varying the λ value and summarize the results (TM-Scores) below:

These results suggest that the model is relatively robust to λ values in a wide range. While λ = 1 yields slightly better performance overall, especially on challenging (TM-Score < 0.5) cases, smaller values like 0.1 or 0.01 also perform comparably well. We will include these findings in the revised manuscript to strengthen our analysis.

Weakness 3: We thank the reviewer for this suggestion. In Section 4.5, we initially selected 10 samples to qualitatively examine whether multi-round DPO can help the model successfully design structures that it previously failed to recover, focusing on cases where improvement is most meaningful. These samples were chosen based on relatively low baseline performance to better illustrate the effect of iterative optimization. We agree that evaluating on a larger and randomly selected subset provides a more comprehensive view. To this end, we extended the multi-round DPO experiment to 100 randomly sampled proteins from the CATH 4.2 test set. The results are summarized below:

We observe consistent improvement in both TM-Score and RMSD during the early rounds, indicating that multi-round DPO can generalize its benefits beyond the small and hard examples. However, the performance appears to saturate in the later rounds. This indicates that multi-round DPO is particularly effective in challenging cases with low initial performance, while for easier or already well-predicted samples, the benefit plateaus more quickly. We will incorporate these results into the revised version to strengthen our empirical analysis.

Weakness 4: We agree that evaluating additional baseline models would further demonstrate the generalizability of our approach. However, due to time and resource constraints during the rebuttal period, we were unable to extend our experiments to new baseline architectures. Notably, our current results already cover a diverse range of datasets and evaluation settings, including challenging OOD structures from CATH 4.3, which collectively demonstrate the robustness and structural awareness gained through DPO training. We consider this a promising direction for future work.

We thank the reviewer for their thorough and insightful comments. We appreciate the recognition of our method’s strengths in terms of clarity, methodological rigor, and the practical significance of integrating folding-based feedback via Direct Preference Optimization (DPO). Below we address each concern in detail.

Question 1, Suggestion 1, Question 3: We thank the reviewer for raising the important question regarding the effect of structural diversity in the training set on generalization. To investigate this, we conducted an ablation study by subsampling the number of training structures from the full CATH 4.2 training set. Specifically, we randomly selected 50% (8,987 structures) and 25% (4,515 structures) of the full set, and re-ran the training pipeline under the same protocol, 0% indicates baseline performance before fine-tuning. The results on the held-out CATH 4.2 test set are summarized below:

These results indicate that our method remains robust even when trained on a reduced number of structures, for example, it achieves comparable performance when using only 50% of the training set. However, as the number of training structures is further reduced, a noticeable drop in performance is observed. We will include these findings in the revised version to better illustrate the relationship between training set diversity and model generalization.

Question 2, Suggestion 2: We use a 50:50 split instead of thresholding for the following reasons. First, the performance of protein design models varies significantly across different target structures. Applying a fixed TM-score threshold, for example, 0.8, would result in many structures lacking valid negative responses, as the model may generate only sequences above the threshold. This would reduce structural diversity in training. Second, for structures where the model performs poorly, it becomes difficult to find valid chosen responses, which leads to severe class imbalance. In contrast, the 50:50 split can be seen as a dynamic, per-structure threshold that ensures a balanced set of preference pairs for each structure. We acknowledge that this approach may introduce some noisy signals—for instance, cases where chosen responses have relatively low TM-scores or rejected responses have high TM-scores. However, since DPO is a contrastive learning objective, it only requires that the chosen sample be relatively better than the rejected one, not absolutely good. Given these considerations, we favored a simple but robust strategy over more complex selection mechanisms.

Question 4: We agree with the reviewer that AlphaFold generally provides more accurate structure predictions than ESMFold. However, due to its significantly higher computational cost, AlphaFold is not well-suited for integration into the training loop. To examine the robustness of our method across different structure predictors, we evaluated designs generated by our baseline and fine-tuned models using AlphaFold 3. Specifically, for each test structure where multi-round DPO was applied, we folded 10 designed sequences using AlphaFold 3, resulting in 200 sequences in total (100 from the baseline and 100 from the fine-tuned model). We report the average TM-Score and RMSD per structure, along with standard deviations, in the table below:

These results confirm that the improvements achieved by our method are consistent across different folding models, suggesting that our approach captures structure-relevant features beyond model-specific feedback.

Suggestion 3: Combining structural feedback with additional objectives such as stability or binding affinity is indeed a promising direction. One of the main goals of our work is to evaluate whether fully in silico simulation alone can improve protein design performance, without relying on wet-lab feedback. Importantly, our framework does not make strong assumptions about the scoring function and is compatible with a wide range of design objectives. In principle, predicted metrics such as stability or binding affinity can also be seamlessly incorporated into our training pipeline. However, such metrics often require costly simulations or are less mature compared to structure-based evaluation. We leave the integration and benchmarking of alternative objectives to future work, and we hope our results can inspire broader exploration of these directions in the community.

Suggestion 4: The CATH 4.2 test set already contains folds that are disjoint from the training set, as we follow the data split protocol from [1], which ensures no overlap at the CAT (Topology/Fold) level [2]. Therefore, the results reported in Table 1 can be interpreted as evaluating generalization to unseen folds. To further assess the robustness of our method, we additionally evaluate it on selected structures from the newer CATH 4.3 release. Specifically, we selected protein domains from newly added Classes in CATH 4.3 that are not present in CATH 4.2, ensuring no overlap with the training data not just at the Fold level, but at the broader Class level. After filtering out sequences longer than 512 residues, we obtained a test set comprising 750 diverse and challenging structures. Compared to the original CATH 4.2 test set, which differs from training data at the Topology (Fold) level, this new dataset represents a more stringent test of generalization. The results show that our fine-tuned model consistently outperforms the baseline across all metrics. This performance gap is especially notable given the increased structural diversity and novelty in the CATH 4.3 set.

Suggestion 5: We appreciate the reviewer’s suggestion to visualize how DPO shifts the sequence distribution (e.g., using t-SNE on embeddings) to better understand the learning dynamics. Due to current constraints during the rebuttal phase, we are unable to include such analyses at this stage. However, we fully agree that improved visualizations would strengthen the work, and we plan to incorporate them in a future revision of the paper.

Suggestion 6: We conducted additional experiments with smaller increments in the number of contrastive samples (0, 10, 20, 30, 40, 80). The results show that performance improves up to around 20 contrastive samples, after which it begins to plateau or slightly degrade. Specifically, both TM-score and RMSD indicate that using 20 samples yields the best balance between structural accuracy and stability, confirming the trend observed in our original submission. These findings support our original choice and reinforce the importance of moderate contrastive sampling in DPO training.

Suggestion 7: We appreciate the reviewer’s suggestion regarding statistical significance. Due to computational constraints, we were not able to retrain the model multiple times with different seeds or bootstrapped training sets. However, the results reported in Table 1 are already averaged over 10 independently sampled sequences per target structure, providing a robust estimate of performance variability at the inference level. Below, we report the standard deviations across these 10 samples for each metric:

We believe these per-target statistics provide meaningful evidence of the improvements achieved by our method, even without full training-level replication. We agree that training-level variance is important and plan to include a more comprehensive statistical analysis in future work when resources permit.

[1]: Ingraham, John, et al. "Generative models for graph-based protein design." Advances in neural information processing systems 32 (2019).

[2]: Orengo, Christine A., et al. "CATH–a hierarchic classification of protein domain structures." Structure 5.8 (1997): 1093-1109.