Venus-MAXWELL: Efficient Learning of Protein-Mutation Stability Landscapes using Protein Language Models

Thank you for carefully reviewing our manuscript and providing constructive feedback. Below we respond to your questions and concerns. We would appreciate it if you could let us know whether your concerns are addressed by our response.

Response to weakness 1.

(1) Applications. Directed evolution (DE) often begins with single-point mutations. MAXWELL can identify stable single mutations of the target protein for experimental validation, which can then be combined into multi-point mutants. The ΔΔG of multi-point mutants can be predicted by existing "train-on-single-and-predict-on-multi" models such as ECNet [1] or ProteinNPT [2]. Unlike these methods, MAXWELL requires no prior mutational data, making it useful in early-stage DE.

(2) Multi-Point Mutations Prediction. Although MAXWELL isn't trained on multi-point mutants, it is able to predict their stability via additive scoring: $Score(M) = \sum_{i=1}^{n} Score(m_i)$, where $Score(m_i)$ is the individual predicted effect of each single-point mutation. On the multi-point mutations from public Mega-scale ΔΔG dataset and M1261 dataset, MAXWELL significantly improves prediction accuracy over the PLM baseline, increasing Spearman correlation from 0.317 and 0.272 to 0.597 and 0.446(Table R1).

Table R1. Multi-point mutant prediction performance

Note that MAXWELL and other ΔΔG models (ThermoMPNN, StabilityOracle) primarily target single-point mutants, because:

• DE workflows prioritize reliable single-mutant prediction.
• Accurate multi-point predictions usually need prior single-mutant data, often unavailable.
• Multi-point datasets are typically smaller, restricting model effectiveness.

We hope this clarifies MAXWELL's design principles.

Response to weakness 2.

MLP serves as an auxiliary component rather than the core predictor for several reasons:

(1) The MLP is randomly initialized and lacks pretraining, whereas the PLM output head is pretrained and estimates ΔΔG via log-probability differences between mutant and wild-type residues.

(2) The MLP outputs ΔΔG values, but ranking is more relevant in practice. Rankings from the language model head are more accurate. For instance, MAXWELL (ESM-IF) trained only with MSE loss achieves a Spearman correlation of 0.492, lower than the language-model head trained solely with ranking loss (0.506 vs. 0.492, Table 2 in the main text).

(3) The MLP mainly aids training by providing a ΔΔG-based loss. It improves PLM head performance (e.g., Spearman corr. from 0.506 to 0.517, Table 2 in main text).

Response to weakness 3.

We performed a qualitative analysis to understand MAXWELL's behavior across proteins. We found a strong correlation (ρ = 0.685, R² = 0.469) between MAXWELL's performance and the PLM's zero-shot accuracy, confirming that our method effectively leverages pretrained knowledge. Correlations with protein length (ρ = -0.378, R² = 0.058) and dataset size (ρ = 0.307, R² = 0.022) were weak, suggesting robustness to these factors. Additionally, we examined structure quality using AlphaFold pLDDT scores and observed only a weak correlation with performance (ρ = 0.264, R² = 0.070), suggesting that MAXWELL is relatively insensitive to moderate structure noise. The slight size trend is driven by a few large datasets (>200 mutants) where MAXWELL performs consistently well. These findings will be included in the revision.

Response to weakness 4. We apologize for not including the code in the supplementary materials. Code and data will be released in the next version.

Response to question 1. Though simple, PLM output heads are pretrained to map representations to logits, enabling zero-shot ΔΔG prediction. To validate their importance, we re-initialized the head randomly. Table R4 shows performance drops sharply, confirming its critical role.

Table R4. Performance of MAXWELL with and without the pretrained head (Spearman corr. on Test12K).

Response to question 2. We quantified the occurrences of duplicates and misalignments in publicly available stability datasets, including Myoglobin, S669, S8754, M1261, vb1432, Fireprotdb, and Thermomutdb. The results are presented in Table R5.

Table R5. Summary of Data Issues in Public Stability Datasets

Response to question 3. We apologize for the typo and will correct it.

Response to question 4. Thank you for pointing this out. The PLM input is indeed an integer token sequence. We will update the figure.

Response to question 5. In short, the MLP is only used for auxiliary tasks. Pearson correlation serves as an alternative ranking loss.

5.1 Pearson correlation serves as a simple yet effective ranking loss. We tested other ranking losses and observed similar performance (see Table R6).

Table R6. Performance of different ranking losses.

5.2 MAXWELL aims to rank ΔΔG values rather than predict their exact ΔΔG values, as ranking is more important in practice. The MLP is auxiliary (see response to Weakness 1). Applying the MLP after the element-wise operation yields similar results (Spearman: 0.514).

5.3 Adding Pearson loss to the MLP branch had negative effect (Spearman: 0.507).

5.4 They are trained jointly. Using only MSE loss reduces performance, and MAXWELL (ESM-IF) drops to Spearman corr. 0.496.

Response to question 6. For zero-shot scoring, we input the full, unmasked protein sequence into a BERT-style PLM, which outputs amino acid probability distributions at each position. The log-probability difference between the mutant and wild-type residues serves as a fitness proxy. We do not apply masking because (1) BERT-style models can handle unmasked inputs due to their pretraining (including random replacements and unchanged tokens), and (2) masking each position individually greatly increases computation and breaks the ability to compute the full mutational landscape in a single forward pass. And, in practice, the performance difference is negligible. For example, with ESM2, Spearman correlation is 0.273 (with masking) vs. 0.268 (without).

Response to question 7. We tested various λ values (Table R7), and λ = 0.1 yields the best performance.

Table R7. MAXWELL (ESM-IF) Performance Across λ

Response to question 8. We have evaluated additional models. Results are shown in Table R8:

Table R8: Performance of Additional Baseline Models.

Unfortunately, we were unable to reproduce StabilityOracle. The official code could not be executed.

Response to question 9. While MAXWELL outperforms baselines on average, it underperforms ThermoMPNN on 143 of 308 proteins (~46%). These often overlap with proteins where the language model's zero-shot performance is low (114/143 below average), highlighting the importance of the PLM's intrinsic capability. Fortunately, our method is general and can improve as language models advance.

Response to question 10. We appreciate your valuable suggestion. (1) MAXWELL's role in DE is discussed in our response to Weakness 1.
(2) While MAXWELL focuses on single-point mutation prediction, it does not directly address iterative design or multi-site optimization. Such tasks are better suited for models like ECNet [1] or ProteinNPT [2] that integrate modeling with experimental feedback. MAXWELL can be trained with experimental data, and we plan to explore this in future work.
(3) Although ESM-IF is structure-aware, its mutant scoring only requires the wild-type sequence and structure, not mutant structures. So it introduces no computational bottleneck. We will clarify MAXWELL's role in directed evolution more explicitly in the revised Introduction. Thank you again for the insightful feedback.

Response to question 11. Yes, your intuition is correct. Theoretically, our framework is task-agnostic and can be adapted for other functional prediction tasks by substituting the training data. We chose ΔΔG as our primary focus because it is currently the ideal use case, benefiting from both the availability of large-scale public data and a theoretical link to PLM log-likelihoods [3]. Extending this framework to other tasks is a future direction, contingent on the curation of suitable datasets and potential loss function adjustments.

Response to limitations
We appreciate your suggestion to clarify the limitations. As discussed in Response to Weakness 3, MAXWELL's performance primarily depends on PLM prior quality and is relatively robust to structure quality. Its role in directed evolution is addressed in Response to Weakness 1. For more advanced DE tasks such as iterative design or combinatorial optimization, we plan to explore these directions in future work. We will make these points clear in the revised manuscript.

[1] ECNet is an evolutionary context-integrated deep learning framework for protein engineering. Nature Communications, 2021.

[2] ProteinNPT: Improving Protein Property Prediction and Design with Non-Parametric Transformers. NeurIPS, 2023.

[3] Zero-shot protein stability prediction by inverse folding models: a free energy interpretation. arXiv, 2025.

Thank you for your valuable feedback to help us improve our paper. We detail our response below, and please kindly let us know if you have any further questions.

Response to the question. We thank you for this thoughtful question, highlighting the need to explain why MAXWELL can outperform methods that explicitly regress from the embeddings of mutated sequences.

While it may seem counterintuitive that an implicit method outperforms an explicit one, the superior accuracy of MAXWELL stems from two key points:

1. MAXWELL leverages the zero-shot mutation prediction capabilities of pretrained protein language models, providing a stronger starting point than methods that regress directly from the embeddings of mutated sequences. Protein language models (PLMs) are pre-trained on millions of naturally occurring proteins, learning statistical patterns that reflect evolutionary constraints -- unstable proteins are unlikely to appear in nature. Consequently, the probability ratio (or log-probability difference) between a mutant amino acid and the wild-type residue at a given position can serve as a proxy for evolutionary preference. Larger differences suggest mutations that are more evolutionarily plausible and potentially more stable.

   Importantly, MAXWELL retains every component of the protein language model, including its output head, thereby preserving the model's full zero-shot capability. Its initial performance is therefore equivalent to the zero-shot performance of the PLM itself. In contrast, methods that regress from the embeddings of mutated sequences typically discard the original language model head and instead attach a randomly initialized MLP regressor. As a result, their initial predictions are nearly random, and they lose the valuable prior knowledge encoded in the pretrained output head. To validate the importance of the pretrained output head, we include an ablation study in which the language model head is randomly re-initialized. As shown in Table R1, this leads to a drop in performance, confirming the critical role of the pretrained head in enabling MAXWELL's accuracy. Additionally, as demonstrated in our paper, if the entire PLM is randomly initialized, performance drops dramatically, falling even below the zero-shot baseline. Taken together, these results prove that MAXWELL’s primary advantage lies in its ability to effectively leverage the evolutionary information captured by the PLM during pre-training. Because MAXWELL is compatible with various PLMs, its performance is poised to improve as this foundational technology continues to advance.

   Table R1. Performance of MAXWELL with and without the pretrained PLM

   Model	Spearman
   MAXWELL (ESM-IF) with pretrained head	0.517
   MAXWELL (ESM-IF) with randomly initialized head	0.432
   MAXWELL (ESM-IF) with randomly initialized whole PLM	0.202
   ESM-IF Zero-shot	0.375

2. MAXWELL utilizes a matrix-based fine-tuning algorithm that enables efficient training on protein mutation ΔΔG data and supports direct landscape-level prediction for unseen proteins. In contrast, traditional methods regress from the embeddings of individual mutated sequences and produce a scalar prediction for each mutation. MAXWELL instead reformulates the task as a two-dimensional mutational landscape regression problem. This matrix formulation treats the set of single-point mutations as a structured landscape matrix, enabling the model to learn a protein's mutational landscape within a single forward pass. This design not only accelerates training but also improves generalization across diverse proteins. To assess the impact of the matrix formulation, we conducted an ablation study in which the model was fine-tuned using a conventional pointwise regression approach. Each individual mutation was scored using a mean squared error (MSE) loss in this setting, corresponding to likelihood-based fine-tuning without the matrix structure. As shown in Table R2, removing the matrix-based structure resulted in a clear drop in performance and slower convergence, with the Spearman correlation decreasing from 0.507 to 0.401.

   Table R2. Performance of matrix-wise (MAXWELL) and pointwise regression on ΔΔG landscape prediction (Base model: ESM-IF).

   Fine-tuning Method	Landscape Structure	Spearman	Training Speed
   MAXWELL	2D landscape	0.507	Fast
   Point-wise, likelihood-based	1D (single mutation)	0.401	Slow

We will revise the main paper to explicitly clarify why MAXWELL outperforms methods that regress from mutated sequence embeddings.

Thank you for the insightful comments. We have addressed each of your concerns below and will incorporate these clarifications into the revised manuscript. We hope our responses resolve your concerns and are happy to discuss further if any points remain unclear.

Response to weakness 1: To answer your question thoroughly, we'll address it in points.

Point 1.1 “The weighting factor λ = 0.1 in the joint loss appears arbitrary.”

Response: We agree the choice of hyperparameters requires justification. The value for λ was determined empirically through an ablation study, with the results shown in Table R1.

The model achieved the best performance on the test datasets when λ=0.1. This result suggests that the framework performs optimally when the ranking loss serves as the primary training objective and the MSE loss acts as a beneficial auxiliary objective to align the scale of the predictions. We will add this ablation study to the appendix of our revised manuscript. Table R1. Performance of MAXWELL (ESM-IF) for various λ values

Point 1.2 “The MLP architecture details (dimensions of $W_1$, $W_2$) are not specific.”

Response: The MLP consists of two hidden layers. The dimensions are $W_1 \in R^{V \times V}, W_2 \in R^{V \times 1}$ where V represents the vocabulary size of the protein language models (PLMs) (e.g., 35 for ESM-IF and 25 for ProSST). A hidden activation function is applied between $W_1$ and $W_2$.

The MLP's operation is defined as: $MLP(L) = SELU(LW_1 + b_1)W_2 + b_2$ We will revise this section in the paper.

Point 1.3 “It's unclear whether "optimal performance" in tuning learning rates refers to validation loss (and how is the validation set constructed?), or inadvertently test set performance, which would be problematic.”

Response: We selected hyperparameters based on 5-fold cross-validation. All validation sets were derived solely from the training set, ensuring no leakage of test data into the training or validation process. Our procedure for selecting the learning rate (lr) and training epochs is as follows:

TrainingSet = {T1, T2, T3, T4, T5}
For each lr in {1e-3, 1e-4, 1e-5, 1e-6,5e-3,5e-4,5e-5,5e-6}:
    Initialize scoreset as empty list
    For i from 1 to 5:
        InternalTrainingSet = TrainingSet \ {Ti}  
        InternalValidationSet = Ti 
        Train model on InternalTrainingSet for 10 epochs:
            For each epoch in 1..10:
                Record Spearman correlation score sj on InternalValidationSet
                Track the epoch with the best score
        
        Add the best score of this fold to scoreset
    CurrentLrScore = Average(scoreset)
End For

Table R2 illustrates the performance of different learning rates during 5-fold cross-validation. We selected 5e-5 as the optimal lr, as it demonstrated the best performance across the 5-fold cross-validation. The best Spearman correlations on the validation sets for this lr were achieved at epochs {7,7,6,8,7} respectively, averaging 7 epochs. Subsequently, we trained the final model on the entire training dataset for 7 epochs, without any intermediate evaluation during this phase. After training, the model's performance was directly evaluated and reported on the test set. Throughout this entire process, the test set (Test12K) was used exclusively for reporting the final scores and was not involved in any hyperparameter tuning, thereby preventing any data leakage.

We will supplement the details of our hyperparameter selection process in the Appendix.

Table R2. Performance of MAXWELL (ESM-IF) with different learning rates in 5-Fold cross-validation

Response to weakness 2: To answer your question thoroughly, we'll address it in points.

Point 2.1 On Motivation Beyond Computational Efficiency

Response: Our framework offers several key advantages:

Versatility with Protein Language Models: As PLMs continue to advance, our model is designed as a universal fine-tuning framework, compatible with diverse protein language models (PLMs), including masked language models and inverse folding language models. This allows researchers to easily integrate the latest or most suitable PLM for their target protein without re-designing the architecture.

Novel Fine-tuning Methodology: We introduce a novel matrix-based fine-tuning paradigm. Unlike existing approaches that often involve adding separate regression heads or utilizing non-matrix-based likelihood methods, our method directly and elegantly fine-tunes the PLM's output logits, offering a more streamlined way to adapt PLMs for specific tasks.

Maximized PLM Knowledge Utilization: MAXWELL fully capitalizes on the knowledge embedded within PLMs. The insights gained during the unsupervised pre-training phase of PLMs are largely preserved and directly integrated. By inheriting the pre-trained PLM, MAXWELL starts from a significantly advanced foundation.

Superior Accuracy: These advantages translate to superior empirical performance. MAXWELL (ESM-IF) outperforms the current state-of-the-art model, ThermoMPNN. Furthermore, ensembling our model with ThermoMPNN boosts the Spearman correlation to 0.548 on the Test12K dataset.

We will elaborate on these model advantages in the Introduction section.

Point 2.2 On Clarifying "Initial Performance"

Response:
The phrase "initial performance" refers to the predictive ability of our framework before any fine-tuning begins. This intrinsic capability is equivalent to the zero-shot ΔΔG prediction ability of the underlying PLM itself.

MAXWELL is initialized with the parameters of a pretrained PLM. Our framework's architecture is designed to be a differentiable, matrix-based version of the PLM's standard zero-shot scoring function. PLMs, pre-trained on millions of protein sequences, can compute a likelihood for a given protein sequence. The difference in likelihood between a mutant sequence and its wild-type counterpart can then be used to estimate the mutation's effect and ΔΔG. This approach, which doesn't require training on mutation-specific data, is known as zero-shot prediction. MAXWELL inherits this powerful zero-shot capability and can further enhance it through fine-tuning.

From a mathematical perspective, each element in the output matrix L from an unfine-tuned MAXWELL model is equivalent to $log (Mutant∣Wildtype) − log (Wild-type∣Wildtype)$, which represents the PLM's zero-shot output for ΔΔG estimation. The relationship between ΔΔG and the logarithmic ratio has also been mathematical derived[1].

Response to question 1: We primarily focused on optimizing the learning rate and number of epochs. The adjustment process relied on 5-fold cross-validation. This involved splitting the training set into five distinct folds. In each iteration, we used four of these folds for training and reserved the remaining one for validation. This process was repeated five times, and the average score from these five evaluations was used to determine the best hyperparameter combination.

Once the optimal hyperparameters were identified, we trained the final model on the entire training dataset using these tuned parameters. Afterward, the model's performance was evaluated on the test set. We strictly ensured that only the training set was used during the hyperparameter tuning process, thereby preventing any data leakage.

Specifically:

Point 1.1 What was the search space for the learning rate grid search?

Response: The lr search range was set to {1e-3,1e-4,1e-5,1e-6,5e-3,5e-4,5e-5,5e-6}.

Point 1.2 How was "optimal performance" defined during hyperparameter selection?

Response: Optimal performance refers to the mean Spearman score achieved across the validation sets in a 5-fold cross-validation, given a specific set of hyperparameters. A higher mean Spearman score indicates a better hyperparameter combination.

Point 1.3 Can you provide ablation studies for the loss weighting factor λ?

Response: Yes，we also conducted an ablation study on the loss weighting factor λ, with the results presented in Table R1. (This is in response to a weakness identified.)

Point 1.4 What are the specific dimensions used in the MLP projection layer? How sensitive is performance to these choices?

Response: The MLP consists of two hidden layers. The dimensions are $W_1 \in R^{V \times V}, W_2 \in R^{V \times 1}$, where V represents the vocabulary size of the protein language models (PLMs) (e.g., 35 for ESM-IF and 25 for ProSST). A hidden activation function is applied between W_1 and W_2. The MLP's operation is defined as: $MLP(L) = SELU(LW_1 + b_1)W_2 + b_2$

The MLP's dimensions are designed to align with the protein language model's (PLM) vocabulary size. Here, V is a fixed vocabulary size, but we can consider adjusting the dimension D. In our paper, we set D=V.

We conducted an ablation study on the size of D, and the results are presented in Table R3. Taking ESM-IF as an example, we performed five sets of experiments with D={V, 64, 128, 256, 512}, while keeping other hyperparameters constant. The results in Table R3 indicate that the model's performance is relatively insensitive to the intermediate layer dimension. Specifically, the model achieved its best performance when the intermediate layer size was set to D=V.

Table R3. Model performance with varying hidden layer sizes

[1] Zero-shot protein stability prediction by inverse folding models: a free energy interpretation. arXiv, 2025.

We would like to appreciate the reviewer for the evaluation and comments. We would appreciate it if you could let us know whether our response addresses your concerns.

Response to weakness 1: We agree that MAXWELL is primarily designed for training on single-point mutations. However, it is still capable of making predictions for multi-point mutations using the feasible prediction formula: $Score(S) = \sum_{(t,j)}L_{t,j}$ where $S = \{(t_1,j_1),(t_2,j_2),...(t_k,j_k)\}$ represents a multi-point mutant constructed by $k$ single-point muttations and $L$ is the mutation landscape as defined in the paper.

Table R1 presents MAXWELL's performance on the multi-point mutation subset of the public Mega-scale ΔΔG dataset and M1261 dataset. The results confirm that MAXWELL effectively enhances multi-point mutation prediction capabilities, demonstrating its practical utility in this regard.

Table R1. Multi-point mutation prediction performance of related models

While MAXWELL can handle multi-point mutation prediction tasks, our primary objective is to contribute a high-performance single-point mutation prediction model. This focus aligns with typical directed evolution workflows, which often begin with single-point mutations and then leverage experimental data to guide multi-point exploration. MAXWELL can provide initial stability predictions without experimental data. These recommendations can then be integrated with other models, such as ECNet and ProteinNPT, to guide the design of multi-point mutations, creating a powerful, iterative optimization cycle when combined with wet-lab validation.

Response to weakness 2: The idea that "probabilities of mutant and wild-type proteins can be used to predict mutation scores" is a well-validated conclusion in protein language models (PLMs). ProteinGym's benchmark quantifies the correlation between these PLM-derived probabilities and mutant phenotype, clearly demonstrating a notable association. This has become a key indicator for evaluating the performance of PLMs. Intuitively, pre-trained PLMs have learned the natural distribution of numerous protein sequences. Their calculated probabilities essentially reflect a mutant's likelihood of existing in nature—an inherent indicator of mutation stability, as unstable mutants rarely persist in natural contexts.

Theoretically, prior research [1] has established that the predictive power of these probabilities for stability stems from an approximate mathematical relationship: the change in thermodynamic stability (ΔΔG) correlates with the logarithmic ratio of probabilities derived from inverse folding models, as expressed by:

$\beta \Delta \Delta G_{a \rightarrow a^{\prime}} \approx -\ln \frac{p_{\theta}\left(a^{\prime} \mid x_{a}\right)}{p_{\theta}\left(a \mid x_{a}\right)} - \ln \frac{p_{\theta}(a)}{p_{\theta}\left(a^{\prime}\right)}$

To enhance clarity, we will provide a detailed theoretical explanation of this relationship at the beginning of the Methods section. We believe this will help readers better grasp the underlying mechanism.

Response to weakness 3: Test12K is a curated meta-benchmark composed exclusively of established, publicly available benchmarks to ensure a comprehensive and challenging evaluation. To directly address your concern, we have tested the performance of MAXWELL on several public benchmarks, comparing it against current state-of-the-art models.

As shown in Table R2, MAXWELL consistently outperforms ThermoMPNN and Stability Oracle on well-known benchmarks including p53, s669, and Myoglobin, all of which are subsets of Test12K. Evaluation protocol is: for each dataset, we calculate the correlation score on a per-protein basis and then average these scores. A full evaluation against Stability Oracle on the complete Test12K was not possible, as the officially provided source code was non-functional.

The results demonstrate that MAXWELL achieves state-of-the-art performance on these public benchmarks. We deliberately excluded the popular ProteinGym benchmark from our evaluation. This exclusion is due to ProteinGym's overlap with the training data of both our model and the ThermoMPNN model, which would lead to data leakage and invalidate the conclusions.

Table R2. Model Performance on Public Benchmarks (Mean Per-Protein Spearman)

Response to weakness 4: Thanks for the valuable reminder. We have additionally compared MAXWELL with other fine-tuning methods (all based on the ESM-IF): likelihood-based fine-tuning and one-hot augmented fine-tuning.

As shown in Table R3, MAXWELL consistently achieves the optimal performance. It's worth noting that while these alternative methods can be adapted for this task, they were originally designed for intra-protein learning (predicting new mutations on a protein seen during training) rather than generalizing to entirely new proteins, which is the focus of our benchmark.

We will add these new comparative results to the baseline section of our revised paper to provide a more comprehensive evaluation.

Table R3. Performance comparison between MAXWELL and other fine-tuning frameworks

Response to question 1: Thank you for your reminder. There was indeed a typo in Equation 2, which has now been corrected to:

Response to question 2: Thank you for this important question. We agree that data sparsity is a key consideration. Our training set, Train226K dataset, is substantial, containing over 226,000 mutation entries across 255 proteins, with an average of around 887 mutations per protein. While the mutation landscape for any single protein is sparse (due to the experimental infeasibility of measuring all possible mutations), the scale and diversity of the combined dataset allow the model to learn generalizable patterns of sequence-stability relationships. MAXWELL's strong performance on the Test12K dataset, which exhibits low sequence and structural similarity to the training set, clearly demonstrates its robust generalization, indicating it's not merely memorizing sparse landscapes.

To verify how sparsity affects model generalization, we conducted ablation experiments. Using ESM-IF as the base model, we randomly removed $p$ of the training data from the training set, where $p=0$ represents the original performance. The results, presented in Table R4, show that our framework is exceptionally robust to data sparsity. Performance remains remarkably stable, with a Spearman's correlation of 0.501 even when using only 10% (23K mutants) of the training data. A significant performance drop only occurs at the extreme of removing 95% of the data. This demonstrates that our model learns generalizable stability principles from the diverse collection of proteins, rather than overfitting to the sparse landscape of any single protein.

Table R4. Effect of training set sparsity on model performance

Response to question 3: Thank you for this insightful question. We performed the suggested experiment, and the results validated our framework's design. As shown in Table R5, when we added the ranking loss to the MLP's output, the model's performance degraded. We hypothesize that primary evolutionary information resides within the language model's output head, and its score rankings correlate with protein mutation ΔΔG. This correlation makes ranking loss suitable for the language model's direct output. Conversely, the randomly initialized MLP lacks this inherent evolutionary information, rendering it unsuitable for ranking loss application. We will include these results in our ablation experiments.

Table R5. Impact of loss in MLP on model performance

Response to question 4: We acknowledge this potential risk. We did not perform full training for all models due to prohibitive computational speed. While MAXWELL only requires a single computation per protein during full training, MLP baselines demand feature computation for each individual mutant sequence, making their full training exceptionally slow. We supplemented experiments on full training experiments for ESM-IF + MLP and LoRA and the training results using LoRA (configured with lora_r = 8 and lora_alpha=16), as presented in Table R6. (Note: Full training for all baselines was limited by computing power and time constraints.)

The results in Table R6 indicate that the performance of ESM-IF + MLP is slightly lower than ESM-IF (LoRA) + MLP but notably higher than ESM-IF (no freeze) + MLP. For the fully trained ESM-IF, its performance drops significantly after training. We speculate this decline is due to overfitting caused by an excessively high number of model parameters. We'll include these results in our ablation experiments.

Table R6. Training results for ESM-IF + MLP

[1] Zero-shot protein stability prediction by inverse folding models: a free energy interpretation. arXiv, 2025.