From Likelihood to Fitness: Improving Variant Effect Prediction in Protein and Genome Language Models

We are very grateful for your thorough review and interesting questions on the work. We address your questions below:

1, 2, 3, 4.

Thank you very much for pointing out these issues in the manuscript. We will fix these in the camera ready version.

While informative in some regards, we also worry that these error bars may be misinterpreted. As such, we opted to remove them from the main text but nevertheless keep them in the supplement. The error bars in Figure F4b are obtained by resampling the DMS, however, the problem is that the assays are diverse in numerous regards. For instance, the correlation between experimental replicates is low in some cases and higher in others. This means that the maximum possible Spearman correlation between a model and an experiment varies between assays, and hence there is a tendency for all models to correlate well with some assays and poorly with others. When resampling we get a sense of this variability, but this has little to do with the statistical significance of the performance gain between one model and another, and we worry that some readers may interpret them as such. We should clarify this in the text.

To complement this analysis, we have performed permutation tests to check the significance of the mean difference in spearman between LFB and the baseline method. We did so with two-sided permutation tests with 100,000 iterations and found significance for both large models (p<10-5). We propose to add this analysis to the manuscript.

Results presented in Figure 2 and 3 and Table 1 used the full sequence neighborhood, without downsampling, except for the results using Evo 2 which were based on randomly selected 9 sequences (as outlined in section 3.2). We agree that a non-aggregated picture of the effect of subsampling on individual datasets/ proteins could be interesting. We will add it as a supplementary table.

The statement is that light versions of LFB with 10 sequences yield significant performance gains. And while according to Figure 4a, 10 sequences does seem like a reasonable choice for some applications, it was not our intention to make a blanket recommendation. We envisage practitioners performing their own cost-benefit analysis in the context of the application of interest.

Perhaps we are not always being sufficiently precise in regards to what we mean by lightweight and would be better off making comparative statements. If the model of interest is computationally heavy in the first place, then a 10x increase in the cost of scoring a sequence is not a negligible difference. On the other hand, scoring a sequence is generally a negligible cost compared to the cost of developing the model in the first place. Thus, as a strategy for improving performance, clearly LFB is extremely cheap compared to most, if not all, alternatives to developing a better model.

Regarding the computational cost of building sequence alignments, in the case of protein models, and as far as we are aware, this is rarely a significant bottleneck. For instance, we found making a UniRef50 alignment with MMseqs2 took around 20 seconds per sequence on 20 CPU cores, and using MMseqs2 GPU it is reported this can be reduced to around a second (Kallenborn et al. 2024). We are aware of some recent work, Protriever, that uses vector search to rapidly obtain related protein sequences. This could be an interesting future direction for LFB or LFB-like models.

For genomic language models the situation is different as whole genome alignment is extremely challenging. Fortunately publicly available precomputed alignments exist, such as the zoonomia alignment used in this work.

Thank you for the suggestion. We show the unmasked marginal formula in appendix D.2. eq 7, but currently it is quite unclear as we didn’t name it there as unmasked-marginal scoring, so we will ensure it is made clearer and link to this section in the main text.

Yes indeed LFB can be used for scoring indels. We have explored this a bit with Evo 2 using the performance assessment from the original paper: (non-coding indels; baseline 0.913, LFB 0.932) and (coding indels; baseline 0.812, LFB 0.816). We have some reservation about the quality of this performance assessment which is why we did not include these results in the manuscript. We could also in principle explore this with the ProteinGym indel benchmark and the ProGen2 family. We have not performed this analysis yet though as our priority so far has been on assessments which could be used across model families.

Given that performance improves in terms of area under ROC curves, the intention with this discussion was to explore in what way this improvement is being achieved. There will certainly be some cases where pathogenicity is underpredicted, but overall, in this case, it seems that the gain in performance is largely coming from a reduced tendency to over predict pathogenicity. That said, we think the situation is more nuanced, and how the distributions shift under LFB also depends on the scoring method. We are tempted to remove this paragraph.

This is a very interesting question. We did some analysis in order to see if there were particular kinds of experimental assays that performed worse (Figure F9) and, we also looked at the relationship to alignment depth (Figure F10), but found no notable trend. One hypothesis may be that among these proteins, the fitness landscape changed rapidly across evolution, invalidating the assumptions underlying LFB. Any suggestions for ways of checking this or other possibilities would be welcome.

The method was developed based on considerations of the data generation process and hence why likelihood does not purely reflect fitness. However, we agree that there are plenty of reasons to think the method might extend to hybrid models. In particular, given the nature of the retrieval method used in Tranception and TranceptEVE, it seems very likely that the two approaches are complementary and that further performance gains will come from combining these approaches. The situation with PoET, ProSST and SaProt is less clear though and warrants investigation.

Thank you for your thoughtful comments and for drawing the connection between Likelihood-Fitness Bridging (LFB) and test-time augmentation (TTA). We agree there is an interesting parallel, though at the same time we see an important distinction. TTA is typically used in supervised learning settings to reduce variance in predictive outputs due to model noise, whereas in contrast, LFB operates in an unsupervised setting, where our goal is to extract an estimate of biological fitness from the sequence likelihoods of language models. The likelihood of the model is not noisy necessarily, but rather reflects both fitness and other factors, including biologically meaningful structure such as phylogeny. The motivation for LFB is therefore not removal of generic prediction noise, but rather the extraction of a fitness-related component from the model’s likelihood outputs. This is why we chose to describe it as a modified fitness inference approach. In practice, in this work, we model these likelihoods as noisy estimates of fitness, so we recast the problem in a form where averaging predictions, just as in TTA, appears to be well motivated. We will revise the manuscript to make the connection more explicit.

With the OU model example, we aimed to show how LFB could reduce the effects of drift or phylogenetic non-fitness signals on our fitness estimates. Our objective was to lay out a simple model of evolution to gain intuition and therefore, for clarity, we decided to leave out correlations among the sequences, as they do not qualitatively alter the conclusions. But we agree with you that the lack of phylogenetic correlation structure among the related sequences makes this example less compelling. So we propose to include them in the example, using the same Ornstein-Uhlenbeck tree model as in Weinstein et al. 2022. In the text (L140, and L559 / Eq25), we currently have,

$

\text{Var} (\sigma_{\rm LFB}) = \frac{4\delta^2}{n^2} \sum_i \text{Var}(\varepsilon_i) = \frac{2\delta^2 \sigma^2}{\alpha n}\

$

and by adding in the correlations arising due to the phylogenetic structure in the OU tree process we have

$

\text{Var} (\sigma_{\rm LFB}) &= \frac{4\delta^2}{n^2} \Bigl( \sum_i \text{Var}(\varepsilon_i) + \sum_{i\neq j}\rm{cov}(\varepsilon_i, \varepsilon_j) \Bigr) \newline &= \frac{2\delta^2 \sigma^2}{\alpha} \Bigl( \frac{1}{n} + \frac{1}{n^2} \sum_{i\neq j}\exp(-\alpha \space t_{i,j}) \Bigr) \newline &< \frac{2\delta^2 \sigma^2}{\alpha} = \text{Var}(\sigma_{\rm LL}),

$

where $t_{i,j}$ is the distance between sequences $i,j,$ on the phylogenetic tree. The reduction in variance is bounded by the contribution of the correlations, $1/n^2 \sum_{i\neq j} \exp(-\alpha \\ t_{i,j})$. In practice, we choose sequences which provide less correlated predictions, in other words, we choose well spread out representatives across the tree by using the clustered/redundancy-reduced Uniref50 database. We also saw that including sequences over larger hamming distances from the reference sequence improved performance (Figure F2), although there appears to be a tradeoff between including distant less correlated sequences and not including sequences which are too distant, possibly within different local fitness maxima (the breakdown of the assumption the sequences are drawn from the OU tree process with one maximum). We hope that in this way the connection with phylogeny is clearer in the example. As you suggest, this example doesn’t rule out other explanations but rather presents how LFB would mitigate the effects of genetic drift on fitness prediction in this scenario.

On the shift in scores on Figure 2b:

We show similar distributions in Figure F6, with masked-marginal scoring on the top row and unmasked-marginal scoring on the bottom row (as included in Figure 2b). Although there is a systematic improvement in the separation between benign and pathogenic variants for LFB with the masked-marginal scoring (improved AUC), the overall shift in distributions is not present. This shift appears to be related to the unmasked-marginal scoring – scores are lower as the likelihoods of non-reference amino acids are lower when the reference sequence amino acids are not masked during scoring. The usefulness of unmasked logits has been explained in Gordon et al. 2024, and we also provide an analysis using masked-marginal scoring in Figure F3. We will clarify this point in the manuscript.

On "The augmented set includes sequences which do not contain the wildtype allele":

As specified in L 117 and Alg. 1, as well as in Figure 1, the set of sequences used in LFB contain the reference allele of the query sequence inserted at the position studied. We will draw more attention to it around L 117.

We are very grateful for your thorough review and interesting questions on the work. We address your questions and comments below:

Regarding not addressing sampling bias:

While we haven’t explored the sampling bias theoretically, we believe that the strategy of averaging over sequences under the same selective pressures should in principle remove multiple sources of noise on the fitness predictions – both the effects of phylogeny as we explore more systematically, but also other effects arising from over or under representation of certain sequences in the database. Our subsampling analysis Fig.4a seems to point in this direction, in the sense that even in low-sample regimes the approach delivers performance improvements.To what extent exactly LFB handles the effects of sampling bias is an interesting question for future work.

Regarding computational times of including MSAs and comparison to finetuning:

While LFB does require retrieving homologous sequences via MMseqs2, this step is relatively efficient: a single search against UniRef50 takes approximately 20 seconds on 20 CPU cores and can be reduced to ~1 second using GPU-accelerated MMseqs2 (Kallenborn et al., 2024). This cost is incurred once per input sequence and is trivially parallelizable across a large dataset. Comparing this to the cost of running a protein language model – for a typical protein of length 700, a single forward pass takes approximately 1.0 seconds with ESM-15B and 0.03 seconds with ESM-650M. Consequently, scoring substitutions across 10 homologs can range from 0.3 seconds (e.g., one-site scoring with ESM-650M: 0.03s × 10) to up to 133,000 seconds in the worst-case scenario (full mutagenesis of a 700-residue protein using ESM-15B: 700 × 19 substitutions × 10 homologs × 1s).

If we were to estimate the computational cost of a fine-tuned model trained on sequences similar to the query, a comparable MMseqs2-style search would still be required to construct the training set, in addition to the cost of training the model and performing inference. The precise cost would depend on the training configuration and hardware, and thus a direct comparison would require additional assumptions.

We note, however, that fine-tuning strategies involve limitations beyond computational expense. As discussed in Gordon et al. (2024), they require careful design to prevent overfitting, including curated training data, tuning schedules, and model selection. In contrast, LFB alters only the inference procedure and keeps the underlying model fixed, avoiding these risks while remaining broadly applicable and computationally lightweight.

Question 1.

Regarding the definition of “related sequences”: The definition and composition of “related sequences” in the LFB framework depend on the underlying model type (protein language model vs genomic language model), as discussed in s3.2, and appendix D1. This is a reflection of both the distinct practical challenges of building alignments for proteins vs genomes, as well as what data is currently available for doing so.

For pLMs, we construct multiple sequence alignments (MSAs) using MMseqs2 searches against UniRef50. We define the sequence neighborhood by selecting homologous sequences with at least 30% sequence identity to the reference. This threshold was chosen based on an empirical analysis of LFB’s sensitivity to neighborhood size, as shown in Figure F2. Furthermore, we demonstrate in Figure 4a that LFB is robust to subsampling: selecting as few as ~10 random homologs is sufficient to recover nearly the full performance gain provided by the complete sequence neighborhood. Motivated by this, for gLMs, we randomly chose 9 species to include from the zoonomia 447-way alignment (in addition to the human sequence for each gene).

Regarding sensitivity to composition and size: Overall, we find minimal sensitivity to the composition of the neighbourhood (as seen by the subsampling experiment Figure 4a), and as mentioned, we studied an overall sensitivity to the size of the neighborhood in Figure F2. That said, we are very interested in the fact that the ideal neighbourhood for each individual gene is not always at 30% identity, and see this fact as an exciting avenue for improvement of LFB in future.

Regarding sequences with poorer homology: We studied the sensitivity of LFB to the size of the MSA in Figure F10, and found minimal sensitivity, but we agree it is important to clarify how LFB behaves when homologous information is absent. LFB is designed to fall back to the base pretrained language model (pLM) predictions when no homologous sequences are available. In such cases, no averaging is applied, and the output corresponds to the standard pLM marginal scores. While LFB does not improve upon the base model in the absence of homologs, it does not degrade performance either.

The challenge of poorly aligned regions is a great point. We would be interested to see whether structure-based alignment techniques could improve LFB in these cases, though in the hardest to align cases such as disordered regions they would unlikely bring an advantage.

Question 2.

In the case where no mutation site aligns to the reference, i.e. where there is a gap in the alignment, we took the score for this sequence in this position to be neutral (i.e. zero), as we describe in appendix D3.

Question 3.

This is a great point of discussion we would like to include in the text. In contrast to the protein alignments we only consider sequences across mammals for the gLM. This could limit the benefits of LFB as we saw when filtering the protein alignments to neighbourhoods too local, Figure F2. It is exciting that the data needed to explore this idea in more detail will be available in the near future through the Earth Biogenome Project.

Question 4.

This is an interesting point. Rather than reweighting, we have focused on strategies to identify the relevant sequence neighborhood for LFB; we have approached it from the perspective of filtering based on distance to reference sequence (or identity, see Figure F2) and model perplexity (Figure 4b). In both cases we see that filtering improves performance, suggesting that reweighting schemes might also be a fruitful approach (though random subsampling shows low sensitivity to the actual composition of the averaging within such sequence neighborhood, Figure 4a).

Question 5.

We will make the code publicly available in a clean and reusable format upon conclusion of the review process, as we are not allowed to update github repos during review, and we are keen for the method to be adopted. For now we have provided code for pier review in the Supplementary Materials.

Thank you for your thoughtful feedback and for engaging with the paper, although the topic lies outside your main area of expertise. Ensuring the work is accessible and well-motivated to researchers from adjacent fields is an important goal for us.

(1) Motivation and the fitness-likelihood gap: We believe the manuscript already lays out both the conceptual and empirical rationale, though we agree this can be explained more clearly. Specifically, in the introduction, we describe how likelihoods from generative sequence models are now used across fields to estimate fitness (proxy for biological viability/quantitative measure of how well a protein variant is expected to function or be tolerated in a biological system), and outline why this assumption may however not always hold (due to phylogenetic structure and sampling artifacts, L30). We then discuss a significant piece of empirical evidence of a gap between likelihood and fitness – as the field fits better density estimators on datasets of biological sequence (obtaining better sequence reconstructions on validation set sequences), they do not perform any better, and sometimes indeed worse, on fitness effect prediction tasks (L36). Examples of this can be seen in Figure 2, 3 and F1 as well as in Table 1. This phenomenon has been noted across multiple studies (e.g., Nijkamp et al 2023, Gordon et al. 2024, Hou et al. 2025), and motivates the need for a method like LFB. This directly presents a problem in practical applications such as predicting disease causing variants (Figure 2, Sec. 4.2).

Section 2.3 is then devoted to exposing this gap between fitness and likelihood which our approach aims to mitigate. We provide further evidence that likelihood reflects not only fitness through references which deal with this issue explicitly, in particular Steinhardt et al. 2024 cover species compositional bias to likelihoods, Weinstein et al. 2022 cover the gap between fitness and likelihood introduced by phylogenetic structure, and Gordon et al. 2024 and Hou et al. 2025 cover the worsening performance of bigger pLMs on fitness estimation.

(2) Performance of the 15B model in Table 1: In Table 1, the relatively poor performance of the 15B model is not an anomaly, but rather a known and pressing problem for the field, as discussed above – larger models are not necessarily better for fitness prediction, despite their improved likelihood estimation. This counterintuitive trend is a central motivation for our work, and one that LFB is designed to address.

(3) Code availability: Reproducibility and adoption of the method is very important for us. The current implementation supporting our experiments is included in the Supplementary Materials for peer review. We are not allowed to update the repository during the review window, but will make the code publicly available in a clean and reusable format upon conclusion of the review process.