SHAP zero Explains Biological Sequence Models with Near-zero Marginal Cost for Future Queries

We thank reviewer GnFL for their comments. We address the following questions:

The notation for Shapley values is appreciated, but probably better suited to a Methods or Preliminaries section than almost immediately in the Introduction. The main text would benefit from pseudocode showing the SHAP zero algorithm.

We will provide a more thorough explanation of Shapley values and add a SHAP zero algorithm box to the main text.

Overall, the connections between the sparse Fourier transform, Mobius transform, and SHAP values are mathematically clear, but not intuitively clear.

Briefly, the connection between SHAP values, the Möbius transform, and the sparse Fourier transform lies in how we represent and recover feature interactions in complex models. SHAP values attribute a model’s prediction to individual features and their combinations by decomposing the function into additive contributions over subsets. This decomposition aligns with the Möbius transform, a mathematical framework that captures how a function defined over feature subsets can be expressed in terms of unique contributions from each subset. However, directly computing this is intractable for high-dimensional models. The sparse Fourier transform offers a computational shortcut by exploiting the empirical sparsity of interactions in biological systems—only a few feature combinations significantly affect the outcome.

We have made this connection clear throughout the paper, specifically around lines 52, 153, and 182. To address the reviewer's concern, we will add additional explanations to make the connection more clear.

The manuscript repeatedly refers to a model “sketch”, and specifies that the Fourier transform of a black-box model $f$ is the “global sketch” of that model. What is the exact definition and meaning of “sketch” in this context?

We formally defined the sketch as the $s$-sparse approximation of the Fourier transform of the model, as detailed in Equation (3). We refer to this as a "global" sketch because this Fourier representation describes the entire function's behavior, independent of any single input query.

We clarify that the term "sketch" is standard terminology in theoretical computer science and machine learning. The core idea of a sketch is to create a compressed summary of a model that captures the most relevant information. We are happy to make this more clear in the text.

What is the rationale for excluding gradient-based attribution methods, considering that all of the models $f$ in the work are differentiable neural networks?

We clarify that the focus of our work is on amortizing Shapley-based explanations in black-box sequence models. Gradient-based attribution methods, which rely on model internals, fall under white-box explainability methods. Regardless, we have addressed white-box explainability methods in the Discussion section, and additionally provided experiments in the Appendix on running DeepSHAP, another white-box method, on TIGER in Appendix D.

We emphasize that with the growth of large-scale AI sequence models with proprietary access, the choice of black-box explainability methods over white-box ones is more of a need than a luxury for biological discoveries.

We thank reviewer b3je for taking the time to review our paper. We address the following questions:

Could the authors clarify how this additional input from the MSA space was handled within the SHAP zero framework (for Tranception with retrieval)?

In our framework, the multiple sequence alignment (MSA) is treated as a fixed component of the overall black-box model, not as a separate input for each query.

Tranception with retrieval scores each mutant by taking a weighted average of two values: the log-likelihood from the main transformer model and the log-likelihood derived from the MSA at the specific mutant's position (see Fig. 3 and Equation (4) in [1]). The MSA-derived likelihood is based on the empirical amino acid distribution at that position.

When using SHAP zero, Tranception with retrieval is treated as a single, consistent black-box function. The MSA is initialized only once at the beginning and remains static for all subsequent mutant evaluations. The user does not need to re-initialize or change the MSA for every new mutant, ensuring our definition of the sequence-function model from the Introduction remains consistent. We will add a summary of this discussion in the Appendix.

The validation is focused on DNA and protein sequence models. Could the authors comment on the potential applicability of SHAP zero to other types of biological data that also have a combinatorial nature but a different structure, such as protein-protein interaction networks or models of RNA-protein binding?

Thank you for your question. SHAP zero can be applied to other combinatorial biological problems. For problems such as protein-protein binding, the application is seamless: we would frame the input as the combined sequence of interacting proteins and the output as the binding affinity. Assuming this mapping is sparse in Fourier (i.e., a small fraction of residue-residue interactions contribute to binding), SHAP zero will generalize. For problems such as RNA-protein binding, an interesting extension of SHAP zero would be to compute the Fourier transform over different alphabet sizes ($q$=4 and $q$=20) using generalized $q$-ary Fourier algorithms [2].

[1] Notin et al. “Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval” (2022)

[2] Tsui et al. “Efficient Algorithm for Sparse Fourier Transform of Generalized $q$-ary Functions” (2025)

We thank reviewer jhji for their constructive feedback. Below, we address specific questions:

how does your Fourier representation correlate with experiments compared to Tranception, linear and pairwise representations

To address this question, we performed new experiments comparing SHAP zero against Tranception and linear/pairwise baselines on the deep mutational scanning data from [1], following the procedure in Appendix E.6:

I think that perhaps a more interesting question than looking at known interacting positions would be to, given many positions, identify the ones that are epistatically interacting… Would this be computationally viable? (you would increase sequence length but would look at a sparser space)... How far in sequence length can you take this approach?

We agree that identifying epistatic positions at scale is an exciting future direction that SHAP zero is well-equipped to handle. The scalability of this approach depends both on the query speed and the relative sparsity of the model. On our single GPU setup, assuming a time budget of 10 days and that the relative sparsity of the model remains constant as we scale up $n$:

• We can recover interactions across approximately $n=30$ sites for larger, slower-to-query models like Tranception.
• For smaller, faster-to-query models like TIGER, the same budget would allow us to scale to $n=100$ sites.

As $n$ grows, we speculate that the relative sparsity level will likely decrease, enabling SHAP zero to analyze significantly more positions than we report here.

We additionally clarify that, in our current experimental setup, finding epistatic interactions given a subset of positions is an important and high impact problem [2-4], especially in the context of discovering proteins with novel biological motifs. Screening just 10 positions results in a combinatorial space of over 10 trillion sequences ($20^10 \sim 1 \times 10^13). Existing methods screen this landscape by restricting their search to small, local neighborhoods within the protein's 3D structure [2, 4] or over a small fraction of amino acids [3]. We emphasize that SHAP zero, unlike these methods, does not require such restrictive assumptions, which enables the recovery of epistatic interactions previously inaccessible at scale.

Have you considered using other sparse representations such as wavelets instead of a sparse Fourier basis?

We have not considered other representations such as wavelets. In the context of biological sequence-function models, Fourier is the only standard basis which defines interactions in sequences, consistent with the definition of epistasis in statistical genetics [5]. SHAP zero could, in principle, be generalized to different bases, which would be an interesting future direction.

Why the difference in total number of high-order interactions between [SHAP zero and SHAP-IQ in the protein experiments]?

Both SHAP zero and SHAP-IQ are approximating Faith-Shap interactions in Tranception. Given the computational budgets provided to both algorithms, we observe that SHAP zero identifies more biologically meaningful interactions. We attribute this success to SHAP zero’s underlying assumption of Fourier sparsity, a principle that previous studies suggest holds for biological fitness landscapes [6-8]. This assumption allows SHAP zero to structurally subsample the model to efficiently isolate key interactions. In contrast, SHAP-IQ's reliance on random feature subset sampling is less efficient for this problem and requires a larger computational budget to achieve comparable results.

Also, why is it the opposite in the DNA repair example, line 244?

Thank you for pointing out the inconsistency in the DNA repair example—that is a typo on our part, and we apologize for the error. The trend is the same as in our protein experiments. The corrected text should read: “interactions (constituting 85% and 3% of the total interactions in SHAP zero and SHAP-IQ, respectively)”

[1] Sarkisyan et al. "Local fitness landscape of the green fluorescent protein" (2016)

[2] Weinstein et al. “Designed active-site library reveals thousands of functional GFP variants” (2023)

[3] Wu et al. “Adaptation in protein fitness landscapes is facilitated by indirect paths” (2016)

[4] Starr et al. “Epistasis in protein evolution” (2016)

[5] Poelwijk et al. "The context-dependence of mutations: a linkage of formalisms" (2016)

[6] Aghazadeh et al. “CRISPRLand: Interpretable large-scale inference of DNA repair landscape based on a spectral approach” (2020)

[7] Brookes et al. "On the sparsity of fitness functions and implications for learning" (2022)

[8] Tsui et al. “On recovering higher-order interactions from protein language models” (2024)

We thank reviewer hhqE for their comments. Below, we address the following questions:

How does SHAP zero perform when asked to evaluate rare and highly deleterious mutations? Even if it doesn’t assign as highly-negative of values as a method like KernelSHAP, does it still correctly identify those mutations as being the most explanatory?

We clarify that Shapley estimation algorithms finds the most impactful mutations on a sequence, which includes both deleterious and beneficial mutations. While KernelSHAP tends to assign more negative values, the overall explanations SHAP zero returns is sufficient to identify rare and impactful mutations. SHAP zero’s explanations achieve a correlation of $\rho=0.97$ with KernelSHAP’s estimates, indicating that both methods agree on the relative importance of mutations. Examining the top mutations (Tables 8 and 9), SHAP zero identifies all the top positive mutations and 17 out of 20 of the top negative mutations found by KernelSHAP. The three negative interactions that were missed were all ranked in the bottom five of KernelSHAP's list. This demonstrates that while SHAP zero's negative value assignments are more conservative, it is still able to capture the most impactful mutations.

We will add a summary of this to the Discussion section.

How does SHAP zero perform in the context of a sequence landscape that would be computationally-intractable for methods like KernelSHAP and SHAP-IQ? Suggestions include models trained on MPRA datasets or even a synthetic dataset where all the feature contributions and higher-order interactions are known. Even though it would be impossible to compare to slower methods, it would add to the paper’s significance if the method was able to pull out known feature contributions/interactions while also identifying new ones.

SHAP zero will be able to recover meaningful interactions in landscapes that are intractable for methods like KernelSHAP and SHAP-IQ under the key assumption of Fourier sparsity. This means that only a small fraction of interactions are relevant to the model’s output. In the context of models trained on MPRA datasets, previous studies show that such models encode regulatory motifs which are known to be sparse in Fourier [1-2], suggesting that our method's core assumption holds in this domain.

it’s unclear why the correlations with SHAP-IQ values are relatively low… how trustworthy those conclusions [(SHAP zero on Tranception)] are… [with] linear or pairwise baselines.

We clarify that the correlation between SHAP zero and SHAP-IQ is $\rho=0.61$, which demonstrates that the interactions recovered are in agreement with one another to some degree. Our analysis of the top biological motifs from Tables 8 and 10 shows that the top interactions recovered by SHAP zero corresponds to known motifs that affect protein stability. Regardless, to better assess the trustworthiness of SHAP zero, we have additionally performed two new experiments. First, we took the top 5 positive and negative interactions identified by SHAP zero from Table 10 and explicitly verified that these interactions are epistatic in Tranception. Second, we compared the performance of SHAP zero’s recovering Fourier coefficients against Tranception and linear and pairwise baselines (see Reviewer jhji for the full Table). We observe that SHAP zero ($\rho=0.55$) is able to closely approximate the performance of Tranception ($\rho=0.57$), suggesting that the epistatic interactions SHAP zero captures are biologically meaningful.

We will add these new results to Section 4 and the Appendix and update Fig. 4 to better highlight the epistatic interactions discovered by SHAP zero.

Model evaluation cost… may be restrictive for some users

The one-time cost required for SHAP zero's initial sketch is relatively low compared to the cost of training modern AI models. SHAP zero is designed to use less computational power in the long run by reducing the amortized cost of future explanations to near-zero.

The model evaluation cost of SHAP zero is primarily determined by two factors: the query speed and the Fourier sparsity of the model. In the case of ML-guided protein design, it is common to screen for a pre-selected list of important sites [3-4], rather than over the entire sequence. When running SHAP zero for larger, slower-to-query models like Tranception, this helps keep the problem well-defined and computationally feasible. For analyses over a very large number of sites, an interesting extension would be to run multiple, smaller SHAP zero instances across different regions of the sequence, and combine the explanations. This approach would be more computationally efficient than a single, large run because each SHAP zero instance would only need to recover a smaller set of sparse Fourier coefficients.

We will additionally release a pip package with the camera-ready version. This package will automatically tune SHAP zero’s parameters based on a user-defined sample budget (following our computational budget selection in Appendix E.1), providing the best possible Shapley estimates for their available resources.

[1] Agarwal et al. “Massively parallel characterization of transcriptional regulatory elements” (2025)

[2] Dalla-Torre et al. “Nucleotide Transformer: building and evaluating robust foundation models for human genomics” (2024)

[3] Bian et al. “Optimizing enzyme thermostability by combining multiple mutations using protein language model” (2024)

[4] Lipsh-Sokolik et al. “Addressing epistasis in the design of protein function” (2024)