A mechanistically interpretable neural network for regulatory genomics

Summary of rebuttal

The comparisons and suggestions cited in the review cover several different fields of research, which are different from our paper. We have clarified below what our field of interest is, and what the appropriate baselines are (and in our current manuscript, we already compare to these baselines). To prevent future confusion, we have further clarified the field of interest in the manuscript.

Clarification on the field of interest

ARGMINN is a sequence-to-function model (i.e. the input is a DNA sequence, and the output is a genomics readout, such as ChIP-seq) for de novo motif discovery. As described in our Related Works (Section 2), there are several studies in the field of de novo motif discovery from neural networks [1, 2, 3, 4, 5]. In this domain, after training a sequence-to-function model, the model is then interpreted to reveal the sequence features (i.e. motifs) which drive the particular function of interest.

Importantly, the goal of ARGMINN (and related works) is not predictive performance; after all, the experiment used for training (e.g. ChIP-seq) is already genome-wide. Instead, the goal of ARGMINN is interpretability, specifically revealing sequence motifs from the model after training. This is why the setup for models such as ARGMINN uses only DNA sequence as the input, and does not rely on known databases of motifs (as that would defeat the purpose of de novo motif discovery).

The main two classes of methods which exist today for extracting motifs from trained neural networks are: 1) interpreting the first-layer convolutional filters as motifs; and 2) computing importance scores on the input sequence and clustering high-importance regions into motifs.

We compared our approach (ARGMINN) to both of these methods (and more) in our work. In each of these cases, including using importance scores, we found better results using ARGMINN (Figure 2, Supplementary Figures S1 - S5, Supplementary Table S1).

Comparisons to existing classifiers and attribution methods

In this review, seven papers were suggested as points of comparison, but they generally are not applicable to this field or this problem of de novo motif discovery. Citation 2 and 7 are not sequence-to-function models, as they rely on additional inputs such as chromatin accessibility, gene expression values (which estimate the free concentration of a transcription factor), known motifs, and known transcription-factor functions. As such, these models would generally not be used for motif discovery, as the decisions made by these models would include many features outside of sequence motifs, thus rendering them uninterpretable/unsuitable for this task. Additionally, although our focus is not predictive performance, these models would also not be fair comparisons for predictive performance due to the extra inputs and information. Citation 3 is a work which predicts promoter - enhancer interactions, and citation 8 predicts transcription-factor properties from the protein sequence (not DNA sequence). These two works are not in the same field as ARGMINN, and thus not comparable. Citation 4 predicts the effects of variants/changes in sequences for individual binding sites, where the motif is already given. It is also not in the same field as ARGMINN, and not comparable. Citation 5 and 6 are sequence-to-function models like ARGMINN, but are solving a different goal. These two works attempt to maximize predictive performance (predicting ChIP-seq binding) from the sequence. Both of these works use sequence embeddings to improve performance, but because of this, they are much less interpretable, and generally would not be used to perform motif discovery.

In our work, we opted to compare ARGMINN to the other relevant methods in this field. First, we compared ARGMINN to another recently published mechanistically interpretable architecture, ExplaiNN. Additionally, we compared ARGMINN to a typical CNN architecture which would be used to perform motif discovery (similar in construction to the suggested citations 5 and 6, but with an architecture which is far more amenable to motif discovery). Furthermore, we compared ARGMINN to motif discovery by using importance scores. We showed results using DeepLIFTShap importance scores, which is the algorithm suggested by other experts in the field [1]. We also demonstrated that other attribution algorithms do not visibly perform better at revealing motifs, including integrated gradients and in silico mutagenesis (Supplementary Figure S1).

Incorporation of personal genomes

Although this is beyond the scope of this work, there is no technical reason why personal genomes cannot be incorporated into ARGMINN (or any sequence-to-function model). However, such data is quite rare; most (publicly available) genomics experiments are in standard cell lines, and not in personal genomes. Hence, it is standard practice to use the reference genome for training [2, 3, 4, 5], and there are very few works (if any) which focus on sequence-to-function on personal genomes.

Reliance on motif databases

The problem/field of interest in our study is de novo motif discovery from data (e.g. ChIP-seq), not the prediction of binding from a database of known motifs. In many ways, it is methods like ARGMINN which populate these databases of motifs in the first place.

It is also worth noting that motifs currently in databases such as JASPAR are not necessarily accurate for real-world tasks. Most of these motifs are generated using in vitro methods (e.g. HT-SELEX), which may not provide an accurate picture of in vivo motifs and binding (e.g. those from ChIP-seq). They are still highly useful for distinguishing and labeling in vivo motifs (and we rely on these database motifs as a proxy for ground truth in our work), but they are biased in their own way.

Additionally, one of the benefits of ARGMINN is the ability to learn syntax between motifs (not just motif discovery) (Section 4.2), and this syntax is dataset-dependent, and is not found in any database currently.

Comparison of predictive performance

While the primary goal of ARGMINN is interpretability, we recognize that maintaining reasonable predictive performance is essential, as interpretability relies on a foundation of adequate performance.

Hence, we show a comparison of predictive performance of ARGMINN to other methods in Supplementary Table S5. As noted in our discussion (Section 6), it is generally well known that interpretable models tend to suffer slightly in predictive performance, as they tend to base decisions on human-interpretable concepts rather than spurious signals, which can still be informative (but are not useful for the downstream task of understanding motifs). Our results here fall in line with these expectations, although ARGMINN still outperformed the relevant baselines in many cases.

We did not compare the predictive performance of ARGMINN to non-deep-learning methods, as it is generally known that deep-learning (specifically CNN-based) methods are far better than others for sequence-to-function prediction. Indeed, this was one of the biggest and most obvious trends which became clear from the ENCODE-DREAM challenge.

Having clarified the differences between ARGMINN’s task compared to other DNA-related neural networks, as well as having demonstrated the suitability of the benchmarks used in this study for the task at hand, we respectfully request a re-evaluation of the review. We remain available to address any further questions or provide additional clarifications as needed. Thank you.

References

[1] Shrikumar, et. al., Learning important features through propagating activation differences, ICML, 2017

[2] Avsec, et. al., Base-resolution models of transcription-factor binding reveal soft motif syntax, Nature Genetics, 2021.

[3] Nair, et. al, Integrating regulatory DNA sequence and gene expression to predict genome-wide chromatin accessibility across cellular contexts, Bioinformatics, 2019

[4] Quang & Xie, DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences, Nucleic Acids Research, 2016

[5] Zhou & Troyanskaya, Predicting effects of noncoding variants with deep learning–based sequence model, Nature Methods, 2015

Why the suggested citations in the original review are not relevant to this work or field

Our apologies, we are confused by this point in the review: “So yes, a direct comparison at the moment does not make sense and is not what I asked for. Rather, how would you include such knowledge and then compare to those methods.”

We agree that comparing to these works does not make sense (because the goal of these works is drastically different from ARGMINN). As such, we are not sure how to think about comparing to these methods meaningfully.

We’d also like to note that Citations 3 and 4 use known motifs as an input, whereas ARGMINN discovers motifs as a goal, so they are not comparable, even hypothetically.

Additionally, Citation 3 actually does not use promoter - enhancer interactions to improve TF binding prediction. Instead, it’s the other way around: it uses known motifs and putative TF binding sites in order to improve the prediction of promoter - enhancer interactions. The paper does, however, interpret these interactions with respect to the input binding sites, which we realize can be confusing.

Why individual genomes/variants are not included in this work

We would like to clarify that individual genomes and variants are effectively synonymous: an individual’s genome is generally stored as a set of variants relative to a reference genome to save space.

As noted in our initial response, in order to incorporate individual genomes/variants, we would need genome-regulatory measurements of specific cell types and specific TFs (e.g. ChIP-seq in lung cells) for many unique individuals. Although measurements on individuals is common in clinical applications (e.g., cancer studies), that is not really relevant to the field of genome regulation, where such data is unfortunately not readily available.

Instead, it is effectively universal that works in our field (AI-driven motif discovery) use the reference genome in many cell lines (which can be seen in the ENCODE project). Fortunately, there is strong evidence that sequence variation across different binding sites in the single reference is sufficient for models to learn unique binding modes, without relying on individual genomes/variants [4].

We sincerely appreciate the time taken by the reviewer to discuss. We hope our responses have provided valuable insights into the field and the context of our work. While we recognize that the critiques offered may not have been applicable to our research or the broader field, we hope that the clarifications across our responses have been sufficiently informative to raise the score from a 3.

References

[1] Novakovsky, et. al., Obtaining genetics insights from deep learning via explainable artificial intelligence, Nature Reviews Genetics, 2022

[2] Shrikumar, et. al., Learning important features through propagating activation differences, ICML, 2017

[3] Lundberg and Lee, A Unified Approach to Interpreting Model Predictions, NeurIPS, 2017

[4] Avsec, et. al., Base-resolution models of transcription-factor binding reveal soft motif syntax, Nature Genetics, 2021

Summary of rebuttal

The comparisons suggested in the review are for a different (albeit related) field of research. We have clarified below what the field of interest is, and what the appropriate baselines are (and in our current manuscript, we already compare to these baselines). To prevent future confusion, we have further clarified the field of interest in the manuscript.

Clarification on the field of interest

The problem that ARGMINN is solving is the discovery of sequence motifs from a sequence-to-function model (i.e. the input is a DNA sequence, and the output is a functional genomics readout, such as ChIP-seq). This is very different from DNA-sequence large-language models like DNABERT and HyenaDNA (even though both models use DNA sequences as an input).

Sequence-to-function models like ARGMINN are trained to predict a functional label (e.g. protein binding) from DNA sequence. In order to perform this mapping, these models implicitly learn the short sequence features (i.e. motifs) which impart function, as well as the syntax/grammar between them. These models are then subsequently interpreted to reveal the sequence motifs which drive function. Examples of such models include ChromDragoNN, DeepSea, DanQ, BPNet, etc., which are all slight variations on a standard CNN architecture [1, 2, 3, 4].

In contrast, models like DNABERT and HyenaDNA are solving a very different task. Instead of learning a sequence-to-function map, they are unsupervised models on DNA (they do not include any functional data). Instead, they learn patterns between parts of DNA without any functional information. As such, these LLMs are generally not used to extract motifs. Hence, they are not an appropriate comparison.

Comparisons with relevant baselines, and validation methodology

In the current literature, extracting motifs from these sequence-to-function models is largely done in one of two ways. In both approaches, we start by training a convolution-based sequence-to-function model (e.g. an architecture like ChromDragoNN). Then, we apply one of:

1. Once the model is trained, motifs are extracted from the first-layer convolutional filters [3, 4, 5].
2. Once the model is trained, importance scores are computed for the inputs (e.g. integrated gradients), and then segmented and clustered into motifs [1, 6].

In our paper, we show that both of these approaches have limited success on typical architectures. Instead, we proposed ARGMINN, a new mechanistically interpretable sequence-to-function architecture which would allow for motifs to be cleanly and successfully extracted from convolutional filters. We compared motifs discovered by ARGMINN to those discovered using these two methods, as these are the most appropriate benchmarks. We also compared our method to another model (ExplaiNN), which is an additional mechanistically interpretable architecture (of more limited expressivity and interpretability) recently published for motif discovery.

We quantified the quality of motifs and motif instances in several different ways, including:

• Number of unique, relevant motifs discovered by each method
• Similarity of discovered motifs to ground-truth motifs
• Amount of redundancy in discovered motifs
• Number of extraneous (”garbage”) motifs discovered
• Accuracy of motif instances in DNA sequences compared to ground truth

We found that in general, ARGMINN’s interpretations were better than those discovered by the other benchmark methods, in all of these categories (Figure 2 - 3, Supplementary Figures S2 - S6, Supplementary Tables S1 - S4).

To the reviewer,

We have clarified the main objectives and contributions of our work both in the manuscript and in our response. With these core aspects now clarified, and as the end of the discussion period is approaching, we would like to ask if there are any remaining concerns upon re-evaluation of our work. We are happy to answer any further questions and confusions. Thank you.

-Authors

Relevance to ICLR

We believe that such ML research for biology certainly does have a place in conferences such as ICLR. Indeed, this exact problem of extracting biological sequence motifs from DNA models is the subject of DeepLIFT (published in ICML), an algorithm which uses importance scores to reveal putative DNA motifs from a trained model (and we compare to this algorithm in our work) [1]. It was also very recently that HyenaDNA (a similar work which developed an architecture for a different task on DNA sequences) earned a spotlight at NeurIPS. [2]. ICLR this year also featured multiple papers on DNA-sequence models and their biological applications [3, 4].

Although the motivating problem and application are biological in nature, our work is still one of the early few which explores mechanistic interpretability at such a deep level [5]. In our work, we designed a novel architecture using fundamental ML techniques, but tailored these techniques to the domain at hand. In many ways, these domain-informed constraints are required to construct a mechanistically interpretable architecture which is both interpretable and sufficiently expressive [5]. In our paper, we additionally gave mathematical/theoretical justification for our architecture and these design decisions, as well as empirical results supporting their benefits.

Biophysical vs mechanistically interpretable

To clarify, ARGMINN is not meant to be a biophysical model. That is, although parts of the architecture (e.g. the decision to learn motifs as convolutional filters, which is ubiquitous in this area) are inspired by biophysics, the model itself is not intended to encode biophysical processes (which would take far too much processing power to represent accurately, particularly the interactions between motifs).

There is a need to learn motifs and their interactions in a data-driven manner, but currently, it is difficult to extract/interpret the learned motifs and their interactions from trained models. In our work, we take the approach of constructing a mechanistically interpretable architecture, where the learned decisions are based on human-interpretable motifs. The techniques we apply are intentionally those which help a model be more mechanistically interpretable, thereby revealing the learned decision rules [5].

Indeed, we show that ARGMINN achieves far better results in motif discovery with its mechanistically interpretable architecture compared to the state-of-the-art methods today, even though it is not a biophysical model.

ARGMINN vs CBMs

CBMs take an input (e.g. an image), and map it to a set of higher-order, human-interpretable concepts (e.g. for a cat image, these features may be binary labels of “whisker”, “pointy ear”, “stripes”, “tail”). These concepts are then passed to another classifier which produces a final output by implicitly learning the logic between these features (e.g. “whisker” + “pointy ear” + “tail” = cat).

ARGMINN can be viewed as a kind of CBM, where the input (a DNA sequence) is mapped to a set of higher-order, human-interpretable concepts (motifs). The attention layers in ARGMINN then learn the final output by learning the logic between these motifs.

ARGMINN differs from typical CBMs in two main ways:

• In a typical CBM, the only part of the model which is interpretable is the concepts (e.g. “whisker”). But in ARGMINN, the attention layers are designed such that the combination of the motifs/concepts is also interpretable. That is, we can easily identify which concepts are being combined to produce the final output. This is not typically achievable (easily) in a standard CBM.
• CBMs typically require labeled concepts for each input (e.g. to train a CBM on images, we need annotations for which images have whiskers or stripes). ARGMINN, however, does not require concept/motif labels at all. Owing to the filter regularization we developed, it learns all of the relevant motifs (i.e., “concepts”) from scratch, and separates them out into individual channels.

ARGMINN vs ExplaiNN

ARGMINN and ExplaiNN both attempt to solve a similar problem: train a model predicting biological function from DNA sequences, and subsequently interpret sequence motifs from the trained model.

ExplaiNN’s architecture learns a set of single-filter CNN modules, where each CNN module learns one motif (in its one filter), and produces a scalar value for an input DNA sequence. ExplaiNN’s final output is a learned linear combination of these scalar values.

In contrast with ARGMINN, ExplaiNN does not have any regularization to prevent multiple CNN modules from learning the same motif; information is still distributed across the network which makes it less interpretable (Figure 2). Additionally, ExplaiNN is limited by a linear combination (instead of a multi-layer attention mechanism), and so ExplaiNN is also limited in its expressivity (Corollary 1.1).

Selection of chromosome splits

Using chromosome splits to define training/validation/test sets for regulatory genomics is standard practice in this area, as it completely prevents contamination of the same sequence (or part of the same sequence) from occurring in both training and testing [1, 2, 3, 4]. Chromosome splits also help prevent weaker contamination arising from genetic duplication events. That is, much of the genome arises from sections of DNA being duplicated across the same chromosome, so chromosome splits also prevent different sequences (but with a recently shared history) from occurring in both training and testing.

These chromosome splits were directly adopted from prior literature in a similar space, and they were selected to obtain roughly a 80/10/10 split of training/validation/testing (the human chromosomes have a wide variety of sizes, so the selected chromosomes in each set are not consecutive). Prior work in ML on regulatory DNA has also shown that different folds of chromosomes yield similar results (the regulatory signals across the genome are fairly uniform by chromosome, likely owing to their large sizes) [1, 2].

Computational resources used

We trained all of our models on a single Nvidia A100 GPU. As for training times, here we report the training time for our SPI1 dataset (others are quite similar). Note that the increased relative training time of ARGMINN largely stems from the computation of the filter-overlap regularization loss.

We have added this table to the supplement of our manuscript.

References

[1] Avsec, et. al., Base-resolution models of transcription-factor binding reveal soft motif syntax, Nature Genetics, 2021.

[2] Nair, et. al, Integrating regulatory DNA sequence and gene expression to predict genome-wide chromatin accessibility across cellular contexts, Bioinformatics, 2019

[3] Quang & Xie, DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences, Nucleic Acids Research, 2016

[4] Zhou & Troyanskaya, Predicting effects of noncoding variants with deep learning–based sequence model, Nature Methods, 2015

Numbers for evaluated tasks

Figure 2 shows both comprehensive and examples of quantitative results comparing ARGMINN to several benchmark methods. The same data but in tabular form can be found in Supplementary Tables S1 - S4.

Ablation studies

We showcased an ablation study (Section 6, L522) in which we applied our filter regularization to a standard CNN. We showed that with our filter regularization, we were able to make the standard CNN far more interpretable and also reveal high-quality motifs (Supplementary Figure S10).

This ablation shows the marginal effects of the filter-overlap regularization, the first of our two architectural novelties. Note that it would not be meaningful to ablate the second novelty (i.e. the modified attention layers), as it is only useful for interpretability when the input activations are separated into individual motifs (which is what the filter regularization does).

Experimental details

The details for the experimental settings can be found in the supplementary methods. All values of hyperparameters, including $n_{f}$ and $n_{L}$, are in Supplementary Table S7.

Species

All of our experiments are on human genomes and human motifs.

Having clarified ARGMINN’s task and its novelties and contributions, as well as having pointed out the experimental methodologies, we respectfully request a re-evaluation of the review. We remain available to address any further questions or provide additional clarifications as needed. Thank you.

References

[1] Avsec, et. al., Base-resolution models of transcription-factor binding reveal soft motif syntax, Nature Genetics, 2021.

[2] Nair, et. al, Integrating regulatory DNA sequence and gene expression to predict genome-wide chromatin accessibility across cellular contexts, Bioinformatics, 2019

[3] Quang & Xie, DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences, Nucleic Acids Research, 2016

[4] Zhou & Troyanskaya, Predicting effects of noncoding variants with deep learning–based sequence model, Nature Methods, 2015

[5] Kelley, et. al., Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks, Genome Research, 2016

[6] Shrikumar, et. al, Technical Note on Transcription Factor Motif Discovery from Importance Scores (TF-MoDISco) version 0.5.6.5, arXiv, 2018

Summary of rebuttal

We have clarified the novelty of our method compared to previous works below. All other requests in the review (e.g. specific numbers, ablation studies, experimental settings, etc.) are in the supplement of the manuscript (and referenced from the main text). To prevent future confusion, we have further clarified the background of the field in the manuscript.

Differences of proposed architecture to previous methods

ARGMINN is a sequence-to-function model: the input is a DNA sequence, and the output is a functional genomics readout, such as protein binding. These sequence-to-function models (like ARGMINN) are trained to predict this functional readout (e.g. protein binding) from the input DNA sequence. In order to perform this mapping, these models must implicitly learn the short sequence features (i.e. motifs) which define function, as well as the syntax/grammar between them. The chief goal of these models is not predictive performance, but subsequent interpretation. After training, these models are then interpreted to reveal the sequence motifs which drive function. Examples of such models include ChromDragoNN, DeepSea, DanQ, BPNet, etc., which are all slight variations on a standard CNN architecture [1, 2, 3, 4].

In the current literature, extracting motifs from these sequence-to-function models is largely done in one of two ways. In both approaches, we start by training a convolution-based sequence-to-function model (e.g. an architecture like ChromDragoNN). Then, we apply one of:

1. Once the model is trained, motifs are extracted from the first-layer convolutional filters [3, 4, 5].
2. Once the model is trained, importance scores are computed for the inputs (e.g. integrated gradients), and then segmented and clustered into motifs [1, 6].

In our paper, we show that both of these approaches have limited success on typical architectures. Instead, we proposed ARGMINN, a new mechanistically interpretable sequence-to-function architecture which would allow for motifs to be cleanly and successfully extracted from convolutional filters. We compared motifs discovered by ARGMINN to those discovered using these two methods, as these are the most appropriate benchmarks. We also compared our method to another model (ExplaiNN), which is an additional mechanistically interpretable architecture (of more limited expressivity and interpretability) recently published for motif discovery.

In order to allow for motifs to be successfully extracted from convolutional filters, our mechanistically interpretable architecture introduces two main novelties: the convolutional layer with a filter-overlap regularizer (Section 3.1), and an attention mechanism which explicitly learns motif interactions (Section 3.2).

In the current literature, the standard architectures (e.g. ChromDragoNN, DanQ, etc.) consist of one or more convolutional layers (without any special regularization), followed by additional layers to learn interactions (e.g. fully-connected layers, dilated convolutional layers, standard transformers, etc.) [1, 2, 3, 4].

Compared with previous methods, our approach introduces a filter-overlap regularizer which makes the first-layer convolutional layer interpretable and reveals motifs directly. Additionally, our custom attention mechanism contrasts with existing works, as it allows for long-range interactions while remaining interpretable, which allows it to reveal motif syntax (an ability which previous architectures did not have at all).

In our work, we also included both theoretical and experimental results showing the improved interpretability and expressivity of ARGMINN relative to ExplaiNN.