Designing Cell-Type-Specific Promoter Sequences Using Conservative Model-Based Optimization

References:

• Reddy, Aniketh Janardhan, et al. "Strategies for effectively modelling promoter-driven gene expression using transfer learning." bioRxiv (2024).
• Trabucco, Brandon, et al. "Conservative objective models for effective offline model-based optimization." International Conference on Machine Learning. PMLR, 2021.
• Jain, Moksh, et al. "Biological sequence design with gflownets." International Conference on Machine Learning. PMLR, 2022.
• Linder, Johannes, et al. "A generative neural network for maximizing fitness and diversity of synthetic DNA and protein sequences." Cell systems 11.1 (2020): 49-62.

We thank the reviewer for their comments and address their concerns below. First, we clarify that the novel contribution of our work is our cell-type-specific promoter design workflow. Although we use existing modeling strategies and MBO algorithms, we combine them in a novel way to tackle a difficult but important problem – an accepted axis of novelty at NeurIPS as per the guidelines. Unlike existing approaches, our elaborate workflow accounts for many practical considerations such as data-efficiency, avoiding adversarial sequences, and obtaining diverse sequences (PDF attached to general rebuttal illustrates it). And unlike most other offline MBO works that rely on computational evaluations, which are often unrepresentative of real-world experiments, we validate our workflow using expensive and time-consuming wet lab experiments. Next, we also explain the benefits of our workflow over GFlowNets, which are generally harder to train and tune, do not account for adversarial examples, and have not been validated using real-world experiments. Finally, we justify the limited number of benchmarks, as the cost of running wet lab experiments is extremely high and our budget did not allow us to benchmark more methods. We hope that the reviewer can reconsider their assessment based on our responses.

1. Our contributions: We would like to clarify that the specific models used in our work are not the main contribution. Instead, the focus of this paper is to propose a novel workflow for discovering cell-type-specific promoters using offline MBO that accounts for various practical considerations. Our workflow has the following advantages over existing promoter design workflows (detailed in the Related Work section):

   • Compared to traditional workflows that use heuristics and manual curation, our workflow is automated and generalizable and can be used to design promoters for even relatively understudied cell types.
   • We improve on existing MBO-based workflows in three main ways. First, we use the transfer learning strategies identified by Reddy et al. (2024) to train models in a data-efficient manner - existing workflows do not employ transfer learning and rely on large training datasets that are expensive to collect. Second, we use Trabucco et al. (2021)'s COMs framework to mitigate adversarial designs - existing workflows use simple optimization techniques (e.g. vanilla gradient ascent) that are more prone to adversarial designs. Finally, as experiments are expensive and a limited number of sequences can be tested, we propose a final sequence selection step to choose a small subset of diverse yet desirable sequences for testing from a large pool of candidate designs – this step is also missing in existing workflows.

   Although we use the modeling strategies identified by Reddy et al. (2024) and the COMs framework, we combine them to solve an important problem in a novel way – an accepted axis of novelty at NeurIPS as per the guidelines. We also add a novel sequence selection step to balance diversity, optimality, and uncertainty. This novel workflow has the potential to improve gene therapies and should be of interest to many ML researchers working on synthetic biology problems such as promoter design. Moreover, we validate the utility of our workflow by performing real-world wet lab experiments - most MBO methods are never validated using wet lab experiments. We show that our workflow is effective and outperforms both a traditional design method (motif tiling) and an offline-MBO-based method (deep exploration networks - DENs) that had previously been validated using wet lab experiments. These points support the novelty and significance of our work.

2. GFlowNets are harder to train, do not account for adversarial examples and have not been validated using real-world wet lab experiments: Our workflow improves on existing promoter design workflows in many ways, especially by accounting for practical considerations such as data-efficiency, adversarial designs, diversity, and uncertainty. The reviewer mentions that GFlowNets also try to optimize a given property while retaining diversity. However, since GFlowNets are generative models, they require specialized architectures that are significantly harder to tune and train vs. training a discriminative model as in our workflow. They also do not account for adversarial designs. Most importantly, GFlowNets have not been validated using real-world wet lab experiments to the best of our knowledge (Jain et al., 2022). DENs are also generative models that are aimed at producing diverse desirable designs and have been validated using real-world wet lab experiments (Linder et al., 2020), which is why we chose DENs as a comparator to benchmark our approach. Figure 3 shows that our workflow outperforms DENs.

3. High experimental costs limit our ability to benchmark more methods: Most existing MBO methods for biological sequence design do not validate their designs using wet lab experiments – instead, their evaluations are based on computational oracles whose predictions are not always reflective of real measurements. In our wet lab experiments, we show that our workflow is indeed capable of designing cell-type-specific promoters. While we benchmark against DENs and motif tiling, the reason for not benchmarking more design methods is the extremely high cost of performing these experiments. They are also time consuming, and took us ~6 months to complete. Since our budget limited the total number of sequences that could be tested, and since we require a significant number of designs from every method to thoroughly validate it, we chose to evaluate a representative set of methods. In particular, we chose motif tiling as representative of widely-used traditional methods, and we chose DENs as the only MBO algorithm that has been evaluated for biological sequence design using wet lab experiments. Our workflow outperforms both of these methods.

Regarding the reviewer's suggestions

We appreciate the reviewer’s suggestions for improving the presentation and readability of the paper, and we will incorporate them in the final version of the manuscript.

Regarding figure 3, although it shows a visible improvement in mean DE only for Jurkat cells when using gradient ascent vs. DENs, the difference in DE is significant also for K562 cells. Moreover, we see that gradient ascent produces many more sequences with high DE when compared to DENs, while also producing generally more diverse sequences (Table 1). Finally, as the reviewer noted, DENs are more computationally intensive and more difficult to tune vs. gradient ascent, since they require training separate generative models in addition to training the design models. We will add a more detailed discussion of these reasons for choosing gradient ascent over DENs in the final version.

References:

• Reddy, Aniketh Janardhan, et al. "Strategies for effectively modelling promoter-driven gene expression using transfer learning." bioRxiv (2024).
• Trabucco, Brandon, et al. "Conservative objective models for effective offline model-based optimization." International Conference on Machine Learning. PMLR, 2021.
• Zheng, An, et al. "Deep neural networks identify context-specific determinants of transcription factor binding affinity." BioRxiv (2020): 2020-02.
• Hutchinson, Gordon B. "The prediction of vertebrate promoter regions using differential hexamer frequency analysis." Bioinformatics 12.5 (1996): 391-398.

Answering the reviewer's questions:

1. Linear probing: As an alternative to fine-tuning a pretrained network, which involves updating all of the weights in the network, one can extract embeddings or outputs from the pretrained network and fit a simple linear model to make predictions for a downstream task without updating the full pretrained network. This approach is called linear probing. We will clarify this point in the final version.

2. Certain promoters could be undiscoverable but our analysis shows that designs are representative of diverse sequence spaces that are also quite distinct from those containing the training set: The conservative regularizer mitigates adversarial designs, but as the reviewer noted, this regularization could prevent the discovery of some desirable promoters that are very different from the promoters in the training set. Discovering these promoters using MBO requires a model to be accurate in the sequence space around them. However, given a limited training set, it is difficult for a design model to generalize to such a sequence space (distribution shift problem). Therefore, it is difficult to discover such promoters even in the absence of the conservative regularizer. Since we have a limited experimental budget, the focus of our workflow is to make the best use of the available training data to design sequences while avoiding adversarial sequences.

   As the reviewer noted, it is also possible that certain promoters are harder to directly evolve from the sequences in the fine-tuning dataset; however, this problem is most pronounced when the fine-tuning dataset is not very diverse. The fine-tuning dataset we use in our experiments is diverse, and from table 1 we see that the resulting designs are also very diverse (with any pair of sequences differing by close to 180 bp on average), indicating that many parts of the sequence space are being represented in the designs. Moreover, table 1 in the document attached to the general rebuttal shows that on average, the designs differ from the most similar training set sequences by 125-140 bp, indicating that our workflow produces sequences that are quite different from the training set.

   We will mention these possible limitations in the final version and open this as an avenue for future work.

3. We use 6-mer frequency vectors to compute the Euclidean distance $\mathcal{K}$ so as to capture the frequencies of potential transcription factor (TF) binding motifs that are around 6-12 bp long: TFs are proteins that bind to promoters to facilitate gene expression, each binding to specific DNA substrings called motifs. Most motifs are around 6-12 bp long (Zheng et al., 2020). By computing 6-mer frequencies, we can approximate the frequencies of different motifs in a given sequence. Then, choosing promoters with distinct frequency vectors (separated by a long Euclidean distance) yields a diverse set of promoters with unique regulatory landscapes, enhancing the likelihood of identifying cell-type-specific promoters. While higher k-mer frequencies (e.g., 7-mer, 8-mer) could be used, they would result in more distinct k-mers and sparser frequency vectors, making it harder to meaningfully differentiate between promoters. Thus, we opt for 6-mer frequency vectors to balance the need to capture TF-binding motifs and maintain sufficiently dense frequency vectors. Notably, hexamer statistics have historically been used to identify promoter regions (Hutchinson, 1996).

4. Base pair entropy is computed with base 2 logarithms: Therefore, a set of purely random sequences will have a base pair entropy of $\log_2 4 = 2$. We failed to mention the base in our manuscript but will clarify this in the final version.

5. Since DENs are generative models, the metrics in figure 2 cannot be computed for them, leading us to use the metrics in figure 3: The metrics shown in figure 2 compare each designed sequence to the starting sequence from the fine-tuning dataset that was used to generate it. These results illustrate that our workflow significantly improves most starting sequences from the fine-tuning dataset, while motif tiling mostly reduces the DE of starting sequences. Since DENs are generative models, they do not take in a starting sequence – instead, they directly generate a pool of design sequences using random noise vectors as inputs. Thus, we cannot use the same metrics as in figure 2 to analyze the performance of DENs vs. gradient ascent. In figure 3, therefore, we directly compare the DE of sequences produced using DENs to those produced using gradient ascent.

We thank the reviewer for their detailed comments and for recognizing the significance of our work. In the responses that follow, we clarify that our data-efficiency claim is due to the pretraining step which is missing in previous promoter design workflows – we will make this clear in the final version of our manuscript. Next, we justify why we expect the pessimistic estimate of DE to underestimate the true DE due to its definition as the lower confidence bound (LCB) of the ensemble’s constituent models’ predictions. We point to more details about the ensemble’s architecture and explain our architectural choices – our explanation will be added to the final manuscript. Then, we explain that we retain the diversity component of our final selection algorithm to allow a practitioner to boost diversity if necessary. Using an additional analysis and results from Table 1, we show that our designs represent diverse regions of the sequence space that are also distant from the space of training sequences, but we acknowledge that certain desirable promoters will be undiscoverable – we will discuss this more in the final manuscript. We also address the other weaknesses and questions mentioned by the reviewer. We hope that our responses make a stronger case for the acceptance of our paper.

Addressing weaknesses identified by the reviewer:

1. Our workflow’s data-efficiency comes from the pretraining step that is absent in existing promoter design workflows: The main reason we claim data-efficiency is indeed due to the pretraining step (lines 71-74 and Section 4.1). Our modeling strategies are derived from Reddy et al. (2024), who showed that pretraining on related genomic datasets can boost downstream performance when the pretrained model is fine-tuned using a small PE dataset. As we point out in the related work section, existing MBO-based promoter design workflows are reliant on large MPRA datasets and train their design models exclusively on these large datasets. By adopting the transfer learning strategies identified by Reddy et al. (2024) in our workflow, we can obtain more accurate design models using the same amount of data from target cells when compared to existing workflows that do not leverage transfer learning, thereby supporting our data-efficiency claim. We will make this point clearer in the final version of our manuscript.

2. Since the pessimistic estimate of DE is computed as the lower confidence bound (LCB) of the ensemble’s constituent models’ predictions (lines 269-270), we expect it to underestimate the true DE if the ensemble is accurate: If we have reasonably accurate models of DE, the mean prediction from the ensemble should be an accurate estimate of the true DE. Since the LCB is computed as the mean prediction minus the standard deviation across predictions, this pessimistic estimate should underestimate the true DE, provided the ensemble is reasonably accurate.

   Ensemble model architectures: Details about the ensemble’s constituent models are in Section D.2 of the appendix. Since it is expensive to pretrain many different architectures, we use the same pretrained backbone in all constituent models while using different output layer architectures during fine-tuning. In particular, we vary the depth and number of hidden units of the output layers. We also vary the activation functions, as using different activation functions in the constituent models’ output layers allows us to approximate the PE function using different piecewise functions, thereby improving the ensemble’s overall robustness.

3. Diversity component of final selection algorithm is retained to have the flexibility to boost diversity if necessary: While we set the diversity coefficient to zero for sequences designed using gradient ascent, we set it to 10 when using DENs (line 351), as the top designed sequences produced by DENs were not as diverse in our preliminary experiments as those produced using gradient ascent. We retained this ability to modulate diversity in our workflow for the sake of flexibility – if a practitioner observes that the top designed sequences are very similar to one another, the diversity coefficient can be increased to avoid selecting highly similar sequences in the final sequence set.

4. The conservative regularizer only penalizes potentially undesirable (i.e. potentially adversarial) sequences among those not seen in the training set – it does not penalize all unseen sequences: We will improve the clarity of this description in line 217. The second term in Eqn 2 prevents uniform underestimation of the function being modeled across all sequences by maximizing the expected value of the function over the training set (see Trabucco et al. (2021) that presents the COMs framework for more details).

Thank you for the detailed response! I believe you have addressed most of my concerns and I have revised my score to reflect this.

For point 4 (on unseen sequences), I suggest you edit the text in the paper to clarify the same point.

We thank the reviewer for their comments and for recognizing the significance of our work. In our responses below, we first justify the use of gradient ascent vs. other optimizers such as genetic algorithms and simulated annealing, due to its computational efficiency. Then, we clarify that our workflow can be used to produce diverse designs, even when conservative regularization is used, due to our use of multiple design models that are trained using different regularization strengths, as well as our final sequence selection algorithm that balances optimality and diversity to select the final set of designs for wet lab validation. Finally, to better illustrate our workflow in the final version of the paper, we will be adding a decision-tree-like figure as suggested by the reviewer - a draft of this figure is in the document attached to the general rebuttal.

1. Gradient ascent is used instead of other optimizers due to its computational efficiency, and the conservative objective mitigates adversarial effects: While discrete optimizers such as genetic algorithms and simulated annealing are suited for making discrete mutations to a DNA sequence during optimization, it can be computationally expensive (requiring many samples and filtering) to use them in a large search space such as ours (250 bp sequence = $4^{250}$ possible sequences), since these algorithms do not use the function’s derivative to take a step towards an optimum. Similarly, we did not experiment with RL-based models for DNA-sequence design since it is difficult to integrate them with the conservative objective models framework, which requires a fast optimizer during training to mine for potentially adversarial sequences. On the other hand, gradient ascent is computationally efficient and allows us to quickly discover an optimum. Furthermore, in preliminary experiments, we experimented with adding discrete mutations during training and sequence optimization in addition to performing gradient ascent, but we did not observe a significant improvement in sequence quality vs. just using gradient ascent (as measured by design model-predicted differential expression). Overall, simplicity and efficiency considerations push us to use gradient ascent instead of other discrete optimizers.

   As the reviewer points out, it is true that gradient ascent algorithms can produce adversarial effects. However, we utilize a conservative objective to mitigate them, as described in Section 4.2 in the paper. The other optimizers are also not immune to producing adversarial effects – if run for a sufficient number of steps, any of the aforementioned algorithms can produce adversarial sequences since the designed sequences can be arbitrarily different from the training set.

2. Collecting sequences from design models trained using different conservative regularization levels, and using our final sequence selection algorithm produces a diverse yet desirable set of final designs: As the reviewer notes, strong levels of conservative regularization could hinder the diversity of designed sequences. In order to maintain a high level of diversity, we obtain a large set of candidate sequences using many different levels of regularization as mentioned in lines 240-242 (controlled by the $\alpha$ parameter in Eqn 3). Then, we use the algorithm in Section 4.3 to choose a subset of sequences for wet lab validation that are diverse yet desirable. This combination of multiple regularization levels and the final sequence selection algorithm yields a diverse final sequence set as illustrated in Table 1, overcoming potential issues of an over-regularized model producing a narrow distribution of candidate sequences. That said, we agree that fundamentally, conservatism (to stay close to the data) and diversity (going beyond the data in several ways) are at odds with each other more broadly from an algorithmic perspective, and studying these topics is a good avenue for future work, even for general ML, outside of biological problems, as we will discuss in the final version of paper.

3. We will add additional figures that illustrate our guidelines to the final version of the paper. A draft decision-tree-like figure is in the document attached to the general rebuttal. We will refine this figure for the final manuscript.