KinDEL: DNA-Encoded Library Dataset for Kinase Inhibitors

Thank you for your careful review of our work and your valuable feedback.

Dataset description: We appreciate your specific comments on making our paper more accessible to machine learning researchers. We will revise Figure 2 by removing unnecessary technical details that do not contribute to the understanding of the data format. For instance, the biotinylation of proteins is detailed in the appendix and can be omitted from Figure 2 to avoid potential confusion. Additionally, we will add a new subplot that outlines the high-level data acquisition process and data format. This subplot will clarify the inputs and labels (pre- and post-selection count data) used in our models. We will also make sure to define all new terms that might be confusing for the machine-learning audience.

Biophysical Assay Validation: We will relocate the appendix reference concerning the selection of validation compounds to an earlier section of the text for clarity. The limited number of validation points is due to the high cost associated with these experiments, as each molecule requires resynthesis, either on-DNA or off-DNA, along with separate biophysical assays for each.

Pre-selection Data: During the synthesis of the DNA-encoded library, not all molecules are produced in equal amounts. Variability can arise from differences in synthesis efficiency, coupling yields, or synthesis errors. Additionally, some molecules may precipitate or adhere to storage containers, resulting in loss and lower observed counts. While this data is typically used to normalize counts, there are important caveats. Sequencing inaccuracies can lead to error propagation, especially for low-abundance molecules. DEL-Compose is a model that addresses these issues by incorporating pre-selection counts as inputs, which serve as a multiplier when modeling the zero-inflated Poisson distribution in the loss function. We will add this discussion in the appendix.

Range of Kd values in the held-out test data: We will provide histograms of Kd values in Appendix C. Our approach involves selecting both binders and non-binders with varying Kd values to evaluate our model's ability to accurately rank these molecules using Spearman’s correlation. Although we propose a regression problem where molecules are ranked from most to least promising, the Kd values can also be categorized into two groups for testing classification models, as the dataset includes both binders and non-binders.

Only 1M top molecules are used for training: We observe that the baseline models struggle with sparse DEL data and require modifications to the loss function or additional inductive biases to handle label sparsity. To simplify learning, we train on the top 1M compounds. This introduces some bias relative to the evaluation set, but since we aim to rank order the molecules rather than make binary predictions, we believe our model can effectively rank molecules in validation sets. Nonetheless, we provide access to the full dataset for researchers to explore better results with more training points.

Ranking molecules by binding affinity: All the evaluated models predict continuous values, employing regression, and we use Spearman's correlation as the evaluation metric. Unlike Pearson's correlation, which identifies linear relationships, Spearman’s correlation effectively assesses ranking quality. We will add a brief discussion on the ranking issue and our rationale behind selecting this evaluation metric. In short, these models are often used to score available molecules to maximize their binding affinity. It is challenging to establish a firm threshold to define a binder because priorities change depending on the target and stage of drug development. Measuring model performance via ranking ensures that good models retrieve the best options for any given target and screening pool combination.

Other data splits: Regarding the splitting of KinDEL, we explored several methods and determined that the disynthon split is optimal. Although we considered splitting datasets by single synthon positions, this approach presented two challenges. Firstly, only one part of each molecule is unique in the testing data, while the model encounters all possible disynthons at the remaining two positions. Secondly, some synthons may feature motifs essential for binding, such as kinase hinge-binding motifs, which could end up entirely in the testing data. We believe that the disynthon approach strikes a sensible balance. Please see Figure 4, where the visible line patterns relate to enriched disynthons, and the absence of clear planes indicates the lack of enriched (mono)synthons.

AIRCHECK datasets: Thank you very much for this suggestion. We will reference this resource in the related work.

Thank you once again for your invaluable review and suggestions. Please let us know if you have any further questions.

Thank you for dedicating your time to review our paper and for your positive feedback and valuable suggestions. We will include the number of data points for the testing sets in the main text as recommended. Furthermore, we agree that evaluating the generalization of the models on external datasets, including compounds with activity against other kinases and decoys, is a compelling research direction. We see this as a promising avenue for future work. Once again, we appreciate your thorough review and insightful feedback! Please let us know if you have any further questions.

We appreciate the Reviewer's time and effort in evaluating our paper and are pleased with the overall positive feedback. The only concern raised pertains to our choice of venue for publication. We would like to address this by outlining our reasons for selecting this machine-learning conference over a journal.

Firstly, we believe that presenting our dataset and benchmark at such a conference maximizes its impact by directly reaching its target audience—machine learning researchers. The lack of high-quality DEL datasets is a bottleneck for developing advanced machine learning models. By contributing this work, we aim to advance research in analyzing the outputs of innovative compound screening technologies.

Furthermore, this year ICML has specifically called for application-driven submissions. The Call For Papers seeks “Application-Driven Machine Learning (innovative techniques, problems, and datasets that are of interest to the machine learning community and driven by the needs of end-users in applications such as healthcare, physical sciences, biosciences)”. We believe our paper aligns well with this focus, as it provides a valuable new dataset for an important area of science that currently suffers from a lack of high-quality data.

Thank you once more for your feedback. Please let us know if you have any further questions.