Rigid Protein-Protein Docking via Equivariant Elliptic-Paraboloid Interface Prediction

We sincerely thank your constructive comments. We address your concerns below accordingly.

• The major concern about the method lies in the comparison with traditional template-based methods, e.g., HDOCK. As shown in Tables 1&2, HDOCK outperforms other methods by a significant margin, suggesting it may already effectively address the rigid-body docking challenge. Although the ElliDock’s inference speed is much faster than HDOCK, its performance is unsatisfactory. Given real-world applications, the value of faster inference time seems negligible, especially if the docking challenge doesn't necessitate high-throughput situations. Please correct me if I’m wrong.

Thank you for the comment. While we respectfully agree with your point that HDOCK is a strong baseline in performance, we believe that models with faster inference time are more suitable in high-throughput scenarios. For example, in drug design, when studying whether inserting, editing or deleting sequence fragments in different regions of a protein affects the binding ability and binding site. If ElliDock is used in this case to simulate the binding process among the edited proteins, its fast speed will nicely support a large number of simulated docking processes within an acceptable time, such that we can locate the desirable protein candidate from its huge amino acid space.

• The notation issues.

Thank you very much for raising these issues. We have made corrections to the notations you mentioned. In Eq. (8), we use $\neg{p}$ to denote the other protein. In Eq. (12), the first three elements of $F_1$ and $F_2$ are used to construct the elliptical paraboloid, and the fourth element is used for rotation refinement, which are both mentioned in the paper. $M$ is a hyperparameter introduced in Eq. (13), which is further specified in the paragraph under Eq. (16). Regarding the problem of excluding $p$, we have formulas that only apply to a single protein (e.g., Eq. (8)), formulas that require information interaction (e.g., Eq. (7)), and formulas that require both proteins (e.g., Eq. (14)), so we believe that the subscript $p$ can make the expression clearer.

• A deeper exploration and connection with surface-based methodologies, such as MaSIF, would be beneficial. It would be insightful to include a comparison with it in the benchmarking section.

Thank you for introducing MaSIF as a surface-based methodology. From a methodological point of view, MaSIF models a real surface with biochemical properties, making use of protein surface chemical features such as hydropathy and free electrons; ElliDock models a fictional elliptical paraboloid as a surface, which does not represent chemical properties, yet represents the position information of the pockets as well. On the other hand, MaSIF can obtain its fingerprint from protein monomers, which is conducive to studying the properties of the monomers, while ElliDock interacts with the features of receptors and ligands during the docking process, which may improve the expressiveness on complexes.

To address your concern, we have added the content to the Introduction section in the revised version.

• Q1: about attention mechanism.

We believe that the inner product of $q$ and $k$ has a larger representation space than direct mapping of the message $m$, because the former is a mapping of $\mathbb{R}^{H\times H}$ to $\mathbb{R}$, and the latter is a mapping of $\mathbb{R}^{H}$ to $\mathbb{R}$. Please correct me if I'm wrong.

• Q2: the possible data leakage of HDock.

Yes, we can show the evidence of the possible data leakage of HDock. From the paper of HDock [1], we find that HDock may have data leakage in two parts: first, during the docking procedure, HDock needs to search the whole PDB (thus may also include the testing data in our experiments) for homologous templates and build the 3D structures through homology modeling, and HDock will update the PDB data through version; second, the protein-protein scoring function [2] used by HDock, referred to as ITScore-PP, was derived using the crystal structures of a training set of 851 protein–protein dimeric complexes containing true biological interfaces.

• Q3: about $F_p$.

Sorry for the confusion. The fourth element of $F_p$ is used during rotation refinement, which is shown in Section 3.5.

[1] Yan, Y., Tao, H., He, J., & Huang, S. Y. (2020). The HDOCK server for integrated protein–protein docking. Nature protocols, 15(5), 1829-1852.

[2] Huang, S. Y., & Zou, X. (2008). An iterative knowledge‐based scoring function for protein–protein recognition. Proteins: Structure, Function, and Bioinformatics, 72(2), 557-579.

Sorry for the inconvenience that the revised part of the paper was not marked in red previously, now we have resubmitted our revised paper. For the surface-based baselines you mentioned, we have added the discussion of MaSIF in the Introduction section. We have also modified the notation issue. Meanwhile, we provide visualization of predicted elliptic paraboloid interfaces in the appendix. Please let us know if you have other concerns.

We sincerely thank your constructive comments. We address your concerns below accordingly.

• In addition to comparisons with machine learning-based docking methods, it would be beneficial to provide an overview of results in comparison to well-established docking methods that do not rely on machine learning techniques.

Thank you for the suggestion of adding baseline models that do not rely on machine learning techniques. In fact, as a template-based model, HDock is not strictly a machine learning model, and we choose HDock as a representative of the traditional docking method due to its state-of-the-art performance among traditional docking methods (based on the experiments of EquiDock [1]).

• Would you kindly offer substantiating evidence or theoretical findings to validate the assumption that elliptic-paraboloids accurately align with the ground-truth docked pockets?

Thanks! We provide more explanations on DB5.5 here. DB5.5 is a well established docking database. From our point of view, IRMSD measures the ability of the model to align to pockets. In Tables 1 and 2, the performance of our model on IRMSD is better than that of models trained from scratch, indicating that our method is better at finding pockets. We are open to discussing if you still have any questions.

• Could you furnish more compelling evidence to substantiate the assertion of potential data leakage in HDock, as stated in the paper?

Thank you for giving us the chance. From the paper of HDock [2], we find that HDock may have data leakage in two parts: first, during the docking procedure, HDock needs to search the whole PDB (thus may also include the testing data in our experiments) for homologous templates and build the 3D structures through homology modeling, and HDock will update the PDB data through version; second, the protein-protein scoring function [3] used by HDock, referred to as ITScore-PP, was derived using the crystal structures of a training set of 851 protein–protein dimeric complexes containing true biological interfaces.

• Can you provide a more comprehensive explanation regarding the reasons behind the improved inference time compared to other methods?

Thanks. Here we briefly analyze reasons for the differences between the inference time of different models. First of all, ElliDock and EquiDock are both regression models and generate the transformation in one step from end to end. Therefore, there is no order of magnitude difference between them in speed. From Table 1, we show ElliDock is $\textasciitilde$2x faster than EquiDock, which is more likely due to the amount of model parameters, number of model layers, etc.. DiffDock-PP is a diffusion-based model, with a denoise process of T steps (T is a hyperparameter) in the inference procedure. Each inference step requires re-running the model, so there is a speed gap of 10$\textasciitilde$100 times compared with the regression model. HDock is a template-based model that not only obtain the templates of receptors and ligands through sequence alignment, but also needs to evaluate the energy of the predicted complex based on the scoring function and perform multi-step updates accordingly. Meanwhile, HDock can only run on the CPU, limiting its speed even more. Multimer is a deep learning model, yet it requires multiple sequence alignment (MSA) during training and inference, and its pre-trained models have a large number of parameters (GB level), so the inference time is extremely long.

• Could you consider releasing the splits of the second dataset utilized in your experiments to facilitate reproducibility efforts within the research community?

We have already uploaded the bash file for downloading and spliting the dataset. Please check README in our source code for more details. All the code sources will be released after our paper is accepted.

[1] Ganea, O. E., Huang, X., Bunne, C., Bian, Y., Barzilay, R., Jaakkola, T., & Krause, A. (2021). Independent se (3)-equivariant models for end-to-end rigid protein docking. arXiv preprint arXiv:2111.07786.

[2] Yan, Y., Tao, H., He, J., & Huang, S. Y. (2020). The HDOCK server for integrated protein–protein docking. Nature protocols, 15(5), 1829-1852.

[3] Huang, S. Y., & Zou, X. (2008). An iterative knowledge‐based scoring function for protein–protein recognition. Proteins: Structure, Function, and Bioinformatics, 72(2), 557-579.

We sincerely thank your constructive comments. We address your concerns below accordingly.

• The paraboloid formulation is elegant, but I would like to see some discussion as to why this is the right choice compared to other considered options.

Thank you for recognizing that our work. We choose elliptic paraboloids other than planes, spheres or complex manifolds to model the binding interface, as elliptic paraboloids are appropriately defined in 3D space and their SE(3) transformations can be devised in a closed form. On the one hand, if using planes, spheres or other symmetric 2D manifolds to model the interface, it will easily cause the docking ambiguity issue since there are usually different ways to transform a 2D manifold from one pose to another. On the other hand, if using complex manifolds, it may introduce bias and higher computational complexity to obtain the docking solutions. Under these two requirements, we believe that the elliptical paraboloid is a good trade-off.

• The argument of why ElliDock's formulation is better than EquiDock seems to be developed sufficiently well, but I would like to see some more discussion contrasting with DiffDock-PP and AlphaFold-Multimer.

Sorry for the insufficient explanations. We know that our work ElliDock and EquiDock are both regression models, which allows an easier comparison between two methods. On the contrary, DiffDock-PP is a diffusion-based model, it successfully builds a model that meets the degree of freedom required by the task. The introduction of noise in diffusion steps gives DiffDock-PP the ability to find multiple binding pockets, while it also leads to 10~100 times of the inference time compared to regression models. Alphafold-Multimer is based on Alphafold, its task setting is the structure prediction from sequence instead of the rigid protein-protein docking, thus the method design is different from other methods mentioned above.

• The inference time is reported in Table 1, but not in Table 2. Could you please report the values for Table 2?

Nice suggestion! We have additionally updated the inference time in Table 2 in the revision. We believe that the inference time should be approximately proportional to the scale of the test set, so we only reported the results of one test set previously.

• Could you please report the inference time in Table 3. It would be interesting to see which losses could be removed for even more speed-up, but without accuracy loss.

Thank you for the comment. We would like to clarify that the losses only play a role in the training process and have little impact on the inference time during validation and testing. Hence, it is not very informative to report the inference time in Table 3 (the time is almost the same no matter which loss we remove).

• Since DiffDock-PP is a diffusion-based model it is capable of modelling multiple conformations. How does ElliDock fare when the ligand-protein pair has multiple possible conformations?

Thank you for raising this question. Like EquiDock and other regression-based methods, our Ellidock is not specifically designed to handle the situation when there are multiple binding sites for one single pair of receptor and ligand. However, for the datasets, particularly the antibody-antigen binding tasks, the specificity of the antibody is almost only determined by a small special region of itself (complementarity determining region, CDRs). In actual scenarios, it might be difficult for the same pair of antibody and antigen to have multiple binding sites.

• Could you argue a bit about why ElliDock's training objective is more suitable than DiffDock-PP's?

In our perspective, since ElliDock and DiffDock-PP are different types of models, the training objectives are difficult to compare directly. The training target of DiffDock-PP is mainly the score function of rotation and translation, which pays more attention to the distribution changes of the overall molecule. ElliDock is based on interface fitting and tends to focus more on pockets in the design of training objectives.

• Why do you think ElliDock outperforms AlphaFold-Multimer on SAbDab?

Thanks. Considering only the SAbDab data set, since the pre-training data of Multimer comes from the general protein domain, it probably performs poorly on the antibody-antigen domain. In Table 2 we show that ElliDock has better performance on both CRMSD and IRMSD metrics. As for the DockQ metric, it can be seen from Figure 5 that ElliDock has better prediction results on most test samples. However, Multimer has achieved extremely amazing accuracy on a few samples, which may be attributed to the powerful representation ability of Alphafold.

We sincerely thank your constructive comments. We address your concerns below accordingly.

• so small font size in figures, for example, Figure 1.

Thank you for your suggestions on fonts. We have adjusted the font sizes of some figures.

• the improvement is minor on DB5.5 when comparing with EquiDock and DiffDock-PP and much worse than Multimer and HDock. We would like to further emphasize the advantages of our Ellidock.

First, as mentioned in the paper, we use the pre-trained models of Multimer and HDock, both of which use the data in the PDB for training (Multimer) or template selection (HDock). Since DB5.5 is included in PDB, we assume that Multimer and HDock perform extremely well on DB5.5 due to possible data leakage. The small scale of DB5.5 might also limit the learning ability of the model.

Second, inference time is another important factor in addition to the performance metric. Clearly, ElliDock is much faster than HDock, DiffDock-PP, and Multimer. In our responses to other reviewers, we believe that models with faster inference time are more suitable in high-throughput scenarios. For example, in drug design, when studying whether inserting, editing or deleting sequence fragments in different regions of a protein affects the binding ability and binding site. If ElliDock is used in this case to simulate the binding process between the editted proteins, its fast speed will nicely support a large number of sumulated docking processes within an acceptable time, such that we can locate the desirable protein candidate from its huge amino acid space.

• [1] and [2] are stronger baselines on solving antibody-antigen complex.

Thank you for introducing two baselines for antibody-antigen complex. Unfortunately, the codes of both baselines have not been published, it is difficult for us to use them as experimental benchmarks. We will add further discussion in the revised paper.

• HDock performs much better than ElliDock. I guess the major reason is the input rigid structure is native unbounded structure. However, using native unbound structure as docking method's input is not realistic. What's the performance if you dock predicted structures.

While we also respect HDock for its promising performance, we have some concerns about its possible data leakage, as already expressed in our paper. From the paper of HDock [3], we find that HDock may have data leakage in two parts: first, during the docking procedure, HDock needs to search the whole PDB (thus may also include the testing data in our experiments) for homologous templates and build the 3D structures through homology modeling, and HDock will update the PDB data through version; second, the protein-protein scoring function [4] used by HDock, referred to as ITScore-PP, was derived using the crystal structures of a training set of 851 protein–protein dimeric complexes containing true biological interfaces.

• The pair of elliptical paraboloids have the same shape. How to handle large proteins or small proteins?

There could be some misunderstanding here. For different protein pairs, due to the model design of global information interaction, the shape of the elliptical paraboloid is determined by both the receptor and the ligand, so the interface shape can be adapted to the scale of the protein.

[1] Yujie Luo, et.al. xTrimoDock: Rigid Protein Docking via Cross-Modal Representation Learning and Spectral Algorithm.

[2] Ambrosetti, Francesco, et.al. Modeling antibody-antigen complexes by information-driven docking.

[3] Yan, Y., Tao, H., He, J., & Huang, S. Y. (2020). The HDOCK server for integrated protein–protein docking. Nature protocols, 15(5), 1829-1852.

[4] Huang, S. Y., & Zou, X. (2008). An iterative knowledge‐based scoring function for protein–protein recognition. Proteins: Structure, Function, and Bioinformatics, 72(2), 557-579.