Protein Multimer Structure Prediction via Prompt Learning

We sincerely appreciate the reviewer y71p's recognition of our paper and his/her valuable time. We will respond to reviewer's insightful suggestions point-by-point.

[Cons. 1 GIN encoder distribution shift] We thank the reviewer for the valuable comments.

The comment relates to the core motivation of this paper. To avoid distribution shift caused by multimers of different scales (chain numbers) in the target task of link prediction, we re-model the link prediction task of any scale as a graph-level task for an $l=3$ path (graph). Please kindly note that the node number of such a path is fixed at 4.

To also avoid distribution shift in the source task (graph-level regression), we should ensure that the scale $N$ of the source data for pre-training the GNN model is around 4. We mentioned this operation (in our original manuscript) at the end of the section "Reformulating the Target Link Prediction Task" as "the pre-trained model should always be fed with graphs of similar scales (i.e., node number ranges from 2 to 7) to the source data."

In fact, to aviod distribution shift on the GIN encoder, we have tried (1) only using multimers with $N=4$ for pre-training; (2) using sufficient multimer data of $2≤N≤7$ for pre-training. Empirically, we found that the latter (option) led to significantly better model performance. This might be due to the presence of not only gaps but also common knowledge that can be utilized between different scales. Thus, selecting multimer data close to the $N=4$ scale is the optimal choice.

We would like to thank the reviewer once again for the insightful comments, and we will provide an analysis of this issue in the revised manuscript.

[Cons. 2 Ablation of the pre-training] Thanks for the valuable suggestion. To address the reviewer's concerns, we establish the baseline without pre-training in the following experiments.

1. Ours: Our complete version of the PromptMSP model.
2. Ours w/o pt: Our method without pre-training the GIN encoder $\theta$ and the task head $\phi$.

Specifically, the prompt model still exists in Ours w/o pt. Unlike Ours, the GIN encoder $\theta$ and the task head $\phi$ are initialized on the target task before training and their parameters are then updated together with the prompt model $\pi$.

The experimental results (with AFM-produced dimers) are as follows:

It can be seen that pre-training the graph-level regression model has a significant effect on almost all chain numbers and average performance.

We sincerely appreciate the reviewer's insightful suggestions on our paper and his/her valuable time. We sincerely appreciate the reviewer's recognition for the contribution and novelty of this paper, and we apologize for any confusion caused by our presentation.

[Cons. 1. Background introduction] Thank you for bringing this important issue to our attention. It is necessary to provide an introduction to the background knowledge involved. Therefore, we will add relevant explanations in the revised manuscript.

For clarity, we briefly introduce the background knowledge here.

• Meta-learning is a technique that aims to enable models to quickly and efficiently learn from and adapt to new data. Meta-learning is a training technology rather than a deep neural network. It is well-known for its ability to learn common knowledge from sufficient data and quickly adapt to (new) scarce data. For the multimer structure prediction (MSP) task, there is a severe imbalance in the number of scales (chain numbers) among multimers, which can be intuitively well-addressed by meta-learning techniques.
• In Section 2, we have introduced the background of prompt learning. In short, prompt learning can narrow the gaps between different data or tasks by introducing learnable flexibility to the model without the need for fine-tuning the pre-trained model. For the MSP task, we verified that regardless of whether the problem is modeled as TM-Score prediction or link prediction, there is a knowledge gap caused by the difference in chain numbers (scales) among multimers. This challenge leads to poor performance of training a unified model using data from all scales (please kindly refer to the experimental results in [Cons. 5]). Motivated by this, we aim to use prompt learning to establish connections between data of multiple scales, which allows us to train a unified (prompt) model to handle all samples.
• $l=3$ PPI path: We apologize for the missing explanation. We respectfully explain that $l=3$ path is an important identifier for PPI prediction. The existence of such a path implies that the proteins at both ends of the path will undergo PPI. Since $l=3$ path always has only 4 nodes, this allows us to re-model the link prediction task for multimers of different scales as a 4-node graph-level regression task (TM-Score prediction).

We sincerely thank the reviewers once again for bringing to our attention the importance of these key background knowledge.

[Cons. 2. Inference process] Thanks for the valuable comment. We have reorganized Figure 5 (framework diagram) and added the description of the inference process (Section 4.4).

The inference process is multi-step. For an $N$-chain multimer, we randomly choose a starting chain and then perform $N-1$ assembly steps in order. During each step, we predict linking probabilities of all possible chain pairs and then select the most possible pair for assembly.

[Cons. 3. C-PPI, Section 3.3 and scales] We thank for the valuable suggestions. We respectfully merge these three questions together because they are closely connected.

One of the purposes of emphasizing C-PPI is indeed to distinguish our method from MCTS and to demonstrate that using only I-PPI modeling for MSP tasks is biased. However, another crucial factor to consider is that predicting C-PPI (i.e. link prediction) with different chain numbers of multimers can lead to the out-of-distribution (OOD) situation for model's generalization. For clarity, we use "10+1" to denote predicting the next right link on an already assembled 10-chain multimer. Such OOD issue means that we struggle to use a model trained on "10+1" (C-PPI) data to perform the "29+1" prediction. This is why we emphasize the importance of "scale" in Section 3.3 and throughout the entire paper.

We apologize for any confusion caused by our unclear presentations. We will remove the wording "oligomers" and restate the relationship between C-PPI and scale in the introduction part.

We thank the reviewer 6yks for acknowledging our method as "impressive", our experiments as "comprehensive" and our prompt method as "interesting contribution". Many thanks for the constructive feedback especially for the questions about the presentation and our detailed settings. These all help us to further enhance the readability and the quality of our paper.

To address reviewer 6yks’s concerns, we provide point-wise responses below.

[Cons 1. Method description and Figure 5's organization] We sincerely apologize for the confusion caused. To address this issue, we have (1) clearly described the proposed method and (2) made revisions to Figure 5 and the description of meta-learning part. We sincerely appreciate your suggestions, as they have indeed improved the readability and quality of our paper.

1. Descriptions:

1.1 The motivation of our paper: Under the multimer structure prediction (MSP) task, our main goal is to address the impact of knowledge gap of multimer scales (chain numbers) on C-PPI prediction.

1.2 C-PPI knowledge gaps: In Section 3.3, we validated the existence of gaps and connections in C-PPI knowledge implied by multimers of different scales (chain numbers).

1.3 $\mathscr{l}=3$ rule prompting (main contribution): We first pre-train the GNN model, which takes an assembly graph as input. Its encoder $\theta^*$ outputs the node-level embeddings and the head $\phi^*$ outputs the predicted TM-Score. During tuning, we update only the prompt model $\pi$ based on the target data (conditional link prediction). For clarity, we refer to the combination of these three modules as a pipeline, denoted as $f_{\pi|\theta^*,\phi^*}$. In short, the pipeline $f_{\pi|\theta^*,\phi^*}$ outputs the linking probability conditioned on an already assembled portion of the multimer and a newly added chain.

1.4 Meta-learning: We respectfully clarify that the meta-learning method we apply is a training strategy, rather than a neural network. It is responsible for reliable prompt tuning, i.e., to update $\pi$ within $f_{\pi|\theta^*,\phi^*}$.

2. Revisions:

2.1 We have re-organized Figure 5, where we remove the meta-learning part and clearly show the specific process of training (pre-training, prompt-tuning) and testing.

2.2 We explicitly state in Section 4.3 that meta-learning is a training strategy and explain the purpose of applying this technique.

2.3 We have added a schematic diagram (Figure 7) to show the process for obtaining prompt tuning results.

2.4 We have included a part (Section 4.4) to introduce the inference stages.

All revisions can be found in our revised manuscript. We sincerely apologize for any confusion we may have caused.

[Cons. 2 The meta-learning part's readability] Thanks for your valuable suggestions. To make the applied meta-learning technique much clearer, we have revised the entire Section 4.3 and visualized specific cases to illustrate how it is responsible for prompt tuning.

In response, we describe the process as follows.

• Assuming that the pre-trained model has been well-trained, now we have the GIN encoder $\theta^*$ and the task head $\phi^*$. They will be frozen from now on.
• Our goal of prompt tuning is to obtain 2 different prompt models $\tilde{\pi}$ and $\pi^*$ for scales $3\leq N\leq 7$ and $N>7$, respectively. For example, to infer a $10$-chain multimer, we perform 9 assembly steps. In each of the first 6 steps, we apply the pipeline $f_{\tilde{\pi}|\theta^*,\phi^*}$ to predict the linking probabilities of all pairs of chains and select the most likely pair for assembly. Upon this assembled $7$-chain multimer, we apply the pipeline $f_{\pi^*|\theta^*,\phi^*}$ to sequentially add the remaining 3 chains.
• The process of getting $\tilde{\pi}$: We construct a pool of tasks from the link prediction datapoints created from multimers of $3\leq N\leq 7$. Each task contains two sets (a support set and a query set) of link prediction datapoints. In each epoch, we do three things in order. (1) Randomly sample $B$ tasks from the task pool; (2) for each of the $B$ support sets, update a separate model starting from $\pi$, resulting in $\pi^{(i)},i-1,...,B$; (3) compute the sum of loss ($\mathcal{L}\_{meta}$) of these all $\pi^{(i)},i=1,...,B$ on the query sets and update $\pi$ with the gradient of $\mathcal{L}_{meta}$ with respect to $\pi$. After multiple epochs of iteration, we obtain prompt model $\tilde{\pi}$.
• The process of getting $\pi^*$: We create link prediction datapoints from multimers of $8\leq N\leq 30$. Over multiple epochs, we update the previously obtained $\tilde{\pi}$ to $\pi^*$.

Thank you very much for your feedback. We sincerely believe that we have made every effort to organize the paper and that the revised version has greatly improved its clarity and readability.

Overall, our method includes two parts: the model part and the meta-learning part. The basic definition of meta-learning is Model-Agnostic, which is a technique that enables the prompt model to obtain more reliable training. In addition, existing work [re1] has already demonstrated that meta-learning can significantly improve the effectiveness of prompt techniques.

If you could provide more specific information on what is unclear, we would be more than happy to continue improving the readability of the paper. Besides, we will release the code and checkpoints to facilitate the reproducibility and further research in this area.

[re1] All in One: Multi-Task Prompting for Graph Neural Networks. SIGKDD (best research paper award), 2023.

We thank the reviewer PqHe for acknowledging our methodology is “interesting” and our motivation is “reasonable”. We sincerely appreciate the reviewer’s valuable time and comments. We provide point-wise responses below.

[Cons. 1 Monomer's ground-truth structures] We acknowledge and apologize for the ambiguity in the task objective stated under Definition 1. We will revise it as: With the above definitions, our paper aims to predict assembly graphs (>>2 proteins chains) that maximize the TM-Scores, taking as inputs the residue sequences of proteins and pre-calculated dimer structures (=2 protein chains). However, we respectfully clarify that pre-calculated dimer structures can be obtained from AlphaFold-Multimer without the need for providing ground-truth monomers.

We strongly agree that the ground-truth monomer or ground-truth dimer structures should not be used in real-world setting. We respectfully emphasize that our approach did not rely on ground-truth monomer to perform inference in real-world setting.

Indeed, in this work we propose a framework to predict the best possible protein multimers (>>2 protein chains), taking as inputs the pre-calcuated dimer structures (=2 protein chains). For the pre-calcuated dimer structures, we considered two settings in our original experiments:

• The "AFM Dimer" setting does not involve ground-truth monomer nor ground-truth dimer structures. We predict dimers with AlphaFold-Multimer by inputting only amino acid sequences and our model also only takes amino acid sequences as inputs for predicting the docking path. This setting is applicable to real-world scenarios and consistent with the experimental setup of MoLPC [re1] (our baseline).
• The "GT Dimer" setting may be less realistic under the application scenarios, but we respectfully think that it is not completely meaningless. The performance under this "GT Dimer" setting suggests the effectiveness of the docking paths predicted by ours and the MoLPC baseline.

[Cons. 2 HDock and xTrimoDock] We sincerely thank the reviewer's comment. Firstly, we respectfully clarify once again that our "AFM Dimer" setting does not require any ground-truth monomers as input for real-world inference.

• HDock and xTrimoDock may not be suitable for serving as the baselines. We have checked HDock's local version and online server, and inquired the HDock team about the potential of performing multimer structure prediction (MSP). We find that HDock is currently unable to perform hetero-protein MSP tasks. We also checked all versions related to xTrimoDock [re2,re3,re4], and even went through the supplementary materials in its NeurIPS'23 submission, but we could not find any available code. Thus, we could not use HDock and xTrimoDock as baselines to directly predicting multimer structures.
• However, we greatly appreciate the suggestion for taking HDock into consideration and have added the "HDock Dimer" setting, where we use HDock to pre-calculate dimer structures.
• To further demonstrate that our method is not dependent on ground-truth monomers or ground-truth dimers, we also added the "ESMFold Dimer" setting, where we use sequences to predict dimer structures with ESMFold. Under all three settings, our method consistently and significantly outperforms the MoLPC and AlphaFold-Multimer baselines.

Overall, we sincerely appreciate the suggestion about HDock which helps yield surprisingly satisfactory experimental results. We consider HDock as a setting to provide pre-calculated dimer structures, rather than a baseline that can directly perform MSP.

Detailed experimental results will be shown in Table 9 in the Appendix of our revised manuscript.