We sincerely thank the reviewer rNSL for appreciations and valuable comments on our paper. To address reviewer rNSL's questions, we provide pointwise responses below.

[Cons1. Reliance on precomputed dimer structures] Thanks for this valuable comment. We sincerely acknowledge that the quality of precomputed dimers is important for our method, and this situation is also present in MoLPC [R1] (our baseline). In fact, our intuition is that given the data of dimers (2 proteins) is rich while the data of protein complex (>> 2 proteins) is sparse, we would like to transfer the knowledge of precomputed dimers to predict high quality protein complex.

Next, we summarize why the applicability of our method in the real-world scenarios is satisfactory.

1. We have considered using AFM and ESMFold to precompute dimer structures, and with either of them, our framework can significantly outperform other baselines.
2. When using AFM and ESMFold to precompute dimer structures, our method does not require monomer (chain) structures as input.
3. During inference, our method also does not require monomer (chain) structures to predict the docking path.
4. In the future, there may be some dimer structure prediction methods that can adapt our framework for further improvement.

To address your concerns, we provide the detailed results with AFM-produced dimers and explore an alternative method, ESMFold, to precompute dimer structures.

All results will be added to Table 2 (the main experiment table) in our revised manuscript.

• MoLPC(GT): the performance of MoLPC with GT dimers.
• MoLPC(AFM): the performance of MoLPC with AFM-produced dimers.
• MoLPC(ESMFold): the performance of MoLPC with ESMFold-produced dimers.
• GAPN(GT): the performance of ours with GT dimers.
• GAPN(AFM): the performance of ours with AFM-produced dimers.
• GAPN(ESMFold): the performance of ours with ESMFold-produced dimers.

Metric: mean TM-Score. The best and second best, other than GAPN(GT) and MoLPC(GT), is bolded and highlighted, respectively.

We have the following observations:

• Using dimers produced from AFM and ESMFold, our model can consistently outperform AF-Multimer (a baseline without precomputed dimer).
• Only MoLPC and our method can handle cases with a large number of chains ($N>15$).
• For our method, the performance with AFM-produced dimers and ESMFold-produced dimers is comparable.

More importantly, two existing dimer prediction methods that do not require given monomer (chain) structures have been verified to adapt well to our model. Among them, ESMFold has very high efficiency (about 0.6 seconds to predict a dimer structure).

As a result, we believe this demonstrates the applicability of our method in real-world scenarios. In the future, perhaps more dimer structure prediction methods will be verified to perform excellently within the GAPN framework.

Thank you for your positive feedback. We appreciate your valuable suggestions and the discussions.

We sincerely appreciate the reviewer's valuable time and suggestions. We address the reviewer's concern with the following response.

[Cons. Idea is not new] In this paper, we propose a reinforcement learning (RL) framework for the complex structure prediction problem. We sincerely clarify that the novelty of this paper is twofold.

1. We model the complex structure prediction problem as a Markov Decision Process for long-term benefits. This differs from the advanced baseline methods [R1,R2], which focus on greedily ensuring locally optimal assembly. This modeling ensures high exploration efficiency w.r.t. the vast action space and achieves excellent experimental results.
2. The design of a global view reward specifically for protein complex problems effectively addresses critical challenges of the task, which have been rarely mentioned in existing research. Overall, the introduction of the generative adversarial reward addresses the challenges of both the gap in knowledge between protein complexes of varied scales (chain numbers) and the scarcity of data for large-scale protein complex.

[R1] Bryant P, Pozzati G, Zhu W, et al. Predicting the structure of large protein complexes using AlphaFold and Monte Carlo tree search. Nature communications, 2022, 13(1): 6028.

[R2] Esquivel‐Rodríguez J, Yang Y D, Kihara D. Multi‐LZerD: multiple protein docking for asymmetric complexes. Proteins: Structure, Function, and Bioinformatics, 2012, 80(7): 1818-1833.

Thank you for your positive feedback. We appreciate your valuable suggestions and the discussions.

We sincerely thank reviewer Sjm2’s valuable time and comments. We provide point-wise responses below.

[Cons1. Terminology description] Thank you for your comments and for bringing this issue to our attention. To address it, we will provide a more explicit explanation of terminology such as docking, dimer and complex. Our revised manuscript PDF will include all these updates.

Here, we respectfully provide the descriptions to the reviewer. In the main body of our paper, we mostly use the terms protein and chain exchangeably. For "Docking", it refers to the action that involves any two proteins, where one protein is transformed to a new position to interact (physically contact) with another protein in space. In other words, "docking" results in the formation of a protein complex where the two protein chains finally come into contact with each other. "Dimer" refers to a protein complex with two chains, which is the result of docking of two protein chains. "Protein complex" refers to a protein group containing two or more chains. A protein complex with $N$ chains is formed as the result of $N-1$ docking actions.

[Cons2. Explanation of Eq.3] We really appreciate the reviewer's detailed comments. Firstly, we apologize for any confusion caused by the term "ground-truth assembly graphs" in our paper. To clarify, in the revised manuscript, we will modify the sentence to: "We search for the assembly graphs that specify true protein complexes. $p_{data}$ refers to the underlying distribution of data for such a set of assembly graphs."

First of all, we would like to clarify that there is no amino acid involved in our formulation. Given a set of $N$ protein chains (a protein chain consists of a set of amino acid), we want to predict the best possible protein complex with $N$ chains. However, due to the large computational space, we use "assembly graphs" to merge possible equivalent protein complex. In other words, an "assembly graph" specifies a protein complex in which we pay no attention to the detailed "docking" order. Based on different "assembly graphs", we can specify different protein complexes in which some protein complexes are true and some are not.

• Details about $p_{data}$: To further illustrate, we take an example of a 3-chain (indices: A, B, and C) complex in the training set. We pre-compute the embedding (vector) with each chain's amino acid sequence. These embeddings then serve as the node attributes of this 3-node assembly graph. The edges of the selected assembly graph are A-B and B-C. With such an assembly graph, we can truly assemble the corresponding protein complexes. Upon this assembly graph formulation, we expect these ones w.r.t true protein complexes to have large likelihood while others to have small likelihood. And this distribution is exactly $p_{data}$.
• About pairs of indices: We respectfully clarify that $p_{data}$ is represented by the protein (node) embeddings and assembly action (edges) rather than "pairs of indices". In Figure 1, we use indices "A,B,C,D" to distinguish each protein. As shown in Eq.4, each assembly graph drawn from $p_{data}$ is input into the discriminator (i.e., Graph Neural Networks) for representation computation. The information of an assembly graph involved in the computation includes its node embeddings and all edges, but not the node indices. Therefore, the indices can be specified arbitrarily as long as it is distinguishable, such as "D,E,F" or "1,2,3".
• Every graph becomes $N-1$ entries: For an $N$-chain protein complex, its assembly graph specifying the true protein complex contains $N$ nodes and $N-1$ edges. Every node has an embedding vector.
• Edge order: In our setup, as long as the set of assembly actions (edges) is determined, the final result is unique. Our discriminator $D_\phi$ will evaluate whether the set of edges is reasonable as a whole. Therefore, edge order dose not effect the output of discriminator for the current assembly graph.

[Cons3. The "AGG" notation] Thank you very much for your suggestion. To clarify, we will revise it using a more commonly used notation in the field of graph learning, namely ReadOut. This notation refers to the process of pooling node embeddings into a graph-level representation. By using the ReadOut function, graphs with different numbers of nodes can be distinguished by the discriminator.

Dear reviewer,

The discussion period is almost ending. Could you please confirm whether our responses have alleviated your concerns? If you have further comments, we will be happy to discuss them. For your interest, we clarify the key information on methodology and terminology description. Please refer to the details in the responses for you.

Thank you very much!

We appreciate the valuable time and suggestions from the reviewer MBaj, and we also thank the reviewer for recognizing the contributions and novelty of our paper. To address these concerns, we provide point-wise responses below.

[Cons1. Assembly Process] We sincerely thank the reviewer for this insightful suggestion.

We strongly agree that providing the assembly process is necessary and important for explaining the effectiveness of our method. As our model considers modeling the problem as a Markov Decision Process (MDP), the reinforcement learning (RL) agent will focus on the long-term impact of current decisions on future rewards (long-term benefits). This means that the assembly process of our method may not greedily maximize the current TM-Score, which is evident in the example 5W66 we have provided in Figure 4.

We will add visualizations of the assembly process for Figure 4's cases in Appendix G.

Here, we provide the assembly process and the current rewards (TM-Score) for each step.

$\rightarrow$: the assembly process; N-B: chain B (newcomer) is docked to chain N (already docked).

• Complex 7DHR with chain indices A, B, G, N, R.

Asssmbly process for 7DHR: N$\rightarrow$ N-B$\rightarrow$ N-A $\rightarrow$ A-R $\rightarrow$ B-G

• Complex 5W66 with chain indices A, B, C, D, E, F, G, H, I, J, K, L, M, N, P, Q.

Asssmbly process for 5W66: M$\rightarrow$ M-F $\rightarrow$ M-A $\rightarrow$ A-E $\rightarrow$ E-H $\rightarrow$ F-C $\rightarrow$ A-K $\rightarrow$ M-N $\rightarrow$ N-B $\rightarrow$ B-J $\rightarrow$ B-Q $\rightarrow$ Q-P $\rightarrow$ B-G $\rightarrow$ G-D $\rightarrow$ B-I $\rightarrow$ J-L

Calculated TM-Scores in order for 5W66: 1.0$\rightarrow$ 1.0$\rightarrow$ 1.0$\rightarrow$ 1.0$\rightarrow$ 0.93$\rightarrow$ 0.76$\rightarrow$ 0.72$\rightarrow$ 0.80$\rightarrow$ 0.83$\rightarrow$ 0.83$\rightarrow$ 0.86$\rightarrow$ 0.88$\rightarrow$ 0.90$\rightarrow$ 0.92$\rightarrow$ 0.92$\rightarrow$ 0.91

It can be observed that our method starts making mistakes from the 5-th step (highlighted as 0.93), but the majority of subsequent actions increase the reward. This indicates that our method does not greedily choose the optimal solution for the current action, but may make early mistakes to ensure the maximization of future rewards. For 5W66, if the correct decision had been forcefully (manually) made at the 5-th step, the final TM-Score that GAPN achieves will been 0.83, which explains the effectiveness of our model.

[Cons2. Exploration efficiency] Thanks for the valuable suggestion.

Empirical analysis: We present numbers of episodes required for convergence when training on datasets with different $N$ (chain numbers).

We will include the detailed convergence curves to our revised manuscript later for better presentation. It can be observed that $N$ has a significant impact on the convergence speed of our model, which is due to the increase in its total action space $N^{N-2}$. Importantly, we find that our model often converges within a small number of episodes. For example, when $N=10$, the entire action space is $10^8$, and our model converges within 1254 episodes.

Theoretical analysis: We have drawn from some existing studies [R1,R2] to demonstrate that the Proximal Policy Optimization (PPO) framework instantiated with neural networks can have local or global convergence guarantee under certain assumptions. We will add these theorems to the revised manuscript.

Kindly note that all reinforcement learning (RL) models using deep neural networks cannot rely on theory to absolutely guarantee convergence, as RL models in real-world practice like our protein complexes assembly application may not meet the strict assumptions and may be affected by many factors [R3,R4], such as learning rate, training set, network structure design, discount factor, replay buffer size and reward function design. However, the successful application of the actor-critic algorithm like PPO in a large number of practical problems with vast action spaces [R5,R6] indicates that it can usually converge efficiently to good policies in practice.