DeepGate4: Efficient and Effective Representation Learning for Circuit Design at Scale

Q6: The sensitivity analysis of hyperparameters is missing. Do the authors use the same hyperparameters across all experiments? It would be more convincing if the authors could conduct sensitivity analysis on the hyperparameters.

Answer During evaluation, to ensure the fair comparsion, we keep all hyperparameters the same and we compute the mean and std in last 5 epochs to get stable result. Furthermore, we provide an ablation study on $k$ and $δ$, illustrating the sensitivity of our model to these hyperparameters:

Here is our observations:

• Our method is insensitive to $δ$ (note that overlap level is $k - \delta + 1$). Settings such as $(k=8, \delta=8)$, $(k=8, \delta=6)$, and $(k=8, \delta=4)$ demonstrate that our method is not sensitive to overlap ratios, as performance across these settings is similar.
• Settings such as $(k=8, \delta=6)$, $(k=10, \delta=8)$, and $(k=6, \delta=4)$ maintain the same overlap level but vary in subgraph size. Results demonstrate that increasing $k$ significantly impacts GPU memory usage. Furthermore, larger $k$ will degrade structural task performance. This is because structural tasks rely more heavily on local information, especially for metrics like $L_{graph}^{ged\_pair}$, $L_{graph}^{size}$, and $L_{graph}^{depth}$ (See Section A.2).

Q5: The experiments do not apply the representation method to downstream tasks

Answer We have add two downstream tasks, Logic Equivalence Checking(LEC) and Boolean Satisfiability Problem(SAT), in Section A.5 and A.6.

Logic Equivalence Checking(LEC) Logic Equivalence Checking (LEC) is a prominent Formal Verification tasks, which determines whether two given designs are functionally equivalent. As circuit complexity continues to grow, the importance of LEC increases, as design errors in these complex systems can result in expensive fixes or operational failures in the final product.

We evaluate LEC on ITC99 Dataset. We randomly cut subcircuits with multiple PI and one PO from ITC99 evaluation dataset. Given a subciruit pair$(G_1,G_2)$, the model will perform a binary classification task to predict whether $G_1$ and $G_2$ is equivalent or not. In the candidate pairs sequence, only 1.29% of pair is equivalent. We evaluate the performance with widely used metrics Average Precision(AP) and Precision-Recall Area Under the Curve(PR-AUC), which are regardless of threshold and are particularly useful in scenarios with imbalanced datasets (e.g., when one class is much rarer than the other).

Note that DeepGate3$^*$ denotes that we use our proposed updating strategy and training pipeline. DeepGate4 outperforms all other methods by a significant margin, achieving the highest AP (0.31) and PR-AUC (0.30), and improve these two metrics by 55% and 42% respectively, compared to the second-best method. These values indicate its superior ability to balance precision and recall, especially in scenarios with imbalanced data.

Boolean Satisfiability Problem(SAT) The Boolean Satisfiability (SAT) problem is a fundamental computational problem that determines whether a Boolean formula can evaluate to logic-1 for at least one variable assignment. As the first proven NP-complete problem, SAT serves as a cornerstone in computer science, with applications spanning fields such as scheduling, planning, and verification. Modern SAT solvers primarily utilize the conflict-driven clause learning (CDCL) algorithm, which efficiently handles path conflicts during the search process and explores additional constraints to reduce the search space. Over the years, various heuristic strategies have been developed to further accelerate CDCL in SAT solvers.

We follow the setting in DeepGate2. We utilize the CaDiCal SAT solver as the backbone solver and modify the variable decision heuristic based on it. In the Baseline setting, SAT problems are directly solved using the backbone SAT solver. For model-acclerated SAT solving, given a SAT instance, the first step is to encode the corresponding AIG to get the gate embedding. During the variable decision process, a decision value $d_i$ is assigned to variable $v_i$. If another variable $v_j$ with an assigned value $d_j$ is identified as correlated to $v_i$, the reversed value $d_j'$ is assigned to $v_i$, i.e., $d_i = 0\ if\ d_j = 1$ or $d_i = 1\ if\ d_j = 0$. The determination of correlated variables relies on their functional similarity, and the similarity $Sim(v_i, v_j)$ exceeding the threshold $\theta$ indicates correlation.

Model Runtime(s)

SAT Solving Time(s)

Since SAT solving is time-consuming, we compare our approach only with the top-3 methods listed in Table 1 in our paper, namely DeepGate3*, Exphormer*, and PolarGate. The Baseline means using the SAT solver without any model-based acceleration. Leveraging its exceptional ability to understand the functional relationships within circuits, DeepGate4 achieves a substantial reduction in SAT solving time, with an 86.33% reduction for case f20 and an 92.90% reduction for case ac1. Regarding average solving time, it achieves a 56.56% reduction, outperforming all other methods. These results highlight DeepGate4’s generalization capability and effectiveness in addressing real-world SAT solving problems.

Q3: The experiments do not compare their method with a recent SOTA method HOGA that is able to scale to large-scale circuits.

Answer We have added HOGA in Tab1 and Tab3. The result demonstrate that DeepGate4 reduce the overall loss by 54.79% on ITC99 and 39.72% on EPFL respectively, compared to HOGA.

Table1

Table3

We further include a comparison on the OpenABC-D benchmark in Sec. A.4. Details of the dataset statistics are provided in Sec. A.1. Due to the time-consuming process of preparing labels for graph-level tasks, we evaluate only on gate-level tasks. The results are as follows:

• Comparison on Effectiveness DeepGate4 demonstrates outstanding effectiveness across all training tasks. It achieves state-of-the-art performance on all gate-level tasks within the OpenABC-D datasets. Notably, DeepGate4 reduces the overall loss by 31.48% compared to the second-best method. Moreover, while baseline models struggle with gate connection prediction, DeepGate4 significantly enhances performance in this area, achieving an accuracy of 79%. This highlights the outstanding ability of DeepGate4 to capture the structural relationships between gates.
• Comparison on Efficiency In terms of efficiency, models like PNA and HOGA-5 encounter out-of-memory (OOM) errors, whereas DeepGate4 can successfully train a graph transformer on large circuits containing over 500k gates.

Q4: The evaluation metrics are unclear. The experiments report many metrics. However, a description of these metrics is missing.

Answer In this work, we follow the setting in DeepGate3, and due to the space limitation, we give a detail definition in Sec A.2. These metrics can be summarized into two types:

• MAE For regression tasks, like predicting the logic-1 probability, we use MAE as metrics, e.g. $MAE=\|pred-label\|_1$
• Accuracy For classfication task, like Gate Connection Prediction, we use Accuracy as metrics, e.g. $Acc=\frac{1}{N}\sum_{i=1}^N 1(pred,label)$.

We appreciate the reviewer’s suggestion to improve our works. Here are our answers to the weaknesses and questions.

Q1 and Q3: It would be more readable if the authors could provide a summary of contributions. The novelty of the proposed method is unclear. The authors incorporates many modules into previous circuit representation learning method. However, the novelty and contribution of these modules seems limited.

Answer We have modified Abstract and Introduction, and summarized our contribution as follow:

• An updating strategy tailored for DAGs based on partitioned graph, ensuring that gate embeddings are computed in logical level order, with each gate being processed only once, thus eliminating redundant computations. Note that even with the graph partition, DeepGate3 is still limited to fine-tuning graphs with up to 50k nodes, while our proposed updating strategy, which is adaptable to any graph transformer, achieve sub-linear memory complexity and thus enable efficient training on graphs with millions even billions of nodes.
• A GAT-based sparse transformer with global virtual edges, reducing both time and memory complexity to linear in a mini-batch. We further introduce structural encodings for transformers on AIGs by incorporating global and local structural encodings in initialized embedding.
• An inference acceleration kernel, Fused-DeepGate4, designed to optimize the inference process of tokenizer and GAT components with well-designed CUDA kernels that fully exploit the unique sparsity patterns of AIGs.

Here we want to strength that the updating strategy is adaptable to any graph transformers. As shown in Tab. 1, GraphGPs, Exphormer, DAGformer and DeepGate3 will suffer from Out-of-Memory when training on large circuits. We further found training GNNs like PNA and HOGA-5 on OpenABC-D[1] on one Nvidia L40 GPU will also encounter OOM error. This denotes that scaling to large circuits still remains a challenge. However, our proposed updating strategy enables training these method on large circuits with sub-linear memory complexity. Futhermore, the ablation study in Tab.4 has demonstrated the effectiveness of our porposed module.

[1] Chowdhury A B, Tan B, Karri R, et al. Openabc-d: A large-scale dataset for machine learning guided integrated circuit synthesis[J]. arXiv preprint arXiv:2110.11292, 2021.

Q2: The method description is unclear. For example, the graph partition module and the multi-task training objective is unclear.

Answer

• Graph Partititon The graph paritition algorithm is included in Algor.1. We follow the graph partition used in DeepGate3[1], therefore, we don't give a detailed descrption in our paper. To summarize, we sample specific nodes with a stride $\delta$, and cut cones for these nodes with depth $k$. Initially, we focus on gathering all the cones, denoted as $cone^k_v$, that terminate at logic level $k$. A cone is defined as:$cone^k_v = \{u \in V : u \preccurlyeq_k v\}$, where $u \preccurlyeq_k v$ signifies that $u$ is a predecessor of $v$ within $k$ logic levels. After collecting cones at level $k$, we proceed in strides of $\delta$. Specifically, we gather additional cones whose output gates are situated at level $k + \delta$. This stride-based progression allows us to partition the circuit iteratively, moving step-by-step through its logic levels. This iterative process continues until all partitioned areas collectively cover the entire circuit. By systematically partitioning the circuit into cones, we ensure that each section is analyzed within its local logic level context while maintaining global structural coherence.
• Training Objective We follow the setting in DeepGate3[1], and due to the space limitation, we give a detail definition in Sec A.2. These training objectives can be summarized into two types:
  • Regression tasks For regression tasks, like predicting the logic-1 probability, we use L1-Loss as metrics, e.g. $Loss=\|pred-label\|_1$, where $pred$ is the predicted value and $label$ is the true value.
  • Classfication tasks For classfication tasks, like Gate Connection Prediction, we use Binary Cross-Entropy Loss(BCE Loss), e.g. $L_{\text{BCE}} = - \frac{1}{N} \sum_{i=1}^{N} \left[ y_i \log(p_i) + (1 - y_i) \log(1 - p_i) \right]$, where $N$ is the number of samples, $y_i$ is the true label for the $i$-th sample and $p_i$ is the predicted probability for the $i$-th sample.

[1] Z. Shi, Z. Zheng, S. Khan, J. Zhong, M. Li, and Q. Xu. DeepGate3: Towards scalable circuit representation learning. arXiv:2407.11095.

Dear Reviewer Avwf,

Thank you for your thoughtful feedback on our paper. We greatly appreciate the time and effort you have invested in reviewing our work.

As today is the final day for paper revisions, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns or if there are any remaining points we can clarify. Your insights would be invaluable in helping us fully address your concerns and refine our work.

We greatly appreciate your consideration. Please feel free to let us know if there are any specific points you would like us to discuss further.

Thank you again for your valuable time and feedback.

Best regards,

Authors

Boolean Satisfiability Problem(SAT) The Boolean Satisfiability (SAT) problem is a fundamental computational problem that determines whether a Boolean formula can evaluate to logic-1 for at least one variable assignment. As the first proven NP-complete problem, SAT serves as a cornerstone in computer science, with applications spanning fields such as scheduling, planning, and verification. Modern SAT solvers primarily utilize the conflict-driven clause learning (CDCL) algorithm, which efficiently handles path conflicts during the search process and explores additional constraints to reduce the search space. Over the years, various heuristic strategies have been developed to further accelerate CDCL in SAT solvers.

We follow the setting in DeepGate2[1]. We utilize the CaDiCal[2] SAT solver as the backbone solver and modify the variable decision heuristic based on it. In the Baseline setting, SAT problems are directly solved using the backbone SAT solver. For model-acclerated SAT solving, given a SAT instance, the first step is to encode the corresponding AIG to get the gate embedding. During the variable decision process, a decision value $d_i$ is assigned to variable $v_i$. If another variable $v_j$ with an assigned value $d_j$ is identified as correlated to $v_i$, the reversed value $d_j'$ is assigned to $v_i$, i.e., $d_i = 0\ if\ d_j = 1$ or $d_i = 1\ if\ d_j = 0$. The determination of correlated variables relies on their functional similarity, and the similarity $Sim(v_i, v_j)$ exceeding the threshold $\theta$ indicates correlation.

Model Runtime

SAT Solving Time

Since SAT solving is time-consuming, we compare our approach only with the top-3 methods listed in Table 1 in our paper, namely DeepGate3*, Exphormer*, and PolarGate. The Baseline represents using the SAT solver without any model-based acceleration. Leveraging its exceptional ability to understand the functional relationships within circuits, DeepGate4 achieves a substantial reduction in SAT solving time, with an 86.33% reduction for case f20 and an 92.90% reduction for case ac1. Regarding average solving time, it achieves a 56.56% reduction, outperforming all other methods. These results highlight DeepGate4’s strong generalization capability and effectiveness in addressing real-world SAT solving challenges.

[1] Z. Shi, H. Pan, S. Khan, M. Li, Y. Liu, J. Huang, H. Zhen, M. Yuan, Z. Chu, and Q. Xu. DeepGate2: Functionality-aware circuit representation learning. 2023 IEEE/ACM ICCAD, IEEE, 2023.

[2]CaDiCaL 2.0, Armin Biere, Tobias Faller, Katalin Fazekas, Mathias Fleury, Nils Froleyks and Florian Pollitt, Proc. Computer Aidded Verification - 26th Intl. Conf. (CAV'24), Lecture Notes in Computer Science (LNCS), vol. 14681, pages 133-152, Springer 2024

Q4 and Q5: This paper uses two old datasets, but the data are insufficient for such a large-scale model, and the results provided are not solid enough. Conducting the experiments on modern, larger-scale datasets/netlists(e.g., [2], [3]) would significantly enhance the value of the paper. Conducting the experiments on modern, larger-scale datasets/netlists(e.g., [2], [3]) would significantly enhance the value of the paper.

[2] Chai, Zhuomin, et al. "Circuitnet: An open-source dataset for machine learning in vlsi cad applications with improved domain-specific evaluation metric and learning strategies." IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 42.12 (2023): 5034-5047.

[3] https://github.com/TILOS-AI-Institute/MacroPlacement

Answer

Currently, our method are designed for AIG netlists, and we haven't started to transfer it to other modalies, like post-mapping netlist[2] and macroplacement[3]. However, we find a recent pubilshed AIG benchmark OpenABC-D[1] with large-scale AIG averaging 91k gates.

We have included a comparison on the OpenABC-D benchmark in Sec. A.4. Details of the dataset statistics are provided in Sec. A.1. Due to the time-consuming process of preparing labels for graph-level tasks, we evaluate only on gate-level tasks. The results are as follows:

• Comparison on Effectiveness DeepGate4 demonstrates outstanding effectiveness across all training tasks. It achieves state-of-the-art performance on all gate-level tasks within the OpenABC-D datasets. Notably, DeepGate4 reduces the overall loss by 31.48% compared to the second-best method. Moreover, while baseline models struggle with gate connection prediction, DeepGate4 significantly enhances performance in this area, achieving an accuracy of 79%. This highlights the outstanding ability of DeepGate4 to capture the structural relationships between gates.
• Comparison on Efficiency In terms of efficiency, models like PNA and HOGA-5 encounter out-of-memory (OOM) errors, whereas DeepGate4 can successfully train a graph transformer on large circuits containing over 500k gates.

Q2 and Q3: The novelty of this paper is limited. Previously, Deepgate3 also proposed a partitioning strategy for large-scale circuits, which weakens the contribution of the partitioning method. What's the difference between the two partitioning strategies used in Deepgate3 and Deepgate4. It would be better to discuss more about this.

Answer

We summarized our contribution as follow:

• An updating strategy tailored for DAGs based on partitioned graph, ensuring that gate embeddings are computed in logical level order, with each gate being processed only once, thus eliminating redundant computations. Note that even with the graph partition, DeepGate3 is still limited to fine-tuning graphs with up to 50k nodes, while our proposed updating strategy, which is adaptable to any graph transformer, achieve sub-linear memory complexity and thus enable efficient training on graphs with millions even billions of nodes.
• A GAT-based sparse transformer with global virtual edges, reducing both time and memory complexity to linear in a mini-batch. We further introduce structural encodings for transformers on AIGs by incorporating global and local structural encodings in initialized embedding.
• An inference acceleration kernel, Fused-DeepGate4, designed to optimize the inference process of tokenizer and GAT components with well-designed CUDA kernels that fully exploit the unique sparsity patterns of AIGs.

Here we want to strength that the updating strategy is adaptable to any graph transformers. As shown in Tab. 1, GraphGPs, Exphormer, DAGformer and DeepGate3 will suffer from Out-of-Memory when training on large circuits. We further found training GNNs like PNA and HOGA-5 on OpenABC-D[1] on one Nvidia L40 GPU will also encounter OOM error. This denotes that scaling to large circuits still remains a challenge. However, our proposed updating strategy enables training these method on large circuits with sub-linear memory complexity. Futhermore, the ablation study in Tab.4 has demonstrated the effectiveness of our porposed module.

[1] Chowdhury A B, Tan B, Karri R, et al. Openabc-d: A large-scale dataset for machine learning guided integrated circuit synthesis[J]. arXiv preprint arXiv:2110.11292, 2021.

We appreciate the reviewer’s suggestion to improve our works. Here are our answers to the weaknesses and questions.

Q1: We hope these methods can be used in practical EDA applications; unfortunately, this paper does not elaborate on how much benefit such methods could bring to actual EDA tasks.

Answer We have add two downstream tasks, Logic Equivalence Checking(LEC) and Boolean Satisfiability Problem(SAT), in Section A.5 and A.6.

Logic Equivalence Checking(LEC) Logic Equivalence Checking (LEC) is a prominent Formal Verification tasks, which determines whether two given designs are functionally equivalent. As circuit complexity continues to grow, the importance of LEC increases, as design errors in these complex systems can result in expensive fixes or operational failures in the final product.

We evaluate LEC on ITC99 Dataset. We randomly cut subcircuits with multiple PI and one PO from ITC99 evaluation dataset. Given a subciruit pair$(G_1,G_2)$, the model will perform a binary classification task to predict whether $G_1$ and $G_2$ is equivalent or not. In the candidate pairs sequence, only 1.29% of pair is equivalent. We evaluate the performance with widely used metrics Average Precision(AP) and Precision-Recall Area Under the Curve(PR-AUC), which are regardless of threshold and are particularly useful in scenarios with imbalanced datasets (e.g., when one class is much rarer than the other).

Note that DeepGate3$^*$ denotes that we use our proposed updating strategy and training pipline. DeepGate4 outperforms all other methods by a significant margin, achieving the highest AP (0.31) and PR-AUC (0.30), and improve these two metrics by 55% and 42% respectively, compared to the second-best method. These values indicate its superior ability to balance precision and recall, especially in scenarios with imbalanced data.

Dear Reviewer YSp1,

Thank you for your thoughtful feedback on our paper. We greatly appreciate the time and effort you have invested in reviewing our work.

As today is the final day for paper revisions, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns or if there are any remaining points we can clarify. Your insights would be invaluable in helping us fully address your concerns and refine our work.

We greatly appreciate your consideration. Please feel free to let us know if there are any specific points you would like us to discuss further.

Thank you again for your valuable time and feedback.

Best regards,

Authors

Dear Reviewer YSp1,

We sincerely appreciate your thoughtful insights and constructive suggestions on our work. As the extended rebuttal deadline approaches, we would be grateful for any further feedback or guidance you may have.

Thank you once again for your time and effort in reviewing our submission. Your expertise and consideration are deeply valued.

Best regards,

Authors

We appreciate the reviewer’s suggestion to improve our works. Here are our answers to the weaknesses and questions.

Q1: Explain the metrics

Answer In this work, we follow the setting in DeepGate3[1], and due to the space limitation, we give a detail definition in Sec A.2. These metrics can be summarized into two types:

• MAE For regression tasks, like predicting the logic-1 probability, we use MAE as metrics, e.g. $MAE=\|pred-label\|_1$
• Accuracy For classfication task, like Gate Connection Prediction, we use Accuracy as metrics, e.g. $Acc=\frac{1}{N}\sum_{i=1}^N 1(pred,label)$.

[1] Z. Shi, Z. Zheng, S. Khan, J. Zhong, M. Li, and Q. Xu. DeepGate3: Towards scalable circuit representation learning. arXiv:2407.11095.

Q2: Clarify the evaluation

Answer All models are evaluated on unseen testing datasets. To ensure a fair comparison, we use identical hyperparameters across all models and compute the mean and standard deviation over the last five epochs to obtain stable results.

Q3: Apply to sequential circuit?

Answer

Currently, we evaluate our method on combinational circuits. However, since the proposed updating strategy is adaptable with any graph transformer on DAGs, we believe it has the potential to be extended to sequential circuits.

Q4: Limitd novelty.

Answer We have modified Abstract and Introduction, and summarized our contribution as follow:

• An updating strategy tailored for DAGs based on partitioned graph, ensuring that gate embeddings are computed in logical level order, with each gate being processed only once, thus eliminating redundant computations. Note that even with the graph partition, DeepGate3 is still limited to fine-tuning graphs with up to 50k nodes, while our proposed intra-level and inter-level updating strategy, which is adaptable to any graph transformer as shown in Tab.1, achieve sub-linear memory complexity and thus enable efficient training on graphs with millions even billions of nodes.
• A GAT-based sparse transformer with global virtual edges, reducing both time and memory complexity to linear in a mini-batch. We further introduce structural encodings for transformers on AIGs by incorporating global and local structural encodings in initialized embedding.
• An inference acceleration kernel, Fused-DeepGate4, designed to optimize the inference process of tokenizer and GAT components with well-designed CUDA kernels that fully exploit the unique sparsity patterns of AIGs.

We thank the reviewer for their valuable feedback. Here are our answers to the weaknesses and questions.

Q1 and Q4:How does the setting of k and d have effects the experiment?

Answer We have added an ablation study on $k$ and $\delta$ in Section A.3 in our paper. These parameters influence memory usage and overlap levels as follows:

• $k$ (Maximum Level): Determines subgraph size, with size always smaller than $2^{k+1}-1$. Larger $k$ increases GPU memory usage significantly.
• $\delta$ (Stride): Defines overlap regions, with overlap level as $k - \delta + 1$.

Furthermore, we provide an ablation study on $k$ and $δ$, illustrating the sensitivity of our model to these hyperparameters:

Here is our observations:

• Our method is insensitive to $δ$, as shown by similar performance across $(k=8, δ=8)$, $(k=8, δ=6)$, and $(k=8, δ=4)$.
• Results on $(k=8, δ=6)$, $(k=10, δ=8)$, and $(k=6, δ=4)$ demonstrate that increasing $k$ impacts GPU memory usage. Furthermore, larger $k$ will degrade structural task since they rely more heavily on local information, especially for metrics like $L_{graph}^{ged\_pair}$ and $L_{graph}^{size}$ (See Sec A.2).

Q2: Can the authors expand more on loss balancer?

Answer The analysis of loss balancer based on gradient norms has been widely studied in various domains[1, 2]. We briefly include an analysis with a simple example:

• Gradient-Based Balancing: Assume there are three losses $L_1$, $L_2$, and $L_3$, and the total loss is $L = L_1 + L_2 + L_3$. During updating, the gradient for model parameter $w$ at iteration $t$ is: $w_{t+1}=w_t-\frac{\partial L_1}{\partial w_t}-\frac{\partial L_2}{\partial w_t}-\frac{\partial L_3}{\partial w_t}$ Here, $\frac{\partial L_i}{\partial w_t}$ represents the contribution of each loss. Since loss scales may significantly due to the difference definition, the loss balancer adjusts weights to equalize gradient contributions:$w_{t+1}=w_t-a_1\frac{\partial L_1}{\partial w_t}-a_2\frac{\partial L_2}{\partial w_t}-a_3\frac{\partial L_3}{\partial w_t}$ where $a_i = \frac{1}{\|\frac{\partial L_i}{\partial w_t}\|_2}$. This ensures the l2-norm of $a_i\frac{\partial L_i}{\partial w_t}$ is the same to balanced updates for each loss term, independent of their scale.
• Why Last-Layer: (1)It ensures the influence of the entire encoder is considered. Balancing earlier layers may ignore gradient contributions from subsequent layers. (2)It is computationally efficient, as balancing earlier layers requires computing gradients for all encoder layers, increasing memory usage and time consumption.

[1]Chen Z. et al., "Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks," ICML, 2018, pp. 794–803. [2]Défossez A. et al., "High fidelity neural audio compression," arXiv preprint, arXiv:2210.13438, 2022.

Q3: The description in section 3.3 is not very clear.

Answer We have modified the Sec 3.3 in our paper for better clarity. We define the receptive field for a node $v$ in a subgraph $cone$ as $\{u\in cone: u\preccurlyeq_k v\}$. In other words, it includes all nodes in the fan-in region of $v$ within the subgraph $cone$.

• Intra-level overalp: As shown in Fig 3(a), the receptive field of node $v$ in the overlap region is identical for both the blue and orange cones, as they share the same fan-in region.
• Inter-level overlap: For node $v$ in the overlap region, the receptive field of the blue cone is the fan-in region within the blue cone, as shown in Fig 3(b), while the receptive field of oragen cone is the fan-in region within the orange cone, as shown in Fig 3(c). The receptive field shown in Fig 3(c) can be regarded as the grean area in Fig 3(b) constrained by the orange cone.

The receptive field affects the computations of the GNN tokenizer and sparse transformer, as they aggregate embeddings within the receptive field for node $v$.

Q5:The figures need to be further explained clearly.

Answer: We have add the caption of the arrows in our paper.

• Fig 1: The black arrows denote edges in the AIG graph, while red arrows indicate message aggregation of DG2.
• Fig 2: The orange arrows denote that we move the node embedding from GPU to CPU, and the blue arrows denote that we move the node embedding from CPU to GPU.