Response to Reviewer qf83

We thank the reviewer for the insightful and valuable comments. We respond to each comment as follows and sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our submission.

Weakness 1.

1. The paper does not formally introduce the logic synthesis problem from input-output examples and the notion of differentiable neural architecture search.

Thanks for the valuable suggestions. We have revised the Background Section as follows.

Formulation of LS from IO examples In recent years, synthesizing circuits from IO examples has gained increasing attention [1][2][3]. Specifically, researchers aim to use machine learning to generate a circuit based on a truth table that describes the circuit's functionality. Each line in the truth table represents an input-output pair, indicating the output produced by the circuit for a given input. In the machine learning domain, researchers formulate the truth table as a training dataset comprising many input-output pairs and use an ML model to generate circuits that accurately fit the dataset.

DNAS for LS from IO Examples Recent works [1][2] propose leveraging traditional DNAS methods for generating circuit graphs from IO examples, showing a promising direction for next-generation logic synthesis. Specifically, they formulate a neural network as a circuit graph, where each neuron represents a logic gate and connections between neurons represent wires connecting these logic gates. For a parameterized neural network, the neurons are fixed as logic gates, and the connections between neurons are parameterized as learnable parameters. To enable differentiable training via gradient descent, continuous relaxation is introduced into discrete components of the neural network. First, the logical operations of logic gates (neurons) are translated into their differentiable counterparts. For example, $a \ *AND* \ b$ is relaxed to $a\cdot b$ [4]. Second, discrete network connections are parameterized using Gumbel-softmax [5].

2. Another difficulty in following the paper is that the authors refer to the appendix for many aspects of the approach.

Thanks for the valuable suggestion. We will revise our manuscript by retaining the key content in the main text and moving the minor content to the appendix. Specifically, we will update the "Background," "Motivation," and "Method" Sections as follows.

For Background, we present the problem formulation of logic synthesis (LS) from input-output examples, and details of the traditional DNAS approach for LS.

For Motivation, we first present the main challenge of the traditional DNAS for LS that the method struggles to generate circuits accurately, especially for large circuits. We then present two major reasons for this challenge: the curse of skip-connection and the structure bias of circuits.

For Method, we first present the three key modules for neural circuit generation. 1) To reduce the learning complexity of large circuits, we present the multi-label transformation module to decompose a large truth-table (dataset) into several small sub-truth-tables (sub-datasets). 2) To address the curse of skip-connection challenge, we present details of the regularized skip-connections module. 3) To leverage the structure bias of circuits, we present details of the triangle-shaped network architecture. We then present details of our circuit optimization approach and provide the pseudocode of the algorithm.

Weakness 2.

3. Since the authors already compare with the top winners from the IWLS contests, it is not clear to me why they do not evaluate the IWLS benchmarks.

Thanks for the valuable suggestion. We have compared our method with Google DeepMind's contest results on 30 more circuits from the IWLS benchmark as shown in Table 2 in the attached pdf in Global Response. The results show that our method achieves 3.03% node reduction compared with Google DeepMind's contest results.

In addition, the 18 circuits used in the main text are indeed sourced from the IWLS benchmark as well, and we compared our method with Google DeepMind's contest results on these circuits in the main text (achieving 5.36% node reduction).

Question 1:

4. Is the circuit optimization evaluated on top of T-Net or is the input circuit obtained in a different way?

Yes, the results of our method are obtained by optimizing circuits generated using T-Net.

Limitation 1.

5. T-Net is not guaranteed to reach perfect accuracy.

To ensure perfect accuracy in our circuit generation, we applied a legalization method to our final generated circuits as detailed in Appendix D.6.

Open-source code.

6. If the authors were to make their code publicly available this would also be a valuable contribution to the research community.

Yes, we will make our code publicly available once our paper is accepted.

[1] Designing better computer chips. Google DeepMind, 2023, https://deepmind.google/impact/optimizing-computer-systems-with-more-generalized-ai-tools/.

[2] Peter Belcak, et al. Neural combinatorial logic circuit synthesis from input-output examples. NeurIPS Workshop, 2022.

[3] IWLS Programming Contest Series Machine Learning + Logic Synthesis. IWLS, 2024, https://www.iwls.org/contest/.

[4] Petersen, et al. Deep differentiable logic gate networks. NeurIPS, 2018.

[5] Jang, et al. Categorical Reparametrization with Gumble-Softmax. ICLR, 2017.

I would like to thank the authors for their text revisions and the evaluation of the additional IWLS benchmarks. I believe the proposed revisions will improve the presentation of the paper. The evaluation of the additional benchmark further strengthens the experimental results. However, the additional benchmarks are again a subset of the IWLS competition benchmarks. I am still concerned that the method is not compared on the full benchmark set.

Dear Reviewer qf83,

We are writing as the authors of the paper "Towards Next-Generation Logic Synthesis: A Scalable Neural Circuit Generation Framework" (ID: 17130).

We sincerely thank you once more for your insightful comments and kind support! We are writing to gently remind you that the deadline for the author-reviewer discussion period is approaching (due on Aug 13). We eagerly await your feedback to understand if our responses have adequately addressed all your concerns. If so, we would deeply appreciate it if you could raise your score. If not, we are eager to address any additional queries you might have, which will enable us to enhance our work further.

Once again, thank you for your guidance and support.

Best,

Authors

I would like to thank the authors for the additional experiments. They provide a full and transparent evaluation of the method. I will raise my score accordingly and encourage the authors to include the results on all IWLS benchmarks into the paper.

Response to Reviewer Qczw

We thank the reviewer for the insightful and valuable comments. We respond to each comment as follows and sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our submission.

Weakness 1.

1. Relevant methods and essential details are pushed into the appendix.

Please see Global Response 1.

Weaknesses 2 and 5.

2. The motivation that "DNAS has weaknesses such as producing excessive skip-connections" is misleading.

Recent works [1,2] have applied traditional DNAS methods, such as DARTS [3], to circuit generation, revealing a promising direction. Our motivating insight revisits the direct application of traditional DNAS methods to circuit generation and identifies several specific challenges for circuit generation, including the curse of skip-connections. Note that the skip-connection problem in circuit generation is significantly different from that in traditional DNAS (see the next response).

3. This "curse of skip connection" is already addressed by methods like Darts.

Differences in the Skip-Connection Challenge In traditional DNAS, the skip-connection problem arises as the skip-connection operation often dominates other operations, such as convolution and zero operations. In contrast, when applying DNAS to circuit generation, a neural network is formulated as a circuit graph, where each neuron represents a logic gate and connections between neurons represent wires connecting these logic gates. The neurons are fixed as logic gates, and the connections between neurons are learnable. Most learnable connections skip layers, called skip-connections in circuit neural networks. In this paper, we found that the traditional DNAS method tends to overfit to skip-connections that bypass a large number of layers, a phenomenon we call the curse of skip-connections in circuit generation. (1) The definitions of skip-connections in DNAS and circuit generation are different. (2) In traditional DNAS, it is required to encourgae balance between skip-connection operation and other operations. In contrast, it is required to balance between numerous skip-connections that span different layers in circuit generation.

Inapplicability of Existing DNAS Methods to Circuit Generation We have compared our method with DNAS-based methods for addressing the skip-connection challenge, including P-DARTS [4], PR-DARTS [5], and DARTS-ES [6]. As shown in Table 1 in attached pdf in Global Response, the results demonstrate that our method significantly outperforms these approaches. The primary reason is that these methods struggles to balance between numerous skip-connections in circuit generation tasks. In contrast, our method proposes a novel layer-aware regularized skip-connection module, which effectively balances skip-connections that span different layers.

4. The loss adaption and the triangle shape prior improves only relatively little.

We have conducted detailed ablation experiments to show that each of our proposed modules is significant. The results in Tables 3, 4, and 5 in attached pdf in Global Response show that the proposed loss enhances accuracy to 99%, and the T-architecture achieves an average node reduction of 16.9% and a training time reduction of 40.8%.

Weakness 3.

5. The math description is poor. It is also not clear what the indexes p mean.

Thanks for the valuable suggestion. We have revised our math description accordingly. Note that each neuron (And gate) in the circuit neural network has two input signals. The $unit^{l, k, p}$ is a tensor with dimension $l \times K \times 2$. The index $p$ represents the $p^{th}$ input signal of current neuron. Details are as follows.

(Revision) We denote the output of the $k^{th}$ neuron in the $l^{th}$ layer by $**o**^{l,k}$. We denote the $p$-th input of the neuron (NAND gate) $**o**^{l,k}$ by $**i**\_{p}^{l,k}$, where $p \in \{0,1\}$. Note that each neuron $o^{l,k}$ has two inputs $i^{l,k}\_{0}$ and $i^{l,k}\_{1}$, and can be connected to any neuron with layer number smaller than $l$ as its input neuron. We parameterize the connections of each neuron $o^{l,k}$ by a tensor of learnable parameters $\mathbf{\theta}^{l,k} \in \mathbb{R}^{2 \times (l-1) \times K}.$ Each parameter in the tensor $\mathbf{\theta}^{l,k}\_{p,i,j}$ represents the probability of connecting the $j^{th}$ neuron in the $i^{th}$ layer to the $p^{th}$ input of current neuron $o^{l,k}$. The computation of the $p^{th}$ input value for the neuron $o^{l,k}$ takes the form of $i_p^{l, k} :=\sum_{i=0}^{l-1} \sum_{j=1}^{K} o^{i,j} \left[\operatorname{softmax}\left(\mathbf{\theta}^{l, k}\right)\right]_{p,i,j}, p=0,1;\ \ o^{l,k} := 1 - \Pi\_{p=0}^{1} i_p^{l, k}$

5. There is no equation in the main text for the loss and other details.

Thanks for the suggestion. We will provide these equations in the main text.

Weakness 4.

6. Prior approaches to the problem are not well explained in the main text.

Please see Global Response 2.

[1] Designing better computer chips. Google DeepMind, 2023, https://deepmind.google/impact/optimizing-computer-systems-with-more-generalized-ai-tools.

[2] Peter Belcak, et al. Neural combinatorial logic circuit synthesis from input-output examples. NeurIPS Workshop, 2022.

[3] Hanxiao Liu, et al. Darts: Differentiable architecture search. ICLR 2019.

[4] Xin Chen, et al. Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. ICCV 2019.

[5] Pan Zhou, et al. Theory-inspired path-regularized differential network architecture search. NeurIPS 2020.

[6] Zela, et al. Understanding and Robustifying Differentiable Architecture Search. ICLR 2020.

Response to Reviewer PMQo

We thank the reviewer for the insightful and valuable comments. We respond to each comment as follows and sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our submission.

Weakness 1 & Question 1.

1. The paper does not provide results for specific cases such as Espresso 1, Espresso 2, Espresso 5, and Espresso 6. Please include more cases from the four benchmarks (Espresso, LogicNets, Random, and Arithmetic) to make the experiment more convincing

Thanks for the valuable suggestion. We indeed conducted experiments on circuits such as Espresso 1, Espresso 2, Espresso 5, and Espresso 6 as shown in Appendix E.2 (Tables 8, 9, and 10) in the main text. The results demonstrate that our method improves the average generation accuracy by 17.5%, reduces the number of generated nodes by 33.4%, and reduces the optimized circuit size by 68.7% on these cases compared to traditional baselines.

In addition, we have conducted experiments on 30 more circuits from the IWLS benchmark. Please see the next response for details.

Weakness 2 & Question 2.

2. Please provide a comparison with Google DeepMind's DNAS Skip on the IWLS benchmark.

Thanks for the valuable suggestion. We have compared our method with Google DeepMind's contest results on 30 more circuits from the IWLS benchmark as shown in Table 2 in the attached pdf in Global Response. The results show that our method achieves 3.03% node reduction compared with Google DeepMind's contest results. For your convenience, we quote Table 2 as follows.

In addition, the 18 circuits used in the main text are indeed sourced from the IWLS benchmark as well, and we compared our method with Google DeepMind's contest results on these circuits in Tables 3 and 10 in the main text (achieving 5.36% node reduction).

Table 2. We evaluate our approach on 30 other circuits in the IWLS benchmark. Our T-Net surpasses the traditional method by 10.64%, and surpasses the IWLS 2023 champion, Google DeepMind, by 3.03% after optimization.

Response to Reviewer gSL5

We thank the reviewer for the insightful and valuable comments. We respond to each comment as follows and sincerely hope that our rebuttal could properly address your concerns. If so, we would deeply appreciate it if you could raise your score. If not, please let us know your further concerns, and we will continue actively responding to your comments and improving our submission.

Weakness 1.

1. The multi-label transformation and regularized loss functions added additional complexity and reduces the efficiency of the model.

We have conducted experiments to demonstrate that our proposed modules not only enhance accuracy significantly but also substantially reduce training time.

• The multi-label transformation module can significantly reduce input complexity by decomposing the entire truth-table (dataset) into several small sub-truth-tables (sub-datasets). As shown in Table 4 (in the main text), experiments show that using multi-label transformation reduces the wrong bits by 90%, and reduces the training time by roughly 40%.
• Our proposed boolean hardness-aware loss also significantly enhanced the efficiency of our circuit generation. As shown in Table 5 in the attached pdf in Global Response, this loss function reduces training time by 20% while preserving accuracy. For your convenience, we quote Table 5 as follows.

Table 5. The ablation study demonstrates that the boolean hardness-aware loss significantly reduces training time by 19.9%. The reported time is the training time required to achieve 99% accuracy.

Weakness 2.

2. More justifications are required for the evolutionary algorithm + reinforcement learning method for the circuit optimization process.

Circuit optimization is recognized as an NP-hard problem, where traditional methods, often greedy and local, struggle to achieve optimal solutions. Reinforcement learning (RL) has demonstrated robust capabilities in navigating extensive search spaces, prompting the exploration of RL-based strategies to identify optimal sequences of circuit optimization operations. Nevertheless, due to the expansive and irregular nature of the search space, RL can suffer from limited exploratory capabilities, often resulting in convergence to local optima. To address this challenge, we incorporate evolutionary algorithms (EA), which preserve a population of diverse circuit solutions, thereby enhancing the exploration of the search space and leading to the discovery of superior circuits. Additionally, as shown in Table 3 in the main text, our novel circuit optimization approach outperforms previous state-of-the-art optimization methods.

3. minor: extra space after line 384 "T"

Thanks for pointing out this typo. We will correct it accordingly.

Question 1.

4. How the proposed T-Net compare to previous SOTA in terms of model size, latency, energy efficiency etc?

Table 6 in the attached pdf in Global Response presents a comparison between T-Net and the SOTA DNAS (PR-DARTS [1]) in terms of model size, latency, and training time. The table shows that our method achieves perfect accuracy with 29% shorter training time. Additionally, our parameters, model size, and latency are comparable to the SOTA. For your convenience, we quote Table 6 as follows.

Table 6. Comparation of T-Net and the SOTA DNAS baseline in terms of model efficiency. The table shows that our method achieves perfect accuracy with 29% shorter training time. Additionally, our parameters, model size, and latency are comparable to the SOTA.

[1] Pan Zhou, et al. Theory-inspired path-regularized differential network architecture search. NeurIPS, 2020.

Question 2.

5. Why no parameter for Appendix Table 7 &st?

Thanks for the question. Indeed, the &st operator does not possess tunable hyperparameters. We will update the table to indicate 'N/A' for this operator's parameter in the revised manuscript.

Dear Reviewer gSL5,

We are writing as the authors of the paper "Towards Next-Generation Logic Synthesis: A Scalable Neural Circuit Generation Framework" (ID: 17130).

We sincerely thank you once more for your insightful comments and kind support! We are writing to gently remind you that the deadline for the author-reviewer discussion period is approaching (due on Aug 13). We eagerly await your feedback to understand if our responses have adequately addressed all your concerns. If so, we would deeply appreciate it if you could raise your score. If not, we are eager to address any additional queries you might have, which will enable us to enhance our work further.

Once again, thank you for your guidance and support.

Best,

Authors