Circuit Representation Learning with Masked Gate Modeling and Verilog-AIG Alignment

Some key information is omitted, not highlighted enough, or not presented in a clear way, which impede a thorough understanding of this paper, including

1. The preservation of logical equivalence is highlighted multiple times (line 74, 90, 101, 531), but with no detailed explanation/proof/example on how this is achieved.
2. How the AIG is reconstructed from latent space representation $\mathbf{X}$? More specifically, if the encoder is $\mathbf{X} = g_E(\mathcal{V}, \mathcal{A})$, why the decoder is not something like $(\mathcal{V}', \mathcal{A}') = g_D(\mathbf{X})$, but formulated as Equation (2) instead? The adjecency matrix $\mathcal{A}$, which I believe should be the output of a typical reconstruction process, serves as the input of Equation (2) without justification, which confuse me a lot. I also checked Section 3.4 but there are only two tasks on the reconstructed circuit, without explanation of how the reconstructed circuit comes out.
3. In line 252, X_V \in \mathbb{R}^{1\times d_v}\ and it works as both key and value, does that mean there is only one key and one value in the cross attention block shown in Figure 4? It is pretty weird to have only a single key-value pair in the cross attention block, since the output will trivially be the value itself.
4. Figure 3 shows the training process of MGVGA, however, it is not clear how MGVGA module (as shown in Figure 5) works in the inference stage. More specifically, Figure 3 includes equivalent Verilog codes for the input outputs, which I hypothesis should not be required during inference, but I cannot confirm it through the paper.
5. In Section 4.2, while the training parameters are provided, the paper does not contain enough details of the model itself. For example, how many layers are included and what is the total number of parameters? Such details are also missing for the baseline method (DeepGate2). Generally speaking, I would like to confirm whether the size of all the compared models are close to each other. For example, in the reserach of LLM, a 7B model is usually compared with other 7B models rather than a 405B model.

1. The motivation of leveraging the masked modeling paradigm for learning circuit representations is unclear.
2. The novelty of the proposed method seems incremental. The authors first apply the masked graph autoencoder to learning circuit representations, and align the learned embedding with LLMs. However, the masked graph autoencoder is a relatively mature technique, and aligning embeddings with LLMs has been widely investigated in previous work [1,2,3,4].
3. Experiments are insufficient. First, some important baselines are missing. It would be more convincing if the authors could compare their method with the recent SOTA method HOGA [5], and the typical GNNs, such as GCN, GAT, etc. Second, the authors may want to conduct experiments on the OpenABC-D benchmark, which is widely used in previous work [5].

[1] Wang, Duo, et al. "LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings." arXiv preprint arXiv:2408.14512 (2024).

[2] Li, Yuhan, et al. "A survey of graph meets large language model: Progress and future directions." arXiv preprint arXiv:2311.12399 (2023).

[3] Radford, Alec, et al. "Learning transferable visual models from natural language supervision." International conference on machine learning. PMLR, 2021.

[4] Liu, Shengchao, et al. "Multi-modal molecule structure–text model for text-based retrieval and editing." Nature Machine Intelligence 5.12 (2023): 1447-1457.

[5] Deng, Chenhui, et al. "Less is More: Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits." DAC 2024.

1. although the framework performs well for circuits of moderate complexity, it is unclear how well it scales to larger circuit representations, especially in terms of maintaining efficient reconstruction performance. The masking techniques could face computational bottlenecks when applied to large-scale circuit datasets.
2. Deepgate3[1] is already release and has significantly better performance than deepgate2. The author should compare to the latest results.

[1] DeepGate3: Towards Scalable Circuit Representation Learning

Figures 3 may benefit from additional legends and annotation perhaps indicating how the model is concurrently trained (details of the loss function).

If Figure 4 was closer to Figure 3 (perhaps below Figure 3) it would help with seeing precisely when the Verilog comes into the picture.

A figure illustrating Qwen-2 and how it is augmented (or plugged-in) to the training process might be needed to clarify for the reader how this is wired in as a teacher, generally the Verilog-AIG section may benefit from additional diagrams depicting how the LLM integrates with the larger training setup.

Also the "bidirectional attention mechanism" may benefit from a figure or a few lines further to discuss this feature of the gte-Qwen2-7B-instruct model chosen for the Verilog-AIG.