Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms

We thank you for your detailed review and constructive feedback. Below, we address your concerns (detailed tables can be found at https://anonymous.4open.science/r/SGSI-Rebuttal-1614/Tables.pdf ):

1. Choice of Evaluation Metrics and Downstream Tasks:

We selected AUROC as our primary metric because it is standard in the structural inference literature, used by ACD, iSIDG, ALaSI, and RCSI, and effectively measures true positive rates over various thresholds, as demonstrated in Pratapa et al. (2020). In our context, where many methods output probabilities rather than hard binary edges, AUROC provides a comprehensive assessment of latent graph recovery.

In addition, although our primary goal is to infer the underlying interacting structure of dynamical systems, SGSI is also applicable to time-series forecasting. For instance, we report Mean Squared Error (MSE) for predicting 10 future steps on several datasets. Table 6 in the attached link compare NRI, SGSI without prior knowledge (SGSI raw), and SGSI with 20% known present edges.

These results show that while SGSI raw matches NRI, the integration of prior knowledge significantly improves forecasting accuracy. This dual capability demonstrates that our method is not only effective for structural inference but also beneficial for downstream prediction tasks.

2. NRI Variants

Actually, we use both variants of NRI (with or without sparsity regularization), but figured out the the one with sparsity regularization outperformed the other variant by 1%-3% AUROC. So we refer to the one with sparsity regularization. We revised our paper to state it clearer.

3. Technical Contribution

Modern autograd frameworks track tensor “versions” to compute gradients accurately. In-place masking, forcing gating values to 0 or 1 within the same tensor, disrupts this mechanism, causing version mismatches and unstable gradients. Our clone-and-clamp strategy first duplicates the gating vector, then clamps the copy, thereby preserving the original tensor’s autograd history. This ensures that (1) the original logits $\theta_e$ remain intact for backpropagation, enabling stable gradient flow, and (2) only the cloned, clamped tensor is used in message passing.

Our ablation experiments (Section 5.3) clearly show that omitting cloning leads to sharp gradient spikes and oscillatory training, while not skipping the KL for pinned edges results in contradictory updates that inflate the loss and harm AUROC. These findings confirm that simply masking in-place degrades gradient quality, whereas our clone-and-clamp technique maintains stable and consistent gradient propagation.

4. Limitations of the Learnable Gate

Our soft-gating mechanism enables the use of roughly estimated prior knowledge, such as overall sparsity or node degree constraints, that is often stable and transferable within our domain. In many application areas (e.g., transportation networks, gene regulatory networks), the underlying graph structures exhibit similar patterns. Thus, the partial knowledge (e.g., a typical density or expected node degree) is largely sharable across datasets. Moreover, our design allows the gating penalties to be adjusted; for instance, one can fine-tune the model through a pretraining and fine-tuning process if needed.

As detailed in our response to Reviewer vBqs, SGSI is robust against moderate errors or deviations in prior knowledge. Although the penalties (e.g., the sum of gating values for sparsity or node-level degree sums) might appear straightforward, their true strength lies in their seamless integration with our variational framework:

1. They are soft: the model can override them when data strongly suggests an alternative structure.
2. They are partial: penalties can be applied selectively, only to nodes or edges where reliable prior knowledge exists.
3. They co-exist with our skip-KL mechanism for pinned edges, ensuring that enforcing known edges does not conflict with the overall latent representation.

This flexibility not only facilitates the transfer of prior knowledge across related domains but also makes SGSI adaptable to different datasets within the same field. We note that approaches like META-NRI have similarly emphasized leveraging common structural priors across domains, further supporting our design choices.

In summary, our soft gating penalties act as additional, adjustable levers that exploit stable, transferable domain knowledge, ensuring that SGSI remains effective whether prior knowledge is abundant or only approximately known. We believe this design is both practical and powerful, and we are confident that these clarifications strengthen the final manuscript.

We thank you for your detailed review and the constructive feedback regarding SGSI. We address your concerns below (tables can be found at https://anonymous.4open.science/r/SGSI-Rebuttal-1614/Tables.pdf ):

1. Importance of Prior Knowledge

Our experiments across multiple benchmarks, including NetSim, Spring Simulations, and StructInfer, demonstrate that even a small fraction of reliable prior knowledge (e.g., 20% known-present or known-absent edges) can improve latent structure recovery by up to 9% AUROC in LL and several VN_NS datasets. We acknowledge that when available domain knowledge is sparse or less reliable, the improvements are naturally smaller. However, in many real-world scenarios (e.g., transportation networks, medical diagnostics), partial prior knowledge is both available and critical for ensuring interpretability and reliability. Importantly, SGSI is designed to gracefully revert to a standard VAE-based method when no prior is available, so its performance scales with the quality of the available knowledge.

2. Contribution of SGSI

We respectfully disagree that our contributions are merely incremental. Our method introduces a novel soft gating mechanism that:

• Clones and clamps gating parameters to integrate known edges without causing in-place gradient conflicts.
• Skips KL costs for fully known edges, effectively reallocating “bits” to uncertain connections.
• Integrates domain-specific constraints (global sparsity and node degrees) via soft penalties.

These design choices overcome critical challenges in merging data-driven latent inference with external knowledge, a capability absent in prior work. For instance, in gene regulatory network (GRN) inference, extensive experimental efforts identify a subset of true regulatory interactions. We evaluated SGSI on two GRN datasets, ventral spinal cord (VSC) development and gonadal sex determination (GSD) (Pratapa et al., 2020), using varying proportions of known-present edges. Table 5 in the attached link summarizes the results. The results show that incorporating prior knowledge not only improves AUROC but also helps the model produce more true positive edges, capabilities that are missing in prior approaches. We further foresee SGSI’s potential in fields such as physics, chemistry, and finance, although resource limitations preclude experiments on those domains at present.

3. Hyperparameter Sensitivity

SGSI introduces additional hyperparameters (e.g., $\beta, \lambda_{\mathrm{sparsity}}$, and $\lambda_{\mathrm{deg}}$). In experiments, we selected these values using preliminary grid searches and cross-validation on a validation set. Our recommended ranges ($\beta=1.0$ and $\lambda_{\mathrm{sparsity}}, \lambda_{\mathrm{deg}} \in [10^{-4},10^{-2}]$) are consistent with common practices in VAE-based models. We plan to include additional guidance in the Appendix:

• For very sparse graphs (e.g., local interactions in physical systems), higher $\lambda_{\mathrm{sparsity}}$ values are tested.
• When node-degree constraints are precise (e.g., “exactly 2 neighbors”), we set a higher $\lambda_{\mathrm{deg}}$.
• If the prior knowledge is approximate, lower penalty weights prevent overshooting data evidence.

These ranges avoid abrupt changes in adjacency, ensuring that SGSI functions as a softly guided VAE rather than a rigidly constrained model.

4. Theoretical Analysis and VIB Connection

Our VIB-inspired design, where known edges are excluded from the KL term, frees the model to devote its “bits” to uncertain edges, leading to notably better performance. Our ablation studies show that not skipping the KL for pinned edges degrades performance, confirming that when the model wastes capacity on fully known edges, it achieves lower accuracy. This aligns with the broader rationale behind VAE-based structural inference (e.g., ACD, iSIDG, RCSI), in which the Variational Information Bottleneck concept helps explain why prioritizing uncertain edges yields more effective adjacency discovery. While we do not claim a formal VIB theorem, we include this perspective because it clarifies the practical benefit of our skip-KL approach and highlights its theoretical consistency with known VAE methodologies.

5. Typos

We appreciate your careful reading. All typographical errors have been corrected and will appear in the camera-ready revision.

6. Datasets

In addition to the synthetic and benchmark datasets discussed in the main text, we evaluated SGSI on the PEMS (California Caltrans Performance Measurement System) dataset (see Appendix E.2). The PEMS evaluation, conducted on real-world sensor data, further demonstrates SGSI’s practical applicability and robustness in realistic settings.

We believe that SGSI’s novel soft gating mechanism significantly enhances structural inference by effectively leveraging prior knowledge while maintaining computational efficiency and scalability. Thank you for your valuable feedback.

We appreciate your constructive feedback regarding our paper SGSI. Below, we address your concerns point by point:

1. Marginal Gains on Certain Datasets

While SGSI shows up to 9% AUROC improvement on some datasets, other cases yield more modest gains (1–2%). We believe this reflects two factors: (i) the inherent difficulty or limited availability of reliable partial knowledge in those tasks, and (ii) that some baselines (e.g., in Spring Simulations) already achieve high AUROC, leaving limited room for further improvement. Importantly, even small improvements can be significant for domain practitioners, especially given SGSI’s additional benefits in interpretability (via clamped edges) and stable gradient flow. Moreover, when no prior knowledge is available, SGSI effectively reverts to a standard VAE-based approach (similar to NRI), explaining the smaller gains in those scenarios. We believe this variation reflects two factors:

1. Domain Impact:

In domains where partial prior knowledge is available, such as transportation networks or biomedical systems, a substantial gain (up to 9% AUROC) can lead to significantly improved decision-making and reliability. For example, in critical applications like traffic management or clinical diagnostics, a 9% improvement in edge prediction accuracy can meaningfully enhance system interpretability and safety.

2. Baseline Ceiling Effects and Additional Benefits:

In some tasks like Spring Simulations, the baselines already achieve high AUROC scores, leaving little room for large numerical gains. Even when improvements are modest (1–2%), these small gains are significant because they come with additional benefits:

• Enhanced Interpretability: The clone-and-clamp mechanism produces a more interpretable latent graph by clearly separating known from uncertain edges.
• Stable Gradient Flow: Our design avoids in-place modifications and contradictory KL signals, leading to more stable and robust training.
• Adaptive Behavior: SGSI gracefully reverts to a standard VAE-based approach (e.g., NRI) when no prior knowledge is available, ensuring that even minimal domain guidance is exploited without harming performance.

Thus, even modest gains are valuable in high-performing scenarios and are complemented by improved interpretability and training stability, factors that are crucial for practical applications. We believe these advantages underscore the practical significance of SGSI beyond mere numerical improvements.

2. Scalability to Larger Real-World Graphs

We agree that demonstrating scalability is important. However, obtaining a reliable dynamical system dataset with thousands of nodes and a trusted underlying structure remains challenging in our field. To address this, we conducted simulation-based experiments on a synthetic dataset with 5k nodes using subgraph sampling (akin to GraphSAGE). In this experiment, SGSI was trained with mini-batching, computing gating parameters only on sampled local neighborhoods. Our preliminary results, summarized in the table below, show that SGSI remains effective at scale, converging in under 60 hours while achieving meaningful AUROC improvements:

These findings suggest that the core mechanisms of SGSI (soft gating, cloning, and KL skipping) remain robust even for large graphs.

3. Computational Efficiency and Training Time

In response to your request for explicit runtime data, we provide the following training-time comparison across our main datasets (Springs, NetSims, LI, LL, and the 100-node VN datasets):

SGSI consistently matches NRI’s training time (e.g., 20.4h vs. 20.1h on Spring Simulations and 47.1h vs. 47.2h for VN_NS_100), while methods such as iSIDG require ~1.5–2× longer. Yet recall that SGSI achieves 79.7% AUROC on Springs, which is higher than NRI. Overall, these results confirm that SGSI’s partial-knowledge gating imposes minimal additional cost while providing meaningful accuracy gains in structural inference.

Thank you again for your helpful comments, and we hope these newly added experiments bolster confidence in SGSI’s robustness, scalability, and efficiency.

We thank you for your positive assessment and valuable comments. In response, we clarify our approach as follows (tables can be found at https://anonymous.4open.science/r/SGSI-Rebuttal-1614/Tables.pdf .)

1. Hyperparameters

SGSI introduces hyperparameters, most notably, the KL weight $\beta$, the sparsity penalty$\lambda_{\mathrm{sparsity}}$, and the node-degree penalty $\lambda_{\mathrm{deg}}$. These parameters are essential for balancing the trade-off between inference and the incorporation of prior knowledge. We conducted Bayesian optimization with systematic grid searches on a validation subset to identify a “reasonable zone” that yields both high AUROC and stable convergence. Our initial search considered:

• $\beta \in \\{0.1, 0.5, 1.0, 1.3, 1.4, 2.0, \dots\\}$,
• $\lambda_{\mathrm{sparsity}} \in \\{10^{-4}, 10^{-3}, 10^{-2}\\}$,
• $\lambda_{\mathrm{deg}} \in \\{10^{-4}, 10^{-3}, 10^{-2}\\}$.

Our domain heuristics further guide these choices:

• When the underlying graph is known to be very sparse (e.g., physical systems with local interactions), a larger $\lambda_{\mathrm{sparsity}}$ is preferable.
• When node-degree constraints are well-established (e.g., each node has exactly 2 neighbors), a higher $\lambda_{\mathrm{deg}}$ ensures the model respects these constraints.
• Conversely, if the prior knowledge is only approximate, lower penalty weights prevent the model from being over-constrained.

We showcase results of sensitivity study on the VN_SP_30 dataset, and the results can be found in the Table 1 in the attached link. The results demonstrate that the optimal performance is achieved with $\beta=1.3, \lambda_{\mathrm{sparsity}}=0.004$, and $\lambda_{\mathrm{deg}}=0.01$, yielding an AUROC of 92.76%. Variations from these values result in a modest decrease in performance, which validates that while SGSI is indeed sensitive to these hyperparameters, it operates robustly within a reasonable range.

2. Robustness to Inaccurate Prior Knowledge

SGSI leverages soft penalties and cloned gating to integrate partial knowledge while preserving flexibility. Because these constraints are applied softly, via mild penalties and by clamping only a cloned copy of the gating vector, the model can deviate from incorrect priors when the data strongly contradicts them. In SGSI, the soft gating mask is applied after the Node-to-Edge operation (but not after Edge-to-Node), which helps preserve residual connectivity and prevents over-reliance on potentially flawed edges.

To further validate this, we conducted “noisy prior” experiments on VN_SP_100 and VN_NS_100, where we randomly flipped 20% or 50% of the known-present/absent edges (with the overall prior knowledge set to 30%) and introduced 10% or 20% errors in global sparsity and degree constraints. Table 2 in the attached link summarizes our preliminary results.

These results indicate that even with moderate noise, SGSI remains significantly more accurate than a no-knowledge baseline with at least 1~2% margin. Although performance gains naturally diminish as noise increases, the model robustly leverages available prior knowledge without catastrophic failure.

3. Scalability to Larger, Heterogeneous Graphs

We appreciate your interest in extending SGSI to larger or heterogeneous graphs. SGSI is designed to scale via mini-batching and subgraph sampling, approaches similar to GraphSAGE, which allow SGSI’s gating parameters to be computed or stored per subgraph rather than across the full $N \times N$ edge space. For extremely large graphs, we can either (i) restrict gating to local neighborhoods, or (ii) compute gating logits on the fly from node embeddings, thereby avoiding an $\mathcal{O}(N^2)$ parameter explosion.

In this paper, we demonstrate scalability on the PEMS datasets (approximately 300 nodes) by leveraging multi-GPU mini-batching (in Appendix E.2). To further validate SGSI on larger graphs, we generated a toy dataset with 5,000 nodes using Springs Simulations. Table 3 in the attached link shows the $\Delta$AUROC values under various levels of prior knowledge. These results indicate that SGSI, when using mini-batching, remains effective even at the 5k-node scale with prior knowledge, and with meaningful improvements in $\Delta$AUROC from 0.56 to 7.68.

SGSI’s flexible architecture inherently supports heterogeneous graphs. By adopting per-relation gating parameters $\theta_{e,r}$ and applying the clone-and-clamp strategy to each edge type, SGSI can differentiate among various relations and node types while maintaining stable gradient flow and effective KL skipping. Although our current domain primarily involves homogeneous datasets, we recognize SGSI’s potential in applications such as multi-modal social networks, biomedical systems, and multi-layer transportation networks, and plan to explore these in future work.

Thank you again for your positive feedback. We look forward to refining our manuscript with these additional experiments.