UnCLe: Towards Scalable Dynamic Causal Discovery in Non-linear Temporal Systems

Response to Reviewer 9Mey

We thank you for your time and for the valuable questions, which help us clarify important aspects of our method.

Weaknesses:

• W1: On not exploring higher-order lags. This is an excellent point. We conducted an ablation study on the Lorenz#1 dataset to investigate the effect of increasing the autoregressive lag in the Dependency Matrices. The results are as follows:

Increasing the lag decreased performance. We hypothesize this is because the TCN's large receptive field already encodes long-range historical information into the latent representation $z_t$. Adding explicit higher-order lags in the linear prediction step introduces parameter redundancy, making the model more prone to overfitting on spurious, non-primary relationships, while also increasing computational cost. Thank you for raising this; we will add this experiment to the final version.

Questions:

• Q1: On the identifiability of the causal structure in latent space. (This is a core question also raised by Reviewers SUxp and T7tD). We agree that this is a critical theoretical point. Please see our detailed response to Reviewer SUxp, Weakness W1. In summary, while our work does not provide a formal proof of identifiability, which remains an open challenge for many deep learning methods, we provide extensive empirical evidence for UnCLe's effectiveness and consider this a key direction for future theoretical work.

• Q2: On the parameter size of the Dependency Matrices in high-dimensional settings. The parameter count of the Dependency Matrices does grow quadratically with the number of variables $N$ (as $C \times N \times N$). However, the number of channels $C$ is a small, constant hyperparameter. More importantly, UnCLe's architecture is highly parallelizable. In cases where memory is a constraint on very large-scale data, the batch size $B$ can be reduced to allow the model to run, trading a small amount of speed for feasibility. This is detailed further in Appendix L ("Implementation of Temporal Perturbation").

• Q3: On the learning strategy for the Dependency Matrices ($\Psi$). Our apologies for not detailing this sufficiently. The Dependency Matrices $\Psi$ are learnable parameters that are optimized jointly with the rest of the network. They are initialized from a normal distribution $N(0, \sqrt{3/N})$, where $N$ is the number of time series. Their values are updated based on their role in the prediction task (via the $L_{Pred}$ loss) and are encouraged towards sparsity by the $L_{L1}$ regularization term. We will add these details to the methodology section in the final version.

Thank you again for your insightful questions and feedback.

Response to Reviewer T7tD

We thank you for your positive assessment and for providing valuable feedback to strengthen our paper.

Weaknesses:

• W1: On the lack of fundamentally new neural paradigms. We agree that UnCLe does not introduce a fundamentally new neural paradigm. Its novelty lies in the specific architectural integration of established components (like TCNs) and a perturbation-based analysis, tailored specifically to tackle the challenging and under-explored problem of scalable dynamic causal discovery.

• W2: On the need for more theoretical guarantees and interpretability. (This point is also raised by Reviewers SUxp and 9Mey). We agree that more theoretical guarantees, such as for identifiability, are an important next step. Please see our detailed response to Reviewer SUxp, Weakness W1. Regarding channel interpretability, we provide a detailed analysis in Appendix I ("Interpreting Channel-wise Causal Contributions"), which shows how different channels capture different facets of the underlying causal structure.

• W3: On the need for more rigorous quantitative evaluation on dynamic benchmarks. We appreciate this suggestion. In the paper, we provide quantitative evaluation on the dynamic SEM (dSEM) dataset via AUROC/AUPRC and on the MoCap dataset via a missing rate metric for skeletal connections. We also complement this with qualitative analysis of the MoCap phases. We will enhance our qualitative analysis by incorporating more specific biomechanics literature, as suggested by Reviewer K7ML, and will continue to explore additional quantitative evaluation methods for dynamic graphs.

Questions:

• Q1: On clarifying the link between Dependency Matrices and true causal effects. We provide a detailed analysis of this in Appendix I ("Interpreting Channel-wise Causal Contributions"), using the Lorenz#1 dataset as an example. In brief, each channel's Dependency Matrix learns a specific "semantic" mode of interaction between variables. The true causal mechanism is modeled by the collective action of all these interaction modes. Aggregating them allows us to recover a comprehensive view of the static causal graph.

• Q2: On the sensitivity of UnCLe to the choice of perturbation strategy. This is a great question. We experimented with three strategies: permutation (our default), zero-masking, and random noise injection. Permutation performed best. We reason that zero-masking disrupts the data distribution, affecting model stability, while noise injection does not fully remove the original signal. Permutation uniquely preserves the marginal distribution while effectively nullifying predictive temporal information. We believe this is a valuable point and will add a summary of this sensitivity analysis to the final version.

• Q3: On avoiding spurious correlations in high-dimensional, limited-data settings. This is a key challenge in causal discovery. UnCLe mitigates this risk through its parameter-sharing TCN autoencoder, which allows the model to learn robust, common representations from the entire dataset, effectively increasing the data available for learning each component. This enhances model stability. We tested this scenario with the Lorenz#3 dataset (N=100, T=500), where UnCLe outperformed all baselines, suggesting our approach is relatively robust in such settings.

• Q4: On providing more detailed runtime and memory analysis for scalability. We agree this is important for assessing practical applicability. UnCLe's scalability stems from its parameter-sharing design and parallelizable computations. In the final version, we will expand the "Time Efficiency" subsection to include a table or plot detailing UnCLe's runtime and memory usage across different data scales (N and T), and supplement the existing Figure 4 with efficiency comparisons on other datasets.

Thank you for your constructive and detailed feedback.

Response to Reviewer SUxp

We sincerely thank you for your detailed review and for highlighting the strengths of our work. We appreciate the opportunity to clarify the points you raised.

Weaknesses:

• W1: On the lack of identifiability guarantees. We concur that formal identifiability guarantees are crucial for causal discovery methods. This is a well-known and challenging open problem for many deep learning-based approaches, whose theoretical properties often lag behind their empirical success. While designing UnCLe, we explored connections to causal disentanglement literature (e.g., [1][2]), but found that proving formal properties for our specific architecture was non-trivial. Therefore, this work focuses on providing strong and extensive empirical validation of UnCLe's effectiveness. We have added providing a formal identifiability analysis as a primary direction for future work.

• W2: On not studying cases where lag > 1. (This point is also raised by Reviewer 9Mey). Please see our detailed response to Reviewer 9Mey, Weakness W1, which includes experimental results for different lags. In summary, the TCN architecture's receptive field allows the model to handle long-range dependencies, and we found that explicitly increasing the autoregressive lag in the Dependency Matrices did not improve, and sometimes hindered, performance, likely due to parameter redundancy.

• W3: On the robustness of permutation compared to other perturbation methods. (This point is also raised by Reviewer T7tD). Please see our detailed response to Reviewer T7tD, Question Q2. In summary, we experimented with permutation, zero-masking, and random noise injection. Permutation yielded the best results because it effectively removes temporal information while preserving the variable's marginal distribution, ensuring model stability. We will add a summary of this sensitivity analysis to the final paper.

• W4: On the performance drop on sparse synthetic datasets with >50% missing edges. We appreciate you pointing this out for clarification. However, we were unable to locate the specific statement about ">50% missing edges" in our manuscript. Could you please help clarify which part of the paper this refers to? We would be very grateful for the opportunity to address this point more directly.

Questions:

• Q1: On handling missing or irregular data. This is an important practical consideration. Our current work assumes complete and regularly-sampled data. For datasets with missingness or irregularities, UnCLe would require a standard preprocessing step, such as imputation, before application. Explicitly handling such data within the model architecture is a valuable direction for future research.

• Q2: On extending the post-hoc perturbation method to other frameworks. This is a very keen observation. Yes, the post-hoc perturbation framework is in principle model-agnostic and could be applied to other multivariate time-series forecasting models. Exploring its effectiveness across different architectures is a promising research avenue that we hope our work will inspire.

• Q3: On whether L1 regularization hampers the detection of temporarily dense graphs. This addresses a fundamental trade-off between interpretability and complexity. Our goal with L1 regularization is to encourage the model to capture the most stable and dominant causal relationships. While a very strong regularizer could potentially suppress transient dense connections, we found that a moderate L1, combined with the aggregation of diverse patterns across multiple channels, allows the model to represent complex structures without being overwhelmed by noise or spurious connections. The key is to find a balance that yields an interpretable yet sufficiently descriptive graph.

Limitations:

• On adding a discussion of broader impacts and safeguards. Thank you for this crucial feedback. We agree that a discussion on broader impacts is essential. In the final version of this paper, we will add a dedicated section to address the potential misuse of causal claims in sensitive domains and outline best practices for responsible application, such as combining UnCLe with domain expert knowledge and performing prospective validation.

Thank you again for your thoughtful review.

References

[1] Song, Xiangchen, et al. "Temporally disentangled representation learning under unknown nonstationarity.", NeurIPS 2023

[2] Song, Xiangchen, et al. "Causal temporal representation learning with nonstationary sparse transition.", NeurIPS 2024

Response to Reviewer K7ML

We sincerely thank you for your insightful review and positive evaluation of our work. We appreciate the constructive feedback and will address the identified points to improve the paper.

Weaknesses:

• W1: On the time sensitivity of causal discovery and the speed-up from weight aggregation (UnCLe(A)). That is an excellent point. While many scientific discovery tasks are performed offline, latency can be critical in applications like real-time monitoring in industrial systems. The speed-up from UnCLe(A) is most significant on large-scale datasets where the perturbation process (UnCLe(P)) becomes more time-consuming. For smaller datasets, the overhead of perturbation is negligible. In the final version, we will add an analysis comparing the post-hoc inference times for UnCLe(P) and UnCLe(A) across different data scales to better quantify this trade-off.

• W2: On visualizing the ground truth (GT) causal strength in the DSEM experiment. We agree that a direct comparison would be ideal for interpretability. The causal direction is explicitly defined in the SEM equations, and we visualized this with color-coded segments and vertical lines. However, we did not plot a GT "causal strength" line because the SEM coefficients, while strongly correlated with strength, are not directly equivalent to a singular strength value, making a direct GT plot potentially ambiguous. Our visualization approach, which includes smoothing the discovered strengths, aims to clearly show how UnCLe correctly identifies the switching dominance between variables in alignment with the ground truth mechanism.

• W3: On clarifying which version of UnCLe was used in Figure 4. Our apologies for the omission. Figure 4 shows the results for the perturbation-based method, UnCLe(P). We will clarify this in the caption in the final version.

• W4: On substantiating the MoCap analysis with domain literature. We thank the reviewer for this insightful and critical question. The reviewer is entirely correct that without a ground truth, asserting a causal graph is "better" is a non-trivial claim, and we agree our argument must be substantiated with external domain knowledge. We will revise the manuscript to incorporate this justification. Our core argument, supported by biomechanics literature (e.g., Hara et al., 2008; DeVita & Skelly, 1992), will be that UnCLe's graphs are "better" because their "structural evolution" aligns with the known, "phase-specific coordination strategies" of the jump. Specifically, UnCLe captures the shift from full-body coordination (in crouch/landing) to localized adjustments (in flight), a dynamic nuance missed by the baselines. We will add the necessary analysis and citations in the final version to fully support this interpretation.

Questions:

• Q1: On the motivation for proposing two distinct causal discovery mechanisms (static/dynamic). This is a great question about our design process. Our work began with developing a robust representation and prediction model for time series. We first explored perturbation for static discovery. In seeking to explain its success, we found that the learned Dependency Matrices themselves contained a static causal blueprint (UnCLe(A)). We then realized the perturbation approach naturally extended to dynamic discovery (UnCLe(P)). We chose to present both because they are complementary. UnCLe(P) is our primary contribution for accurate dynamic discovery. UnCLe(A) serves as a zero-cost, interpretable diagnostic tool derived directly from the model's parameters, providing a valuable static summary and a basis for understanding the model's learned structure. We believe both approaches offer valuable insights to the community.

• Q2: On validating the assumption of linear latent dynamics. Thank you for pointing us to this highly relevant literature. We agree that framing our work within the context of dynamical systems theory, such as linearization of nonlinear systems, would greatly strengthen the paper's theoretical underpinnings. This is a crucial point for understanding the limitations of our method and for guiding future work. We will incorporate this perspective and the suggested references into our discussion.

• Q3: On emphasizing the mechanism behind perturbation-based discovery. This is a key insight. We will revise the methodology section to more explicitly state that UnCLe(P)'s effectiveness stems from its ability to model the data's causal generative mechanism. An inaccurate prediction under perturbation indicates that the model's learned causal dynamics did not anticipate the next state, which is a direct consequence of disrupting a true causal link.

Thank you again for your valuable time and constructive suggestions.