LLM Interpretability with Identifiable Temporal-Instantaneous Representation

Thank you for acknowledging our contribution between empirical interpretability and formal causal representation learning and, more importantly, the scalability issue we address. We now address your concerns one by one.

W1. The paper's objective lacks clarity—while it aims to enhance LLM interpretability, the connection between the linear representation hypothesis and LLMs is not well established. In particular, it remains unclear how variables like $x_t$ and $z_t$ are concretely defined within the LLM context.

AW1: The Linear Representation Hypothesis is well-established for LLMs through prior work [A], which demonstrated that LLM activations can be effectively modeled using linear methods.

To clarify our notation: $\mathbf{x}_t \in \mathbb{R}^m$ represents the activation vector at layer $L$ for token position $t$, where $m$ is the model's hidden dimension. The latent variables $\mathbf{z}_t \in \mathbb{R}^n$ capture $n$ interpretable concepts that generate these activations through the linear decoder $A$.

Specifically within the LLM context (line 73-90): Given input tokens $(v_1, v_2, \ldots, v_T)$, we extract activations $\mathbf{x}_t$ from a target transformer layer (e.g., layer 12 of 32) at each position $t$. Our model decomposes these activations into interpretable concepts $\mathbf{z}_t$ (Eq. 1, 2) that exhibit temporal dependencies $B$ (how concepts evolve across tokens) and instantaneous relationships $M$ (how concepts interact within each token). We will add this clarification to Sec. 2 with specific examples to demonstrate this mapping concretely.

[A] Park, Kiho, Yo Joong Choe, and Victor Veitch. "The linear representation hypothesis and the geometry of large language models." ICML (2024).

W2. The use of synthetic time series data undermines the relevance of the evaluation, as it does not accurately reflect the structural or statistical properties of LLM activations, which the method ultimately targets.

AW2: This synthetic-to-real validation paradigm is a widely accepted methodology in causal representation learning and related fields. We establish theoretical identifiability guarantees for learning concepts and relations. Since ground truth is unavailable in real-world scenarios, this identifiability cannot be directly testable on real LLM data. Therefore, controlled synthetic experiments with known ground truth are essential for verifying both our theoretical analysis and algorithm implementation.

Once we establish confidence that our theoretical guarantees translate into practical performance, subsequent experiments on real LLM activations (Sections 5.2 and 5.3) demonstrate the method's relevance to authentic data. We have revised the manuscript to clarify this rationale and emphasize that synthetic studies complement and validate, rather than replace, our evaluations on real LLM data.

W3. The paper lacks comparison with strong baseline methods and omits quantitative evaluation metrics, making it difficult to assess the practical effectiveness or improvements of the proposed approach.

AW3: As acknowledged by other reviewers and mentioned on pages 6–7, our work is the first to address temporal and instantaneous relations at the LLM activation scale. We extensively searched for baselines from the causal representation learning (CRL) literature, but to our knowledge, no existing method can efficiently handle the high-dimensional activation data produced by LLMs (Figure 5). We would appreciate clarification on any specific missing baselines you have in mind.

We agree that quantitative analysis can strengthen our approach. While no existing benchmark captures causal representation learning for concept relations in LLM activations, we demonstrate our method's effectiveness through two evaluations: (1) testing on SAEBench [A] to show our model matches mainstream SAEs' performance on standard SAE tasks, and (2) presenting quantitative results on a new task tailored to our unique contribution—recovering concept-to-concept relations.

(1) SAEBench: For fair comparison, we trained TopK and ReLU SAEs with hyperparameters (feature dimension, training cost, etc.) similar to our model:

These results demonstrate that while not specifically designed for SAE tasks, our model achieves comparable performance on these tasks.

(2) Concept Relation Recovery: Furthermore, building on the experiments in [B], we quantitatively evaluate our model on concept-to-concept relation extraction.

Data preparation: We constructed two contrastive collections of texts drawn from the Pile dataset. One collection consists of legal documents, which often follow structured syntactic formats from start to end, leading to stable temporal concept patterns. The other contains non-legal texts without such structure. We hypothesized that only legal texts would yield meaningful relational patterns, and we directly tested whether the model can successfully recover such patterns.

Baseline: Although there is no directly applicable baseline, we leveraged standard SAEs we had trained above to serve as our baseline method. Since SAEs themselves cannot capture the concept-to-concept relations, we train a regression model to find temporal relation coefficient matrices $\tilde{B}$s by solving the following regression task:

$\text{min}\_{\tilde{B}\_{\tau}} ||\mathbf{z}\_t - \sum_{\tau} \tilde{B}\_{\tau} \mathbf{z}_{t-\tau}||\_{2}.$

Evaluation Pipeline: We calculate the concept recovery score by first getting the top fired feature index pair $(i,j)$ related to legal context and the corresponding entry $B_{i,j}$ in the aggregated temporal relation coefficient matrices. We then calculate the relation recovery score, similar to a signal-to-noise ratio, by: $`relation recovery score` = \frac{B_{i,j}}{\sigma(B)}$, where $\sigma(B)$ denotes the standard deviation of matrix $B$. Such ratio indicates the extent that the target concept relation entry in the matrix is more significant than a random noise; the larger the score is the more significant the relation recovery.

From the table below, we notice a significant difference, showing that our method can successfully recover the relation.

These two experiments together outline our improvement over existing SAE methods by first showing our matching performance on SAE tasks and then demonstrating that our method can successfully extract meaningful concept-to-concept relations, which cannot be handled by SAEs.

[A] Karvonen, Adam, et al. "SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability." Forty-second International Conference on Machine Learning, 2025

[B] Joshi, Shruti, Andrea Dittadi, Sébastien Lachapelle, and Dhanya Sridhar. "Identifiable steering via sparse autoencoding of multi-concept shifts." arXiv preprint arXiv:2502.12179 (2025).

Q1 Will the performance drop after the method is applied?

AQ1: We assume the performance you mentioned here refers to the language model's original metrics, such as downstream task accuracy and perplexity. Our interpretability method only reads transformer layer activations; it does not modify any weights, architecture, or forward computations. The model’s task performance therefore remains unchanged. While some interpretability approaches do fine‑tune model weights and can reduce performance, our analysis is not in this category. If the reviewer is considering a different notion of performance, we would appreciate further clarification.

L1. This paper does not mention the limitation. As far as I know, the Interpretability will harm the performance.

AL1: We kindly refer the reviewer to Appendix Section A.5 for the limitations discussion. The performance issue is discussed in the previous question.

Dear Reviewer KYay,

Thank you again for reviewing the paper and for taking the time to consider our updates. We sincerely appreciate your observations and hope for the opportunity of further engagement to avoid misunderstandings. If there are still aspects that remain unclear, we would be very grateful for the opportunity to provide further clarification to illustrate our approach.

Sincerely,

Authors of Paper 7618

We sincerely thanks for your insightful feedback. We're glad you appreciate the strengths in our novelty for being the first to scale causal representation learning to LLM activations, and we appreciate your constructive suggestions for strengthening our contribution. We address each point below.

W1,Q6 While more scalable than existing causality methods, the method seems to still have poor scalability compared to standard SAEs. The paper only evaluates dictionaries up to size 6144, which is miniscule by LLM SAE standards. In the paper the trained LLM models are still very small - is it feasible to train a more standard size version of this method to compare with SAEs, e.g. 32k+ latents?

AW1,Q6: Thank you for this important question about scalability. While the reported dictionary size of 6,144 was indeed limited by GPU memory constraints, as you also mentioned in your question, this dimension already represents a significant advancement in terms of dimensionality for identifiable representation learning methods. Our experimental results demonstrate that even with this dictionary size, we can extract meaningful interpretations of concept-to-concept relations.

For larger dictionary sizes, our approach can be parallelized using tensor parallelism or data parallelism to accommodate greater scale. However, this would require substantial refactoring of existing open-source infrastructure tools like TransformerLens and dictionary_learning to support multi-GPU training, which falls well outside the scope of our current work. We acknowledge this limitation and consider scaling to 32k+ latents an important direction for future research.

W2,Q7 The paper also does not compare this technique directly with LLM SAEs (e.g. on interpretability benchmarks like SAEBench), so it's hard to gauge if / how much this improves on standard SAEs in practice. Can this method be directly compared to LLM SAEs using standard SAE benchmarks like sparse probing, or other metrics from SAEBench?

AW2,Q7: Thanks for the suggestion. Our main contribution is the recovery of temporal and instantaneous concept‑to‑concept relations, which is not reflected in SAE benchmarks. But we do agree with the reviewer that making a comparison on established benchmarks for SAEs will strengthen our position. For a fair apples-to-apples comparison, we trained two baseline SAEs (ReLU and TopK) under similar hyperparameters and evaluated all three methods on a range of SAEBench metrics, including sparse probing. Since we did not modify the SAE objective, we expect our model to perform on par with existing SAEs on SAEBench tasks, which is confirmed by the table below.

W3. The temporal relations are modeled by fixed token distances, which seems inflexible and likely cannot capture real syntac relationships between tokens well.

AW3: We totally agree that fixed limited token distances might not be the most ideal way to capture the rich semantic and syntactic relationships between tokens. However, it is worth pointing out that even though the distance is limited, it still successfullly reveals some meaningful temporal relations (together with instantaneous relations) among the concepts as we have shown in the real-world experiments. We further conducted the analysis for longer temporal relations ($\tau=100$). Due to the rebuttal policy, we may not show the visualization results. We will add those results to our appendix to more comprehensively showcase the interpretability our proposal may achieve.

Q1. Can this method model hierarchical relationships between latents? E.g. if z1 can only fire if z2 is firing, but z2 can on its own, can this be captured by this model? This would correspond to something like "z1 = noun" and "z2 = dog", since "dog" is a "noun". Hierarchical relationships seem to be break standard SAEs (causing feature absorption), so if this method can handle that it would be huge.

AQ1: Yes, it can! The graph for instantaneous relations can naturally model the hierarchical relationships among concepts. Our case study for instantaneous relations Month (e.g. March) to Full date (e.g. March 23, 2000) is exactly such a case, Month can be fired on its own, but if Full Date is fired, then Month will also be fired. Such an advantage is also captured quantitatively by the better feature absorption score in the table above.

Q2. Can this method only handle relations between 2 latents at a time? e.g. if a latent is and AND of 3 latents, can this method capture that?

AQ2: Thank you for this great question. Indeed, the relations recovered by our method are represented in matrices where each entry reflects pairwise relations. But they can still support the inference of higher-order logical relations, even though it's not immediately apparent. For example, if a concept activates only when multiple other concepts are active (e.g., an AND of three latents), this can be inferred by analyzing joint activation patterns over many observations. By thresholding and binarizing concept activations, one can construct truth tables and recover logical relations such as AND, OR, or XOR. While not explicitly modeled in the relation matrices, such higher-order dependencies are reflected in the data and can be recovered through post-hoc analysis.

Q3. In Figure 3, is (e) the correlation between A and $\hat{A}$ not shown? Also, why is estimated $\hat{B}$ not identical to B? It seems like the method should be able to perfectly recover B in this synthetic setting?

AQ3: Thanks for pointing this out, our identifiability was established for latent variables $\mathbf{z}_t$ (up to permutation and scaling) and their relations $B$ and $M$, hence the exact value matching for $A$ and $\hat{A}$ is not required and we allow some indeterminacy for $A$. Below is an example of the ground truth $A$ and our estimated $\hat{A}$, and $\mathbf{z}_t$ for this setting is already identifiable (MCC > 0.99).

$A$ = [[ 0.38, 0.92, -0.02],[-0.67, 0.29, 0.68], [ 0.64, -0.25, 0.73]]

$\hat{A}$ = [[-3.38, -7.56, 0.22],[ 5.34, -2.63, -5.63],[-5.22, 1.85, -6.01]]

Despite the theoretical guarantee for $B$, in practice, the perfect recovery requires optimization using an L0 sparsity penalty, which is known to be non-differentiable. In the literature, the L1 alternative is widely adopted. That explains the small difference in the estimated $B$ while it can still effectively preserve most of the structure information.

Q4. In Equation 7, why is there no nonlinearity (presumably ReLU) in the encoder? Should there also be encoder and decoder bias terms?

AQ4: Nonlinearity: Thanks for pointing this out! Yes, in real-world LLM experiments, we use top-k activation in the encoder, following mainstream SAE architectures, which is non-linear. We use simple affine transformations in synthetic experiments for theoretical tractability.

Bias terms: Yes, there should be bias terms for more modeling flexibility. We modified the equation to include them. We also compared these two settings in the real LLM activations (feature dimension=3072, $\alpha=0.1$, $\beta=0.01$, $\tau=5$). The results shown in the table below indicate that the flexibility gain of the bias terms is not significant in our model.

Q5. It seems like the $\hat{A}$ matrix is equivalent to an SAE decoder dictionary - can this matrix be directly used as an SAE? Would you expect this method to learn a more correct dictionary than standard SAE training methods?

AQ5: Yes, this can be used as an SAE. As shown in AW2,Q7, the performance on SAE tasks for our method is at least on-par with existing SAEs. We acknowledge that there may not be a universally more correct dictionary; we could say our learned dictionary is relation-aware, which can be more reasonable and capable than standard SAEs.

Thank you for your timely and valuable feedback and suggestions. Following your recommendations, we trained our simplified instantaneous-relation-only model with 16k latents on Gemma-2-2B using the same dataset. The results are presented below:

The 16k model exhibits a slightly higher absorption score (0.0167) compared to our reported 6k model, but this value still seems negligible. Performance on other SAEBench metrics remains at the same level.

We are wondering whether these experimental results would help answer your questions properly. If you have any further feedback, please kindly let us know, and we hope for the opportunity to respond to it.

Dear Reviewer WPzT,

Your words are so encouraging and exciting to us. After your previous comments, we extended the training scale to 500M tokens for the Pythia-160m model, and to 200M tokens for the gemma-2-2b model. In both cases, we still observed very small absorptions, as seen from the table below:

Once again, we sincerely appreciate your valuable comments and suggestions for helping to improve our work. We hope this effort can contribute to refining the architecture of LLM interpretability methods. By the way, the reviewer-author discussion will end soon but Reviewer Uu1b still hasn't yet responded to our response--if it is appropriate, a gentle nudge from you at some point would be much appreciated.

Sincerely yours,

Authors of Paper 7618

Dear Reviewer WPzT,

We are pleased to inform you that the training process for the setting you recommended (gemma-2-2b, 16k dimension, 300M tokens) has just completed. The results align well with the previous observations. Please find the summary below:

Once again, all of the authors would like to sincerely thank you for your encouraging and insightful feedback on our manuscript and proposed method. Your perceptive observations on the absorption score have been especially valuable. We truly hope our work will inspire further reflection and discussion within both the mechanistic interpretability and causal representation learning communities.

Sincerely,

Authors of Submission 7618

We thank the reviewer for the thoughtful review and the opportunity to clarify key aspects of our work. We address each concern below.

W1. It does not seem to be shown that the SEM can model the distribution of activations; case studies are performed (selecting <=6 examples of specific types of successes in total), but I did not see an objective or large-scale analysis supporting this claim. In other words, evaluation appears to be limited.

AW1: Thank you for this thoughtful comment. Our primary contribution is to identify latent concepts and their temporal or instantaneous relations under sparse structural assumptions with theoretical guarantees. While modeling the full distribution of LLM activations is not our focus, our method captures the structure of the activation space well—evidenced by near-zero reconstruction loss (the first columns of Figures 8 and 9).

Though only a few case studies appear in the main text due to space limits, we include over 20 additional examples in Tables 2 and 3 (appendix), along with longer-range temporal analyses demonstrating richer semantic dynamics. Additional visualizations and discussions will be included in the updated manuscript.

To address evaluation rigor, we added SAEBench experiments using standard metrics (e.g., reconstruction loss, sparse probing), showing our model performs comparably to strong SAE baselines (see our responses AQ3 to Reviewer WPzT). We also introduce a semi-synthetic legal-vs-non-legal task to evaluate our model’s ability to extract interpretable relations (see our responses AW2,W4 to Reviewer gsa4).

These tasks are aligned with our central goal: interpretable representation and relational discovery, rather than generative modeling. Finally, we emphasize that meaningful, human-aligned examples are critical for LLM interpretability. We will clarify this distinction and further strengthen both empirical and illustrative support in the revision.

Q1a. Usually, LLM activations are viewed as attention computations, and there is no sequential time dependence. Why was the modeling choice of sequential time dependence made in this work? Is there evidence support the claim that this is superior to an attention style of activation modeling?

AQ1a: We first clarify that there is indeed sequential time dependence in attention computation, as the current position needs to attend to previous positions. Additionally, there is natural sequential dependence in the semantic concept space during text generation, i.e., what you think or write now definitely depends on what you thought or wrote in the past. Therefore, when using learned activations to interpret LLMs' internal processes, sequential modeling is a very natural choice. Regarding attention-style modeling, it is known to be highly inexplicable, which is precisely the motivation behind our proposal.

Q1b. Is there a benefit of having an SEM, which induces interventional distributions, rather than a probabilistic model?

AQ1b: Actually SEM is a probabilistic model; the distinction is that an SEM specifies how variables respond to explicit interventions, not just how they co‑vary observationally. Our objective is to explain how one internal concept can influence another within the LLM. This question is inherently interventional. The benefit of an SEM over any associational probabilistic model is that it explicitly models causal mechanisms, allowing us to understand not just correlations but directional influences between concepts.

Q2a. Are the latent variables $\mathbf{z}_t$ usually confirmed to be identifiable, given that there are many more latent than observed variables?

AQ2a: Yes, this is a standard setting in CRL and sparse dictionary learning literature which usually referred with term "over-complete" setting. A number of approaches [3,A,B] including our work leverage the structural sparsity to establish the identifiability.

Q2b. What is the value of showing that identifiability holds in the synthetic setting in Experiment 5.1?

AQ2b: For sanity check! Theoretical identifiability proofs do not automatically guarantee that practical algorithms can recover latent variables under realistic constraints (finite data, noise, optimization challenges). And experiment 5.1 bridges this theory-practice gap by using synthetic data with known ground truth to confirm our implementation converges to the theoretically identified solution, and establish performance baselines before applying to real data where ground truth is unavailable. Such synthetic-to-real validation is a standard methodology in CRL.

Q3. Is there evidence to support the claim that the proposed SEM in equations (1), (2), and (3) is consistent with observational activations generated by real LLMs? how well does the proposed model fit the actual distribution of activations? Is there any baseline for how well a model can fit this distribution, and does the proposed model improve upon it?

AQ3: Yes! We first demonstrate our modeling consistency with observational activations via the convergence of reconstruction loss (Figures 8-9 in Appendix) to sufficiently small values. We also compare it with estabilished SAE baselines:

Our method also achieves comparable reconstruction loss (~0.01), which further confirms the consistency is at least on-par with those SAEs.

We emphasize that distribution matching, while necessary, is not our primary contribution. Any universal approximator can achieve near-perfect reconstruction. The critical challenge is uniquely identifying the latent causal structure underlying these activations. Our method not only fits the data distribution but also provides theoretically grounded guarantees for recovering the true causal relationships--Section 5.2 demonstrates that recovered causal dependencies align with known semantic patterns in language--which existing methods like SAEs cannot provide.

Q4. Given that the model proposed is an SEM rather than a purely observational model - is there evidence to support the claim that there are the instantaneous and temporal mechanisms can be modeled linearly in the specific way this work models them?

A4: Yes! And we address this question through three key points:

SEM v.s. purely observational model Please allow us to clarify that SEM could model both observational and latent variables; so they are not mutually exclusive.

We employ latent variables in SEM to model LLM activation dynamics (rather than observationally) because LLM activations are inherently polysemantic [D]: each dimension encodes multiple entangled concepts, making direct concept-to-concept analysis infeasible in the raw activation space. Such latent variable approach is well-established in literatures.

Evidence for instantaneous and temporal mechanisms The existence of such causal relationships has been extensively studied in the mechanistic interpretability community, with recent empirical studies [1,28] confirming these relations (line 32 and footnote 1).

Justification for our linear modeling Existing work [41] showed semantic features occupy linear subspaces, and SAEs [28,50,D] successfully use sparse linear model for finding concepts. Those established the fundation for linear modeling. More importantly the evidence comes from our own successful recovery of semantically meaningful causal structures (Section 5.2). This establishes linearity as an empirical reality at the relevant abstraction level rather than a simplifying assumption.

Q5. Is there evidence to indicate whether or not the preconditions of Thm 1-3 are met, sparsity in particular?

A5: This is a very good question! Verifying the latent assumptions about hidden variables is a fundamental challenge to all research on latent variable identification [E] since latent variables are not directly measurable. We approach to their reasonableness through indirect validation: if the model successfully fits the observed data distribution and consistently yields to meaningful solutions, this provides evidence that the underlying assumptions are moderate and feasible. This approach provides a practical solution for this challenge and are widely acknowledged in the community [1,54].

Regarding sparsity in particular, evidence partially comes from empirical success from the SAE and CRL literatures [25,50,A,B,C,D] to use sparsity constraints to extract concepts recover non-trival causal relations. More importantly, evidence also comes from our own successful recovery of semantically meaningful causal structures (Section 5.2) together with the convergence behavior of our loss curve during training.

We acknowledge that these assumptions, like any in this field, may not hold universally, our empirical results suggest they seem effective and feasible for the complex and real data we study.

[A] Sun, Yuchen, and Kejun Huang. "Global identifiability of overcomplete dictionary learning via l1 and volume minimization." The Thirteenth International Conference on Learning Representations. 2024.

[B] Chen, Siyu, et al. "Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders." arXiv preprint arXiv:2506.14002 (2025).

[C] Wang, Kexin, and Anna Seigal. "Identifiability of overcomplete independent component analysis." arXiv preprint arXiv:2401.14709 (2024).

[D] Elhage, et al., "Toy Models of Superposition", Transformer Circuits Thread, 2022.

[E] Peters, Jonas, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT press, 2017.

I thank the authors for their detailed response.

I am updating my score by one point, given the new results comparing reconstruction to the SAE baseline with a significance test. I will also respect the decision of the ACs on this work.

As a side-note, the statistical test on the comparison to SAEs seems to be missing a power value for indistinguishability - what is the probability that the null hypothesis would be rejected if an alternative hypothesis was true? Was it by random chance that the p-value was not significant? What was the sample size of the experiments run?

The claims regarding improvements on concept relation recovery and absorption are strong, but they appear to be missing p values, so I cannot evaluate their statistical soundness.

We sincerely appreciate the reviewer's detailed feedback and constructive suggestions. We address each concern below and will incorporate these improvements into our revised manuscript.

W1. The motivation for extending prior temporal CRL approaches to high-dimensional settings feels somewhat unclear. The proposed method relies on the assumption of a linear mixing function between concepts $z$ and observations $x$. Therefore, comparisons to existing CRL methods such as IDOL which are designed to handle the more general nonlinear case do not seem entirely appropriate. I suggest the authors tone down this aspect of their contributions. As currently framed (e.g., lines 57–58), the claim that this work significantly advances prior CRL methods appears overstated.

AW1: We acknowledge this concern and here revised our framing to better reflect our contribution: "this work provides first scalable CRL framework capable of LLM interpretability and representation learning."

Meanwhile, please also allow us to clarify our motivation. Our ultimate goal is to reliably discover the concept-to-concept relations behind the LLM activations. Temporal CRL methods such as IDOL are very good fit for this task, since they provide both empirical success in other domains and, more importantly, identifiability guarantees. But almost all existing CRL methods heavily suffer from scalability issues (as shown in Figure 5, they cannot scale beyond 200 dimensions). This naturally guided us to find a proper simplification for computational feasibility, in which linear approximation is one of the options.

Luckily, existing work including the Linear Representation Hypothesis together with Sparse Autoencoder studies demonstrated an empirical result that LLM activations can be modeled via linear model. Such evidence makes the linear simplification / assumption less problematic in this specific LLM activation setting.

W2,W4. The synthetic data results in Section 5.1 serve as a useful sanity check; however, a more comprehensive analysis of concept identifiability using (semi-)synthetic language benchmarks with ground-truth concepts would strengthen the evaluation. For instance, the authors could adapt synthetic benchmarks from SSAE [1] to the temporal domain. This would provide clearer insights into whether LLMs can recover true concepts from language-based synthetic datasets. While the case study results are interesting, the authors could further evaluate their approach on related tasks such as finding steering vectors using identifiable concept learning methods with LLMs [1, 2]. It would be valuable to know if the proposed approach can infer accurate steering vectors, and if not, what limitations the authors anticipate.

AW2,W4: Thank you for this insightful suggestion. We agree that synthetic benchmarks like SSAE [1] can offer valuable insights into concept identifiability. Our primary focus is on identifying concept relations, though we acknowledge its close ties to concept extraction and steering vector interpretation.

To address this, we tested whether our model can recover steering vectors from paired text. Specifically, we constructed five categories of word pairs where only a single interpretable concept changes, including gender, plurality, comparative, tense, and negation. While these changes are intuitive, ensuring the word pairs capture a clear ground-truth concept is non-trivial. Despite this challenge, our model demonstrated strong performance in identifying the underlying concept differences. Specifically, (1) the average correlation of concept differences within each category exceeded 0.86; (2) assuming one ground-truth pair, the correlation rose above 0.93; and (3) the maximum correlation within a category reached over 0.94. These results support our claim that our model can indeed recover meaningful steering vectors.

Moreover, building on your suggestion, we adapted SSAE [1] to test whether our model captures meaningful text relations. We constructed two contrastive collections of texts drawn from the Pile dataset. One collection consists of legal documents, which often follow structured syntactic formats from start to end, leading to stable temporal concept patterns. The other contains non-legal texts without such structure. We hypothesized that only legal texts would yield meaningful relational patterns. Indeed, we found that in the legal texts, two consistently activated concepts demonstrated a strong temporal relation, with a normalized weight of 0.70 in our recovered $B$. These concepts did not fire in non-legal texts, offering a clear contrast.

These additional experiments reinforce our model’s ability to recover accurate, interpretable concept representations and relations. We have updated our manuscript to incorporate these experiments and expand on the related discussion. Thank you for your suggestion again.

W3. The observation that a sparsity penalty of $\beta=0.01$ outperforms $\beta=0.1$ for learning sparse causal mechanisms is somewhat surprising. To better understand this behavior, I recommend the authors conduct experiments with a broader range of $\beta$ value, such as randomly sampling from the interval $(0.001, 0.1)$ to analyze the trend. Additionally, an important sanity check would be to evaluate the case $\beta=0$ which removes the sparsity regularizer entirely. Moreover, including ablation studies on the hyperparameter $\alpha$ would strengthen the paper; it would be valuable to see how variations in $\alpha$ affect training dynamics and the recovered mechanisms.

AW3: Thank you for the constructive suggestion. We conducted additional comparisons with $\beta = 0$, $0.001$, $0.005$, $0.05$, $1.0$ and $\alpha = 0$, $0.001$, $0.01$ to cover a broader hyperparameter range, using $\tau = 5$ and feature dimension 3072. The results are shown in the two tables below, with our selected setting in italics. The tables highlight that (1) concept relationships are inherently sparse, while a large $\beta$ disrupts optimization, and (2) $\alpha$ has a stronger effect, with 0.1 being a well-balanced choice. We also expanded our manuscript to include a broader discussion of model dynamics with respect to $\beta$ and $\alpha$.

W5. The paper overall is well written, however, the writing of the theoretical aspects needs improvement.

AW5: Thank you for this detailed feedback on improving the theoretical presentation. We have addressed all the suggested modifications and have updated to the paper:

1. We have relabeled our main result as "Proposition 3" instead of "Theorem 3" to more accurately reflect that it builds upon the foundational Theorems 1 and 2 from prior work.
2. A component‑wise transformation on a vector $\mathbf{x} = (x_1, \dots, x_n) \in \mathbb{R}^n$ is a map $T : \mathbb{R}^n \to \mathbb{R}^n$ that permutes the coordinates using a permutation $\sigma$ and applies possibly different scalar functions $f_i : \mathbb{R} \to \mathbb{R},\; i \in [n]$, resulting in $T(\mathbf{x}) = (f_1(x_{\sigma(1)}), \dots, f_n(x_{\sigma(n)}))$.
3. The $\mathcal{M}_{c_t}$ denotes the Markov network defined on the variable set of consequtive timestamps.
4. Isomorphism of Markov networks is formaly defined as follows. Let $V(\cdot)$ be the vertex set of any graph. An isomorphism of Markov networks $M$ and $\hat{M}$ is a bijection between the vertex sets of $M$ and $\hat{M}$, $f : V(M) \to V(\hat{M})$ such that any two vertices $u$ and $v$ of $M$ are adjacent in $M$ if and only if $f(u)$ and $f(v)$ are adjacent in $\hat{M}$.

Q1-4. Comments and clarifying questions

AQ1-4: Thank you for pointing these out. We have corrected the typo on line 77: the matrix $A$ should indeed have dimensions $m \times n$, as $\mathbf{x} \in \mathbb{R}^m$ and $\mathbf{z} \in \mathbb{R}^n$. The expression on line 146 has also been corrected to $z_{t,i}$.

Regarding the symbol $c_t$, we acknowledge the inconsistency in its usage across Theorems 1–3. In Theorem 2, $c_t$ refers to a context of two consecutive time steps, while in Theorems 1 and 3, it involves three time steps. To clarify, we will explicitly label the context length as $c_t^{(2)}$ and $c_t^{(3)}$ respectively.

We also now specify that the diagonal matrix $D$ on line 131 is required to be invertible.

Q5. Could the authors explain the intimate neighbor set used in Theorem 1?

AQ5: Certainly. The Intimate Neighbor Set is defined as follows: consider a Markov network $M_{Z}$ over variable set $Z$. The intimate neighbor set of variable $Z_i$ is given by

$\Psi_{M_Z}(Z_{i}) \triangleq \{ Z_{j} \;|\; Z_{j} \text{ is adjacent to } Z_{i} \text{ and is also adjacent to all other neighbors of } Z_{i}, Z_{j} \in Z \setminus \{Z_{i}\} \}$

This definition ensures that $Z_{j}$ shares maximal context with $Z_{i}$ in the conditional dependency structure, hence quantify the sparsity of the latent process. We have added this definition to the footnote on page 4 for clarity.

Thanks a lot for your response! My concerns regarding the theoretical aspects of the work are resolved and I am glad that authors agree stating results as Proposition instead of Theorem would be better. Further, I am happy with the additional experiments on finding steering vectors, they definitely make the paper strong. I have the following comments.

• I am still not convinced by the motivation of scaling up existing CRL methods. As stated before, prior CRL methods were not specifically designed for the linear case, hence they cannot scale up with large latent variables. The authors can state in the introduction that leveraging the linear representation hypothesis, we design method specifically for the linear mixing scenario. Comparing the proposed approach against IDOL which is designed for a general case and not specialized for the linear case seems incorrect. I suggest the authors should move this experiment to appendix and instead focus on adding the new experiments where they show the method can recover latent variables from textual data and find steering vectors.

• Regarding the ablation on $\beta$, its weird that $\beta=0.0$ also works quite well. It obtains Bs Sparsity (L1) as $0.0003$ while the best choice for $\beta$ leads to $0.0007$, is that difference meaningful? Maybe the authors should be incorporating sparsity penalty in a different manner as the optimization dynamics with the current objective seem non-intuitive.

I would be happy to raise my score to leaning to accept (4) but I think the paper in its current form has issues as highlighted above, which make me hesitant to increase the score to accept (5). I will engage in discussions with the AC and other reviewers later and make my final decision then.

Thanks again for the very thoughtful feedback and for being open to raising your score! Your comments have been very helpful for sharpening the paper. We are happy to address both points.

On the motivation and IDOL comparison:

We completely agree with your comment. We appreciate this great point that helps us sharpen the focus of our work. Yes, our primary contribution is not beating existing methods such as IDOL, but rather leveraging the linear representation hypothesis to design a CRL method specifically tailored to the linear mixing scenarios found in large-scale LLMs. The goal is to bridge the gap between theoretically rigorous CRL algorithms and their practical application to modern NLP, thereby encouraging the CRL community to make theoretical advances more accessible for large-scale empirical studies.

Following your valuable suggestion, we have revised the manuscript to reflect this focus. (1) The introduction now explicitly frames our work as a specialization for the linear case, motivated by the need for practical CRL tools for LLM analysis. (2) We have moved the scalability discussion and the direct comparison with IDOL from the main experiments section (section 5.1) to the appendix. (3) The main body of the experiments section now includes the new results on identifying steering vectors and concept relations, which better showcase the unique capabilities of our approach.

We hope these changes address your concern on this point. We thank you again for guiding us toward this clearer presentation.

On the ablation result for $\beta$:

We appreciate you pointing out the seemingly counterintuitive results, and we are happy to provide more context on the optimization dynamics.

First, as you noted, the absolute sparsity values (0.0003 vs. 0.0007) both seem negligible, particularly when considering that the Independence Loss term is approximately three orders of magnitude larger. Comparing these minimal values might not yield meaningful conclusions about model performance.

Second, we wish to point out that the architecture may embed an inherent sparsity constraint on the relation matrices even without an explicit L1 penalty. For example, the TopK nonlinearity naturally encourages sparse activations, which in turn leads to sparse relationship matrices ($B$ and $M$). Furthermore, standard regularization, like weight decay, also contributes to this effect. In fact, when we attempted to train the model with both $\beta=0$ and no weight decay, the training process became unstable, showing the effect of these implicit sparsity-inducing mechanisms.

Third, the optimization process involves a trade-off between multiple objectives, not just minimizing sparsity, which means the choice of $\beta$ impacts the other loss terms. We found that the $\beta=0$ case, despite its low L1 value, achieved a higher Independence Loss (0.1522) compared to our chosen $\beta=0.01$ (0.1448). Since the independence of the noise terms is a cornerstone of our method's identifiability guarantees, successfully optimizing this loss is critical. This demonstrates that focusing solely on the final sparsity value can be misleading (e.g., zero sparsity trivially leads to an empty and meaningless causal graph); considering the overall optimization landscape and adherence to theoretical assumptions should be more reasonable.

Finally, regarding the choice of sparsity constraint, we acknowledge that the L1 norm is a practical proxy for the theoretically desired L0 norm. As is well-known, the non-differentiable nature of the L0 norm makes it very challenging to optimize directly. As the first work to handle real, LLM-scale data with a provable CRL method, we used the most direct and widely adopted solution in the sparsity constraint literature. We acknowledge this challenge and leave more detailed exploration of this topic for future work.

Thanks again for the constructive review! We have integrated this detailed discussion into the appendix of our revised manuscript to clarify these dynamics for readers. We hope these changes address your concerns and look forward to the opportunity to engage in any further conversations with you.

Dear Reviewer gsa4,

Thank you again for your detailed and constructive review. We sincerely appreciate your thoughtful suggestions, which have helped us strengthen several aspects of our paper.

We hope that our recent responses have addressed your remaining concerns and clarified the points you raised. As the discussion period nears its end, we would be grateful for any further feedback or questions you may have. If appropriate, we would also appreciate an update to your recommendation. We would be happy to provide any additional clarifications that could assist in your final assessment.

Sincerely,

Authors of Paper 7618