Identifiability Guarantees For Time Series Representation via Contrastive Sparsity-inducing

Dear zkBJ,

Thank you for your thoughtful and detailed feedback on our work. We greatly appreciate the time and effort you put into reviewing our paper and highlighting its strengths. Below, we provide a detailed response to each point raised:

Clarification of the setting

a(i). Unobserved States Sources in Figure 1

We apologize for any confusion regarding Figure 1. The "unobserved states sources" refer to hidden components modeled as part of the generative process. To improve clarity, we will update the figure and revise the main text to ensure these components are properly defined and consistent with the modeling assumptions.

a(ii). Notation in Equation (2.1)

We apologize for any confusion and have clarified the notation as follows:

• $\mathbf{x}_{i} \in \mathbb{R}^{C \times T}$ represents a single sample from the dataset, and we consider a set of $N$ samples across the entire dataset.
• In our setting, there are $n$ sources $\mathbf{y} = \{y_1, \dots, y_{n}\}$, where each source $y_k \in \mathbb{R}^{T}$ contribute to the mixed signal $\mathbf{x}$. Specifically, $\mathbf{x}$ is the sum of $y_k$ at each timestamp $t \in \{1, \dots, T\}$.
• The index $i$ in Eq. (2.2) refers to a sample $\mathbf{x}\_{i}$ and its corresponding sources $\mathbf{y}\_{i}$ (i.e., the decomposition of $\mathbf{x}_{i}$).

We have updated the text to make this setting more explicit and hope this resolves the concerns raised.

a(iii). Lines 170-173 are unclear

We have clarified the formulation in this section. Specifically:

• $\mathbf{z}$ is generated by $g_{\theta}^{-1}$. Our decoder takes $\mathbf{z}$ as input and reconstructs the output $\mathbf{x}$.

This updated explanation should make the relationship between $\mathbf{z}$ and $\mathbf{x}$ clearer.

a(iv). Views in the Contrastive Formulation

Are $\mathbf{x}$ and $\mathbf{x'}$ different time-series samples? Yes. In the context of our method, "views" (see Fig. 2) represent different time-series samples used to train the model in a contrastive manner.

We have clarified this in the revised version and linked it to the assumptions underlying our model, specifically:

• Partial Pairing and support indices sharing.
• $\mathbf{i}$ refers to the indices of sources active in both $\mathbf{x}$ and $\mathbf{x'}$.

✅ Question: Relation to Nonlinear ICA (lines 123-127)

Thank you for your question. This problem is a source separation task, where the goal is to recover the sources $\mathbf{y} = \{y_1, \dots, y_{n}\}$ from the mixed signal $\mathbf{x}$. While the process involves summation, it remains nonlinear due to:

• Energy time series data (e.g., NILM): Nonlinearities arise from interactions between multi-state appliances, power distortions, and continuous power fluctuations [1].
• Audio data: Nonlinearities arise due to harmonic distortions and reverberations [2].

Clarification of our claims

2(a): Lines 49-50 - Ill-defined risks leading to unstable and unreliable model outputs Thank you for pointing this out. In the introduction, we mentioned a series of related works studying this issue. We have now added a more in-depth discussion to ensure the claim is properly supported.

2(b): Lines 203-204 - "First identifiability study in real-world time series representation" -> "We clarify that our work focuses on enhancing disentanglement and generalization in time series, not claiming to be the first on identifiability". The phrasing has been revised to avoid ambiguity and properly attribute prior work. Relevant work [3], already discussed in the initial version, is now more explicitly highlighted. We appreciate your comment.

2(c): Lines 209-210 - "We place no assumptions on..." vs. Line 159 (GMM assumption)

• By "no assumption," we mean that we do not assume independence in the distribution (i.e. we do not assume independence of sources), as done in [3,4].
• We clarify that the GMM prior is a weaker assumption, generalizable to exponential family mixtures [5] (Kivva et al.). Moreover, GMMs can approximate complex distributions [6], preserving the flexibility and generalization of Eq. (2.2). In our approach, we impose no constraints on: (1) ReLU architectures, (2) independence of $\mathbf{z}$ or (3) the complexity of the mixture model or neural network.

See Part2

References

• [1] Michalec et al., "Impact of harmonic currents on power quality," Energies, 14.12 (2021).
• [2] Goldstein, "Auditory nonlinearity," JASA, 41.3 (1967).
• [3] Hyvärinen et al., "Nonlinear ICA for disentanglement in unsupervised deep learning," Patterns, 4.10 (2023).
• [4] Hyvärinen et al., "Nonlinear ICA using auxiliary variables," AISTATS, 2019.
• [5] Kivva et al., "Identifiability of deep generative models," NeurIPS, 2022.
• [6] Nguyen & McLachlan, "On approximations via convolution-defined mixture models," Comm. Stat. Theory Methods, 2019.

Dear Reviewer zkBJ,

Thank you once again for your valuable feedback. This is Part 2 (see below for Part 1), where we address the remaining points. We’re happy to answer any further questions. The revised version has already been uploaded, and we kindly request a reconsideration of the current score.

Organization of Section 4

b(i): What is $\mathbf{z}$ vs $\hat{\mathbf{z}}$?

$\hat{\mathbf{z}}$ refers to the learnable latent representation of the ground truth. While this was initially included in the notation section, we have now explicitly clarified it in Section 4.

b(ii): Assumption 4.1 Missing Details

Assumption 4.1 (Sufficient Partial Selective Pairing) ensures enough pairs $(\mathbf{x}, \mathbf{x'})$ share some sources except for a specific source $k$. Unlike stricter requirements in [1], our assumption allows overlaps, provided the union of shared indices spans all sources, enabling flexible modeling.

b(iii): Lines 288-289 (Sparsity-Inducing)

Assumption 4.1 implies that at least one source is inactive, naturally inducing sparsity. Support indices $\mathbf{i}$ define active appliances in pairs $(\mathbf{x}, \mathbf{x'})$. We’ve clarified this further in the revised version.

b(iv): Claimed Theoretical Contribution

Thank you again for pointing this out. The proof was initially included in the appendix, but due to an oversight, it was not uploaded correctly to OpenReview. We’ve now formally stated the results in Theorem 4.2, with proof sketches in the main text and complete proofs in the Appendix.

b(v): Compositional Generalization.

We clarified that $\hat{\mathbf{z}}$ and $\hat{\mathbf{z'}}$ are latents learned by the autoencoder $(\mathbf{\hat f}\_{\phi}, \mathbf{\hat g}\_{\theta})$ for $(\mathbf{x}, \mathbf{x'})$, respectively. The decoder enforces inversion of the encoder. Composing $\mathbf{z}\_{c}$ from $(\hat{\mathbf{z'}}, \hat{\mathbf{z}})$, decoding it via $\mathbf{\hat g}\_{\theta}$, and re-encoding with $\mathbf{\hat f}\_{\phi}$ must satisfy $\mathbf{\hat f}\_{\phi}(\mathbf{\hat g}\_{\theta}(\mathbf{z}\_{c})) = \mathbf{z}\_{c}$ to ensure compositional generalization, i.e., $\mathbf{g}\_{\theta}^{-1}\circ \mathbf{g}\_{\theta}(\mathbf{z}\_{c}) = \mathbf{z}\_{c}$. See lines 320–357 for details.

b(vi): Optimization Objective/Algorithm

The complete loss function and hyperparameters are now detailed in Section 4.1 for more clarity.

💬 Answers to Questions:

1. What is a "latent slot"? A "latent slot" refers to a latent vector in space, i.e., $\mathbf{z} \in \mathbb{R}^{d \times n}$. This is defined in Section 2 and aligns with concepts familiar in the field of disentanglement, as discussed in (Locatello, 2020) [2] and (Wang, 2023) [3].

2. Line 159: Gaussian Mixture Model (GMM)

• We’ve provided the full formulation in lines 162-169 and Algorithm 1 (Appendix 1.3) to clarify the components and their relevance.

3. In Line 144-145, what is $\mathcal{M}\_{+}^{1}(\mathcal{X})$? . By $\mathcal{M}\_{+}^{1}(\mathcal{X})$, we refer to the probability positive measure of the set $\mathcal{X}$. We have updated the text for more clarification, apologies for any confusion.

4. Line 144-145, Line 257: Small Magnitude vs Zero Components True, components may be near-zero due to the condition on the ratio of mean and variance in $\log \sigma^{2}_\phi(\mathbf{x}_i)$. In practice, we observed some values à ~1e-4. We have correctly discussed this in the revised version.

5. Why not use linear ICA? Thank you for your question. As mentioned in lines 125-128, nonlinearities such as distortions make linear ICA unsuitable. However, we have included nonlinear ICA baselines such as TCL, iVAE, SlowVAE, and TDRL.

Comments on Figure 1 (Now Figure 2)

• 6.a: ✅ Please note that we have to make sure that the figure it well described in both section 4 and the introduction.
• 6.b: There are 4 OFF/ON views, but 5 state variables We note that the Noise contributes to the 5th state. This is clarified in the figure. ✅
• 6.c: Slot numbering corrected. ✅
• 6.d: What is "stop-gradient"? We have provided explanations for “Stop-gradient", in the main text and also in the figure. ✅
• 6.e: Cropped figure fixed. ✅
• 6.f: Consistent variable use in the figure and caption. ✅
• 7 Spelling Errors: We have thoroughly reviewed the text to address grammar and clarity issues.

Thank you for your efforts and consideration. The revised version has been uploaded ✅, and we kindly request a reconsideration of the score. We kindly ask that if our responses address your concerns, you consider updating your review to reflect the latest changes.

Reference:

• [1] Hyvärinen et al., "Nonlinear ICA using auxiliary variables," AISTATS, 2019.
• [2] Locatello, F., et al. "Object-centric learning with slot attention." NeurIPS, 2020
• [3] Wang, Y. et al. Slot-vae ICML, 2023.

Dear Reviewer DZWn

Thank you for your thoughtful and thorough review of our work. We greatly appreciate the time and effort you devoted to reviewing our paper and highlighting its strengths. In response to your feedback, we have carefully revised the manuscript. Below, we provide detailed responses to your comments, and we have released an updated version of the paper incorporating your valuable suggestions.

🔹 Identifiability Results
Thank you again for pointing this out. The proof was originally included in the appendix, but due to an oversight, the correct version was not uploaded to OpenReview. We have now formally stated the results in Theorem 4.2, with proof sketches in the main text and complete proofs in Appendix A.3.

🔹 Additional Time Series Comparisons Provided
Thank you for your comment. In the initial version, we compared our work to SlowVAE, which is somewhat similar to TDRL results in our experiments. In the revised paper, we have also included comparisons to TDRL, IVAE, LEAP, and TCL. This strengthens the contribution of our paper by providing a more comprehensive comparison.

☑ TDRL ☑ iVAE, ☑ LEAP ☑ TCL

🔹 Assumption 4.1 and Structural Sparsity
We appreciate your suggestion to discuss the differences between our assumption and assumption-6 presented by Hygiene [1]. In our original version, we focused more on the assumption presented by Zheng [2] (joint work with Ignavier Ng), which is similar to the work by Lachappelle [3] (Structural Variability). In the revised version of our work, we have included a more detailed discussion in lines 267-269. The key differences are as follows:

1. Structural Sparsity (Assumption-6 of [2]), ensures that each pair of sources influences distinct observed variables.
2. However, in real-world time series, overlapping influences often occur, presenting practical challenges (as discussed in App. A.5). Our Partial Selective Pairing assumption (Eq. 4.1) allows some overlap, provided that the union of shared support indices (excluding the specific source) spans all sources. This enables more flexible modeling of source dependencies under contrastive learning. 3- We provide an example in the Appendix A.5 to validate our assumption 4.1, as well as assumption 6 in [1].

🔹 Experiments in Table 1.1

We have added additional experimental results and clarified this in Remark C.1. Initially, when considering only 3 factors ({FR, THR, HTR}, where FR = Fridge, DW = Dishwasher, WM = Washing Machine, HTR = Heater, LT = Lighting), the results for S3VAE+HFS, C-DSVAE+HFS, SparseVAE, and TimeCSL were quite similar due to common signal combinations. However, when more factors ({FR, DW, WM, HTR, LT}) were included in training and testing, we observed distinct differences. These results are discussed in detail in Table 5 of Appendix B.9.1.

We also want to emphasize that we have provided checkpoints and code for reproducibility. To further support clarity and reproducibility, we’ve included an example.

🔹 Checkpoints & Code

• Please there is no main.py, we use train.py, and src_timecsl/evaluation.py for evaluation
• The architecture of the network is depicted in Figure 3 in our paper, with the implementation available in /src_timecsl/models/timecsl.py.
• Run command:

cd src_timecsl/
python train.py --dataset_path "./datasets/data/ukdale.csv" --model_name "TimeCSL" --num_slots 5 --epochs 200 --use_generalization_loss True

Thank you once again for your insightful feedback. We believe these revisions strengthen the paper, and we look forward to your further thoughts. We would appreciate it if you could consider adjusting the rating accordingly.

Best regards,
Authors,

References:

[1] Ng, I., et al. On the identifiability of sparse ICA without assuming non-Gaussianity. NeurIPS 2023

[2] Zheng, Y., et al. On the identifiability of nonlinear ICA: Sparsity and beyond. NeurIPS 2022

[3] Lachapelle, et al. Nonparametric partial disentanglement via mechanism sparsity: Sparse actions, interventions, and sparse temporal dependencies.

Dear Reviewer JQPq,

Thank you for your thoughtful and thorough review of our work. We greatly appreciate the time and effort you put into reviewing our paper and highlighting its strengths. In response to your feedback, we have carefully revised the manuscript. Below, we provide detailed responses to your comments, and we have released an updated version of the paper incorporating your valuable suggestions.

Addressing weaknesses and Questions:

1. 🔹 Misspelling in Line 77
   We have thoroughly reviewed the manuscript and corrected the spelling error you pointed out.

2. 🔹 Extending Normalizing Flows or Diffusion Models
   Thank you for your insightful suggestion. Normalizing Flows (NF) and Diffusion Models indeed offer powerful, flexible transformations, and could potentially enhance our framework. We have discussed the use of Diffusion Models in combination with VAE (D3VAE) [1] in our experiments. We are actively exploring this direction and plan to include [2] as additional experiments in a future version of the paper.

3. 🔹 Justification for Assumption 4.1 in Practical Time Series Scenarios

   Thank you for raising this point. Assumption 4.1 is crucial to our approach, and we understand the need for empirical justification regarding its realism in real-world time series. The assumption suggests that the influence of sources on observed variables is partial and selective, i.e., some sources are more influential at different times. In comparison to other work [3,4,5,6], such as Structural Sparsity [5] or Sparse Variability [6], our assumption is more relaxed but still effective.

We conducted experiments with both synthetic (Tables 7-8 in Appendix B.9.2) and real-world data, demonstrating that our model performs well even when Assumption 4.1 is not perfectly satisfied. However, we acknowledge that deviations from this assumption can impact performance. To address this:

• We provide further discussion in Section 4.1 and Appendix A.5 on the realism of Assumption 4.1.
• We propose grouping sources, i.e., considering groups of sources as active at the same time, as a relaxation of this assumption. This approach still allows us to achieve effective disentanglement in real-world applications.

Thank you once again for your insightful feedback. We believe these revisions strengthen the paper, and we look forward to your further thoughts. We would appreciate it if you could consider adjusting the rating accordingly.

Best regards,
Authors,

References:

• [1] Li, Y., et. al. Generative time series forecasting with diffusion, denoise, and disentanglement. NeurIPS 2022
• [2] Sorrenson, et al. Disentanglement by nonlinear ica with general incompressible-flow networks (gin), ICLR 2020
• [3] Zheng, Y., et al. On the identifiability of nonlinear ICA: Sparsity and beyond. NeurIPS 2022
• [4] Lachapelle, et al. Nonparametric partial disentanglement via mechanism sparsity: Sparse actions, interventions, and sparse temporal dependencies.
• [5] Ng, I., et al. On the identifiability of sparse ICA without assuming non-Gaussianity. NeurIPS 2023
• [6] Zheng, Y., et al. On the identifiability of nonlinear ICA: Sparsity and beyond. NeurIPS 2022

Dear Reviewer agV7,

We truly appreciate your initial feedback. Below, we present our responses to Part 2 of your review, with updates based on your suggestions.

Experiments

🔹 Experiments in Table 1.1
We have added additional results, and Remark C.1 now clarifies that when we considered only 3 factors {FR, THR, HTR} (where, Fridge "FR", Dishwasher "DW", Washing Machine "WM", Heater "HTR", and Lighting "LT"), the results for S3VAE+HFS, C-DSVAE+HFS, SparseVAE, and TimeCSL were quite similar due to common signal combinations. However, when more factors {FR, DW, WM, HTR, LT} were included in training and testing, we observed distinct differences. These observations are thoroughly discussed in Table 5 of Appendix B.9.1.

🔹 Experiments in Table 1.2 (MMC, $R^{2}$, Sparsity)
We greatly appreciate your comment. To eliminate any confusion, we have clarified that two versions of MCC are commonly used in the literature [1]. In our initial version, we reported the difference, which is why we stated "Lower is better". We have elaborated on both metrics for clarity:

1. The strong MCC refers to the value before alignment via the affine map $\Gamma$ (we provide a complete procedure in appendix B.4.1 for this alignment).
2. The weak MCC refers to the value after alignment.

In the updated version, we have chosen to present both the weak and strong MCC metrics together. We hope this clarification addresses your concern, and if it meets your satisfaction, we kindly request that you update your review.

🔹 Why the 70%-20%-20% data split?
For clarity, our data split consists of 60% real data (with 10% augmentation), totaling 70% for training, and 20% each for testing and validation. This setup ensures efficient datasets for testing and validation, as explained in the 'Experimentation' section (lines 401-405). We have updated the text to avoid any potential misunderstandings regarding the data split.

🔹 Architecture of the Model:
The model architecture is provided in Figure 3 (page 7), with further implementation details available in Appendix B.5. We hope this answers your question. Please do not hesitate to reach out if you need additional clarification.

🔹 Reproducing Our Results:

1. We have added more clarified instructions for running the code and all requirements. Pre-trained model checkpoints are also provided in the zip file used in the paper with Reproducibility_Guidelines.md documentation.

2. About our implementation Thank you for your comment. You noted that some parts of the implementation seem based on image data rather than sequence data. To clarify, some components are adapted from sequential image ("video") disentanglement [2]. We've included a source map in Appendix B.1 (TimeCSL-Lib) to show the code we used to build our framework.

☑ The GMM-based VAE sampling is inspired by VaDE (Jiang et al., 2016), and we adapted the implementation from https://github.com/mperezcarrasco/ Pytorch-VaDE.

☑ For the Diffusion model D3VAE (Li et al., 2023), we utilized the authors’ implementation from https://github.com/PaddlePaddle/PaddleSpatial/tree/main/research/D3VAE.

☑ TCL model was adapted from https: //github.com/hmorioka/TCL/tree/master/tcl, while the other models are derived from https://github.com/rpatrik96/nl-causal.

Running the Code:

• Please there is no main.py, we use train.py, and src_timecsl/evaluation.py for evaluation
• The architecture of the network is depicted in Figure 3 in our paper, with the implementation available in /src_timecsl/models/timecsl.py.
• Run command:

cd src_timecsl/
python train.py --dataset_path "./datasets/data/ukdale.csv" --model_name "TimeCSL" --num_slots 5 --epochs 200 --use_generalization_loss True

Code Running in Terminal

| Epoch | Valid LOSS | Valid MAE | Test LOSS | Test MAE |
|-------|------------|-----------|-----------|----------|
| 1     | 0.483      | 0.473     | 0.333     | 0.382    |
| 2     | 0.456      | 0.461     | 0.331     | 0.379    |
| 3     | 0.474      | 0.465     | 0.330     | 0.379    |

(best val_loss: 0.456200, current val_loss: 0.474380)

Thank you for your feedback. We’ve addressed your concerns in the updated version and kindly ask you to reconsider the rating based on our clarifications. Please let us know if you have any further questions.

Thank you again for your time.

Best,
Authors

References

• [1] Kivva, B. et al. Identifiability of deep generative models without auxiliary information, NeurIPS, 2022
• [2] Li, Y., & Mandt, S. (2018). Disentangled sequential autoencoder. arXiv preprint arXiv:1803.02991.

Dear reviewer agV7,

Thank you for your initial review of our work. We greatly appreciate the time and effort you put into reviewing our paper and highlighting its strengths. In response to your feedback, we have carefully revised and updated the manuscript. Below, we provide detailed answers to your comments, and we have also released an updated version of the paper incorporating your valuable recommendations.

We hope that these revisions address your concerns and enhance the clarity of our work.

Section 2 - Clarification of the Setting:

• First, this problem separation is kindly studied in the context of nonlinear ICA [1]. We consider a mixing signal $\mathbf{x} \in \mathbb{R}^{C \times T}$ with $C = 1$ (one feature) and $T$ time steps. The signal $\mathbf{x}$ is the sum of $n$ sources $\mathbf{y} = \{y\_1, \dots, y\_n\}$, where each $y\_k \in \mathbb{R}^{T}$ contributes to $\mathbf{x}$ at each time step.

• The index $i$ in Eq. (2.2) refers to a sample $\mathbf{x}\_{i}$ and its corresponding sources $\mathbf{y}\_{i}$ (i.e., the decomposition of $\mathbf{x}_{i}$).

🔹 Does it infer that the mixing function is not injective? No. When $C = 1$, the observed data $\mathbf{x}$ is essentially a single-channel time series. If $T$ (the length of the time series, in our case $T = 256$) is sufficiently large relative to $n$ (the number of sources, in our case $n = 3$ and $n = 2$), the injectivity of the mixing function can still be preserved.

🔹 Difference between $\mathbf{y}$ and $\mathbf{x}$ in Eq. 21:

We apologize for any confusion and have clarified the notation as follows: $\mathbf{x}\_{i} \in \mathbb{R}^{C \times T}$ represents a single sample from the dataset, and we consider a set of $N$ samples across the entire dataset. In our setting, there are $n$ sources $\mathbf{y} = \{y\_1, \dots, y\_{n}\}$, where each source $y\_k \in \mathbb{R}^{T}$ contribute to the mixed signal $\mathbf{x}$. Specifically, $\mathbf{x}$ is the sum of $y\_k$ at each timestamp $t \in \{1, \dots, T\}$. The index $i$ in Eq. (2.2) refers to a sample $\mathbf{x}\_{i}$ and its corresponding sources $\mathbf{y}\_{i}$ (i.e., the decomposition of $\mathbf{x}\_{i}$).

🔹 In Lines 144-145, what is $\mathcal{M}\_{+}^{1}(\mathcal{X})$? By $\mathcal{M}\_{+}^{1}(\mathcal{X})$, we refer to the positive probability measure of the set $\mathcal{X}$. We have updated the text for further clarification. The notation $\mathbf{g}\_{\theta}♯p(\mathbf{z})$ refers to the transformation of a probability distribution $p(\mathbf{z})$ under a function $\mathbf{g}\_{\theta}$, often representing a model or transformation parameterized by $\theta$. This can be understood as the pushforward of the distribution $p(\mathbf{z})$ by the function $\mathbf{g}\_{\theta}$, where the distribution of $\mathbf{z}$ is transformed according to $\mathbf{g}\_{\theta}$. Apologies for any confusion.

🔹 Line 162: recover? As we have reconstructed the $n$ sources $\mathbf{y} = \{y_1, \dots, y_{n}\}$, we recover up to some noise from the mixed signal, thus $\mathbf{x}$.

🔹 Definition 2.2: Thank you for your feedback. Our definition is aligned with standard work on disentanglement and identifiability [3]. We have clarified the distinction between the two concepts and provided explicit definitions for both in lines 193-198. Please refer to the updated definition, as we have aimed to eliminate any potential confusion.

🔹 The ratio $\frac{|\mathbf{\mu}\_{\theta\, k}(\mathbf{x})|}{\mathbf{\sigma}\_{\theta\, k}(\mathbf{x})}$ has been well discussed in lines 1124-1134, focusing on the impact on the sparsity of $\mathbf{\hat{z}}$.

🔹 Assumption 4.1 We would like to kindly clarify that our assumption is inspired by the work of Lachapelle [3] and the Structural Sparsity [4] approach. However, our focus is on pairs $(\mathbf{x}, \mathbf{x'})$ that share some active sources. The shared activation support is denoted by $\mathbf{i}$ (bold), and the union of all $\mathbf{i}$ defines the subset $\mathcal{I}$. The sufficiency is rejected based on the fact that for each factor $k$, we have enough pairs to cover all factors except $k$, denoted as $[n] \setminus k$. We provide further details and examples in Appendix A.5, where we also compare this approach with Structural Sparsity.

To ensure clarity, we have updated the section and made it more explicit. We hope this resolves the concern related to this.

We address other concerns in Part 2. Please refer to that section for further details.

References

• [1] Michalec et al., "Impact of harmonic currents on power quality," Energies, 14.12 (2021).
• [2] Goldstein, "Auditory nonlinearity," JASA, 41.3 (1967).
• [3] Lachapelle, et al. Nonparametric partial disentanglement via mechanism sparsity: Sparse actions, interventions and sparse temporal dependencies.
• [4] Zheng, Y., et al. On the identifiability of nonlinear ICA: Sparsity and beyond. NeurIPS 2022

Dear Reviewer Ubdd,

Thank you for your detailed review and constructive feedback. We appreciate the time you invested in evaluating our work and acknowledging its strengths. We have carefully addressed your concerns in the revised manuscript, which we have uploaded for your review.

Addressing weaknesses :

🔹 Independent vs Dependent Source Separation

We appreciate your question on this point. In our setting, we do not assume that sources are independent because, in practice, sources are often dependent. For instance, the operation of a heater and an air conditioner can be influenced by shared environmental factors such as temperature.

1. Dependency in Time and Context: Sources are often dependent on the time of usage. For example, in the kitchen, activating one appliance increases the likelihood of another being used shortly after.
2. Approach: Our methodology is designed to model and accommodate such dependencies, avoiding the assumption of independence on the prior.

• We have clarified this in the introduction and added references to prior works addressing time-dependency, such as LEAP [1] and TDRL [2].

🔹 Additional Comparisons with State-of-the-Art In the revised manuscript, we expanded our comparison to include TDRL, iVAE, LEAP, TCL, and others, beyond our initial comparison to SlowVAE/DiffusionDisentangledVAE. This broader comparison enhances the paper’s contribution with a more comprehensive evaluation

🔹 Provable Identifiability Results

You noted an issue with the missing proof in the initial submission. We have addressed this by formally stating the results in Theorem 4.2. Proof sketches are now provided in the main text, with complete proofs in Appendix A.3 for further clarity.

🔹 Appendix Manuscript Sections We sincerely apologize for the oversight in our initial submission, where the incorrect appendix version was uploaded. The revised manuscript now includes the correct appendix and all the missing sections to ensure the completeness of the manuscript.

💬 Answers to questions

1. Line 76: "Sparsity alone is insufficient to ensure reliable identifiability..."
   This is rooted in Darmois' theory (1953) [1] which shows that nonlinearity can cause unidentifiable representations even when sparsity the identifiability problem preexists, as sparsity does not alter the inherent properties that lead to unidentifiable representations. For example, the latent space $\boldsymbol{\hat{y}}\_{k}$ may depend on multiple latent slots $\mathbf{z}$, leading to uncertainties. We clarified this in lines 287-289 of the revised manuscript.

2. Relationship Between $\mathbf{y}$ and $\mathbf{z}$
   You are correct. In our nonlinear ICA setting, $\mathbf{x}$ is generated from a latent space $\mathbf{z}$ via a nonlinear function $\mathbf{g}\_{\theta}$. For identifiability, each source $k$ is controlled by $\mathbf{z}\_{k}$: $\boldsymbol{y}\_{k} = \mathbf{g}\_{\theta, k}(\mathbf{z})$ and $\mathbf{x} = \sum\_{k=1}^{n} \mathbf{g}\_{\theta}(\mathbf{z})$. Accurately estimating the latent allows recovery of $\hat{\boldsymbol{y}}\_{k}$ (i.e, source $k$) summing to $\mathbf{x}$. $\mathbf{g}\_{\theta}(\mathbf{z})$ captures the full output, not just latent/slot $\boldsymbol{z}\_{k}$.

3. Model Input and Framework
   Thank you for your question. We’ve added Figure 4 to illustrate the framework. While labels for active sources are helpful, they are not strictly required. Instead, we use pairs $(\mathbf{x}, \mathbf{x'})$ with shared source activation. For example, in Figure 2, $\mathcal{S}(\mathbf{x}) = {1, 2, 4, 5}$ and $\mathcal{S}(\mathbf{x'}) = {2, 3, 4, 5}$, giving shared support indices $\mathbf{i} = {2,3,4,5}$.

• Our approach can work in an unsupervised or semi-supervised manner (with a small labeled dataset), where we minimize $||\sum_{k=1}^{n}(\mathbf{\hat g}_{\theta k}(\mathbf{\hat z})) - \mathbf{x}||_2^2$.
• In a fully supervised setting (with $\mathbf{y}$ available), we minimize $||\sum_{k=1}^{n}(\mathbf{\hat g}_{\theta k}(\mathbf{\hat z}) - \mathbf{y}_k)||_2^2$.

This makes the approach more flexible, but labeled data can improve results.

4. Evaluation Metrics
   We provided both strong MCC and weak MCC metrics in the original manuscript. Strong MCC refers to values before alignment via the affine map $\Gamma$, while weak MCC is measured after alignment. The procedure is detailed in Appendix B.4.1. We clarified this further in the revised text.

We hope these updates and clarifications address your concerns. If you find the revisions satisfactory, we kindly request that you consider updating your review.

References

[1] G. Darmois. Analyse des liaisons de probabilité. In Proc. Stat, 1951. [1] Weiran et al. Temporally Disentangled Representation Learning. NeurIPS, 2022.
[2] Weiran et al. Learning Temporally Causal Latent Processes from General Temporal Data. NeurIPS, 2021.

Thank you for your feedback. We would like to emphasize that we took all necessary measures to ensure anonymity during the submission process of the code. However, it appears there was an unexpected issue with the initial link, which was directed to the Huggingface account instead of the intended TimeCSL account. We are unsure how this redirection occurred, as it was not our intention, and we sincerely apologize for any confusion it may have caused. We appreciate your understanding and will double-check our processes in the future to avoid such occurrences.

I would like to clarify that in the previous version of the paper, the link in the "anonymous" GitHub repo in the paper had a README, which linked to the author's personal Huggingface account. This is an oversight on the author's part, not an attempt by me to subvert the anonymous review process. I would advise that the authors are more careful in their future submissions instead of blaming the reviewers for their own negligence.