Disentangling Time Series Representations via Contrastive Independence-of-Support on l-Variational Inference

We appreciate your thorough review and insightful feedback on our paper. The suggestions and concerns raised have been duly noted, and we have made significant updates to address them, we will update the current version during this discussion. Major changes in the paper from the previous version will be highlighted in gray color.

Thank you for acknowledging the strengths of our paper in tackling a challenging problem. We note that additional experiments were conducted during the review phase, and our code repos are updated with more ablation studies. This establishes itself as the inaugural framework for disentangling time series in a library.

Modifications

• Sensitivity Analysis: We've integrated supplementary experimental findings and engaged in a thorough discussion covering all aspects of the training process. The forthcoming version will include corresponding adjustments to the codebase, ensuring transparency and reproducibility.

• Hyperparameter Selection Discussion: We have included a detailed discussion on our parameter choices, addressing the sensitivity of our method to data and its correlations.

We believe these enhancements contribute significantly to the overall quality and completeness of our work. Your valuable input has been instrumental in refining our paper, and we are grateful for your time and expertise.

Should you have any further comments or queries, please feel free to let us know. We look forward to your continued guidance.

Best,

Authors,

We appreciate the thorough review and constructive feedback on our paper. The suggestions and concerns raised have been duly noted, and we have made significant updates to address them, we will update the current version during this discussion. Major changes in the paper from the previous version will be highlighted in gray color.

Thank you for acknowledging the strengths of our paper in tackling a challenging problem. We emphasize these contributions more explicitly in the revised version.

We note that additional experiments were conducted during the review phase, and our code repos are updated with more ablation studies. This establishes itself as the inaugural framework for disentangling time series in a library.

Weaknesses and Modifications

We appreciate the feedback regarding the readability and abstract nature of the experimental results and application description. We have addressed these concerns by providing additional details and clarification in the revised version (which will be updated during this discussion). Specific modifications have been made based on the suggestions provided in the Questions section.

[A/Q]

1. We have provided a more elaborate more explanation for Hausdaurff Factorization support (HFS) in the revised version.

2. Section 3.2 is moved to 4.2 and now includes a dedicated illustration (Figure. 3) demonstrating Attentive l-Varitiaonla Autoencodeur along with corresponding notations.

3. Experimental results section now includes specific examples illustrating how $**x**$ and $**y**$ values (see section 5, other results in the appendix) for a time window are displayed for each experiment, addressing concerns about abstractness we add the figure of mixed signal input and unmixed results (prediction).

4. Negative and Positive Samples. We define a positive sample $**x**$ as a mixed power state where an appliance $m$ is activated ($y_m > 0$), and $**x**^{+}$ represents its augmentation. For this setup, labeled data is required. Finding a negative sample is comparatively straightforward, as any other sample $**x**$ can be chosen where $m$ is not activated $y_{m}=0$, but at least one other appliance is activated (no need for their labels). Augmentations (see our answer below) of both positive and negative samples generate sets for positives and negatives for appliance $m$. This process is repeated for $M$ appliances as preparation data.

5. Given access to labels for each appliance, we meticulously consider the correlation and similarity among appliances, assessing whether they exhibit similarities, correlations, or are uncorrelated. In response to your feedback, we elaborate on this aspect in the revised version and explicitly specify the number of samples utilized for training.

6. *Time-based Augmentation. In Section 5.1 experiments, we have provided detailed insights into the augmentation process to enhance clarity. Additional algorithmic details are available in the appendix. The four augmentations were sequentially applied to both negative and positive: 1). Crop and delay: crop and delayed in time by 5 steps with a probability of 0.5. 2). Channel Masks powers: Randomly mask out each power channel (reactive, active, and apparent) with a probability of 0.4. 3). Cutout or Masking: Time cutout of 5 steps with a probability of 0.8. 4). Add Gaussian Noise: Introduces random Gaussian noise to window activation $y_{m}$ and $\mathbf{x}_{m}$ with a standard deviation of 0.1. We add a comparison of performance between different choices of augmentation.

7. Section.3 now provides explicit clarification on the evaluation of appliance dissimilarity in $\mathbf{x}$ and $\mathbf{x}^{+}$ addressing concerns about labeling and similar behavior. Further details on the preparation of negative and positive samples have been included, along with an illustration to enhance clarity.

8. After Equation. 2, we have introduced and defined $K$ and $d_{z}$ fixed before training ($K$ unknowing appliance to find) in the revised manuscript.

9. We favor Cosine Similarity over alternative distance measures based on specific considerations within our context. Our choice is motivated by the computational efficiency of Cosine Similarity compared to Earth Mover's Distance (EMD), as outlined in our study detailed in Appendix C. Notably, we observed that Cosine Similarity exhibits scale-invariance, focusing on direction rather than being affected by the magnitude of the latent. In contrast, EMD incorporates transportation costs between distributions, introducing a sensitivity to scale.

We value your insightful review and remain committed to addressing all raised concerns diligently. Should you have further inquiries about our paper or its underlying assumptions, we are more than happy to provide detailed responses.

Best,

Authors,

Sorry there was a slight problem of pdf upload. Could you please check the last pdf we uploaded. Please if you have any further questions about our paper or its underlying assumptions, we will be happy to provide detailed answers.

Best,

Authors,

We regret the delay in updating the version due to an error on our part in the PDF upload and any resulting confusion about the paper's name. We have now rectified the situation by updating everything, including the separate appendix.

To avoid any potential confusion, we have revised the paper's name from Disco to Diosc: Disentanglement and Independence-of-Support Factorization via Contrastive. We have also been notified that other works [2,3], far from our paper, coincidentally bear the same name. In response to your question, our paper is distinct from [1], and the only common characteristic is both use contrastive methods. We have also modified our citation to acknowledge [1] in related work of our updated version. We clarify the difference below:

1. In [1], the focus is on exploiting in an unsupervised manner the pre-trained generative models non-disentangled such as VAE, and GAN by navigating latent space, specifically using it on a pre-trained GAN. In contrast, our work emphasizes enforcing disentanglement during the training phase of VAE itself, by relaxing the assumption of attribute independence factorization to independence-of-support (refer to assumption 4.1 in our paper).

2. The use of a pre-trained model, assuming independence of factors, is deemed unrealistic in our case. Instead, we focus on identifying factors learned in the latent space, which distinguishes our approach from [1] and during the training phase of VAE. In addition, our contrastive objective is not similar to those of [1]. In our contrastive objective, there are two terms, one for invariance and the other for alignment of latent variables.

Please if you have any further questions about our paper or its underlying assumptions, we will be happy to provide detailed answers.

Best,

Authors

[1] Ren and al. "Learning Disentangled Representation by Exploiting Pretrained Generative Models: A Contrastive Learning View", ICLR (https://openreview.net/forum?id=j-63FSNcO5a)

[2] https://openreview.net/forum?id=oUeYSTIhpE

[3] https://openreview.net/forum?id=GAXedKmbFZ