TransPL: VQ-Code Transition Matrices for Pseudo-Labeling of Time Series Unsupervised Domain Adaptation

We thank the reviewer for thoroughly going through our manuscript, and providing valuable comments. We want to highlight points that were misunderstood and should be clarified. While we wish to thank you individually for each point, we are constrained by the length limit. Thank you for your understanding.

C1. Missing Ref. ACON. We will add them to our updated manuscript.

C2. SFUDA. This is a misunderstanding. Our TransPL is not source-free, as it utilizes the source training data alongside the pseudo-labeled target training data for adaptation, as noted in Line 162: where the model is fine-tuned using the labeled source and pseudo-labeled target data for domain adaptation. We note that our work can be extended to the SFUDA, if we only utilize the target data for adaptation, but we believe this is out of our scope where our current focus is on the UDA setup. However, based on the reviewer's insight, we found that this could be better clarified. As such, we added The adaptation is complete by fine-tuning the model with both labeled source and the pseudo-labeled target training set in Lines 321.

Q1. Weakly-Sup. UDA. The term Weakly-Supervised means that the label distribution of the target domain is given as an additional source of information. Such scenarios are highly practical as users might not label all of their data but can self-report on the proportion of time they have spent on each exercise. This term was first used in time series by CoDATS [KDD'20] (we would like to address the reviewer to Lines 100-109 and Lines 353-355, where we have provided explanations and proper citations needed).

Q2. VQVAE. The use of VQVAE is essential, and all the proposed works build on top of VQ. We are not aware of other architectures that could better replace VQVAE. Simply, a patch can now be represented as discrete codes. These discrete codes are then used for modeling the temporal transitions and channel-wise shifts, which are the main contributions of this work. Here, VQ enables the mapping of each time series patch into coarse code vectors. This makes the semantics of the time series intact, while making the transition matrices sensible.

Q3. Recon. Performance. The reviewer is correct. Reconstruction is important for our use, as good reconstruction ensures meaningful code vectors to be constructed, which is critical for valid transition matrices. Constructing non-representative code vectors that can't represent the time series task at hand leads to meaningless transition matrices, leading to degraded pseudo label accuracy. In Table 1, we show that obtaining good reconstruction performance as well as making the transition meaningful (by limiting the # of coarse codes) are both essential to making accurate pseudo labels.

Q4. Recon. and Classification. While there is no strict guarantee that the representations for classification and reconstruction are identical, we argue that they are well aligned for the following:

1. Empirical Evidence: Our results show strong performance in both classification and reconstruction across multiple data, with no signs of overfitting to one task. The pseudo labels consistently achieve the best results, and both losses are well optimized, leading to stable outcomes.
2. Prior Works: Prior works in time series [1,2] have successfully used VQVAE for both reconstruction and classification. Additionally, optimizing auxiliary tasks alongside reconstruction is a common practice in the broader literature [3].

[1] TOTEM: TOkenized Time Series EMbeddings for General TS [TMLR'24]

[2] VQ Pretraining for EEG TS with Random Projection and Phase Alignment [ICML'24]

[3] DeWave: Discrete EEG Waves Encoding for Brain Dynamics [NeurIPS '23]

Q5. While we have empirically observed that zero dead coarse code was achieved in all of our tasks, we acknowledge that this is a practical result. To practically ensure this, we have explicitly designed the coarse codes to have a limited number of codes (N=8; modeling the trend pattern) compared to the more large use of code size (N=256, 512), making sure that there is a higher chance of using all codes in the coarse codebook to map the trend pattern. However, we understand this is a practical outcome and would like to include it in the limitation section.

Q6. EMD was necessary, as different codes may contain similar semantics. For instance, if code 1 and code 2 capture similar information, the transition between 1->1 and 1->2 should be deemed similar. Other metrics such as MSE cannot take this semantics into account (they treat 1 and 2 differently), while EMD can incorporate such information with the use of cost matrix M (defined as similarity in Eq 6). We direct the reviewer to the section "Channel Alignment via Optimal Transport, where we have provided a detailed example and the motivation behind the use of EMD.

Q7. Eval. protocol. We direct the reviewer to our response E1 of Reviewer n6fN (3rd reviewer)

We first and foremost thank the reviewer for the thorough review of our work. We greatly appreciate your services. Here, we have prepared a detailed explanation to address each of the reviewer's questions.

E1. Performance of Baseline Methods. Thank you for pointing this out. The performance gap stems from our use of patch transformer versus the 1D-CNN backbone in previous works. For fair comparison, we standardized all baselines with the same transformer architecture. While transformers typically require larger datasets to outperform simpler models in time series applications [1], our choice was deliberate: patch transformers preserve the semantic meaning of individual time segments (patch) in latent space, making coarse code transition matrices interpretable. As we are trying to model the temporal transition in time series, having meaningful and interpretable patch representation is necessary for our whole methodology. Unfortunately, 1D-CNN representations lack this interpretability as it mixes all features in the latent space. We acknowledge this design choice prioritizes interpretability of the constructed pseudo labels at some cost to overall performance, and we would like to include this in our limitation. Thank you for addressing this point.

[1] Are Transformers Effective for Time Series Forecasting? [AAAI 22]

E2. Applying to 1d-CNN backbones. Thank you for the suggestion. We believe that using 1d-CNN models could lead to enhanced model performance. However, as mentioned in E1, our methodology relies on the use of patch representation to keep the semantics. One possible way is to jointly optimize the 1d-CNN model alongside the patch encoder, but this would increase the computation, and also, we would not be able to solely focus on the performance gain that can be brought with our coarse code transition matrix modeling. As the whole paper focuses on developing pseudo labeling strategies that reflect temporal transitions and the channel-wise shift, the use of patch transformer was necessary.

E3. Need for additional baselines. Thank you for this valuable comment. We acknowledge that some of the baselines were missing (CauDiTS did not release their code), however, we have compared our work to 12 other more relevant baselines. These baselines include relevant works such as RainCOAT (UDA) and T2PL (pseudo-labeling), which help us understand the significance of TransPL. While adding additional baselines could be helpful, we courteously request the reviewer to have a look at the additional analysis we have performed, which is novel and both needed to support our claims in the paper. For instance, we showed class-conditional likelihoods to

C1. Time Complexity. The dominant computational cost comes from counting code transitions across all source samples. With N_s source samples, N patches per sequence, and D channels, this requires O(N_s × D × N) operations. However, implementation is practical due to (1) efficient vectorization techniques (da_utilities/transition_matrix.py file) for counting transitions and (2) the extremely limited number of coarse codes (only 8). For the UCIHAR dataset (9 channels, 14 patches per sequence), the total transition matrix construction time takes only 9.954±0.478 seconds.

C2. Use of Transition Matrix. The purpose of this section was to highlight the benefits of using transition matrices to construct pseudo labels instead of directly constructing classifiers (1d-CNN, GRU, LSTM) that could be used to predict the pseudo labels. These discriminative models were trained using the (source) coarse code sequences and were trained to predict source classes. However, we show that such discriminative approaches fall behind our use of the generative approach (modeling the conditional likelihoods using the transition matrix), validating the use of transition matrices. Based on the reviewer's suggestion, we will refine this section for better clarity in the updated manuscript.

Q1,2. We believe that we have addressed the questions in E1 and E2.

We hope we have addressed the reviewers' concerns. Please let us know if any points require further clarification.

We sincerely appreciate the reviewer’s thorough evaluation of our manuscript and the valuable feedback provided. We are pleased that the reviewer found our work to have strong experimental results and well-presented visualizations. Below, we have prepared a detailed response to address the reviewer’s concerns.

W1. Justification for VQVAE. We respectfully disagree that VQVAE is merely a technical add-on—it is the cornerstone of our approach, which enables the density estimation of the source (which is infeasible without VQVAE, Sec 4. Source Training)

VQVAE enables learning meaningful discrete representations of time series segments, which is essential for modeling the proposed temporal transitions in TransPL. To provide a very simplistic example, in a sequence like 1,2,5,6,1,2, VQVAE learns to map similar patches to discrete codes (1,2→"A", 5,6→"B"), creating a simplified and abstracted transition sequence A→B→A. Consequently, the use of transition matrices relies on these code transitions, and our proposed class-conditional likelihood generation for pseudo-labeling and the quantification of channel-wise shift rely on these transition matrices. As such, VQVAE is essential to our work. Other methods like SAX rely on predefined quantization schemes that may not capture data-specific patterns as in VQVAE. We would like to address the reviewer to Section 2.3 for detailed reasons behind our use of VQ.

W2. Design Principles We respectfully note an oversight in this assessment. While the design may appear complex initially, each component serves a critical function in our framework. These elements work together to achieve our stated goal: effective time series pseudo-labeling that accounts for both temporal transitions and channel-wise shifts—the core motivations we established at the introduction.

(VQVAE) - Please kindly refer to W1.

Optimal Transport (OT) - TransPL requires comparing coarse code transition patterns between source and target domains while preserving semantic relationships. Unlike L1/L2 metrics that calculate absolute usage differences, OT acknowledges semantic similarity between patches. For example, if patches A and B encode similar information, transitions A→A and A→B should be considered similar. OT incorporates these semantics through a cost matrix defined by patch similarities, enabling a more meaningful comparison of transition patterns that respects the underlying data structure.

Both modules are necessary designs for constructing pseudo labels, and we believe that they are not overly complex. They are the most simplistic method for obtaining pseudo-labels that account for temporal transitions and channel-wise shifts. No other pseudo labels take these into account.

W3. Lack of Ablation for Coarse Code. We direct the reviewer to Table 1, which contains our detailed ablation study on the use of coarse codes. The first three rows evaluate single codebook performance with varying sizes, while the last four rows compare different codebook size combinations for coarse and fine codes. These results demonstrate that our proposed design principle yields the best pseudo-labeling accuracy.

W4. Unclear Motivation. We respectfully disagree that our motivation is unclear. As we have noted in our Introduction, our motivation is to design a pseudo-label method that considers the temporality and channel-wise selective shifts that happen in time series adaptation. No other pseudo label methods consider both problems nor incorporate these into the pseudo labeling process of time series. Other pseudo-labeling methods (SoftMax, NCP, T2PL) rely on the clustering ability, representation performance of the source classifier, and without any focus on time series characteristics, while our methodology focuses on the two most important aspects of time series (time and channel). We have shown in Table 2 that other method fails to obtain strong adaptation performance, and that pseudo label accuracy is low compared to ours in Table 3. We also show that channel-wise alignment distance is better captured through our proposed work. We believe that we have sufficiently addressed our motivation throughout the whole paper, and the experiments conducted are well justified to back our motivation.

We hope that we have addressed the concerns of the reviewers. If there is anything that is still unclear, please kindly let us know.

We sincerely thank Reviewer fkT4 for the thorough, insightful, and constructive feedback. We appreciate that the reviewer has found our work a novel contribution for UDA time series that has not been explored elsewhere. We also thank the reviewer for thinking through our problem definition with us. We have provided detailed responses to address each of its comments.

Q1. Training Cross-Entropy. The cross-entropy is calculated using a [CLS] token appended to the input patches. After processing through the Encoder, this [CLS] token—having attended to all time series tokens via transformer—is used to train the classifier. We will add a guiding arrow in Fig. 1 to clarify this flow. We also refer to Lines 194-196.

Q2. Class Conditional Likelihood. After encoder training, we process unlabeled target time series to obtain discrete code sequences (e.g., A-B-A). We then calculate the likelihood of observing this sequence under each class-specific transition matrix from the source domain, similar to maximum likelihood estimation in Hidden Markov Models. For instance, for a given sequence A-B-A, the Class 1 Transition matrix may output higher likelihood, and the Class 2 Transition matrix may output low likelihood. This allows us to assign the most probable class label to each target sequence. We direct the reviewer to Eq. 7 and Fig.1 for additional clarification.

Q3. Correlations between Channels.

Currently, we use the same codebook to model all sequences in a time series but construct channel-wise transition matrices to represent each channel. As such, we could presumably say that the shared codebook can capture correlations between channels. While we have also tried modeling each channel with separate codebooks, we empirically found that this did not lead to better pseudo-labeling results (It might be due to increased complexity; alternative design methods should be further looked into).

Q4. Transition Matrix Remains Similar. This is also an interesting point. We believe that in such a scenario (e.g., DC shift), the transition patterns would greatly differ, as our codebook does not solely focus on the shapes of the time series pattern. If a large DC shift occurs, it would most likely return different time series signal values that should be reflected in the coarse code usage pattern.

Q5. Channel Importance. We find this really interesting and appreciate this insightful point. The reviewer is correct that only certain channels may contain class-relevant information [1], and when class information is concentrated in channels experiencing significant domain shift, our method would assign lower weights to these channels due to the large EMD distance. This is a limitation we'll explicitly acknowledge in our revision. We plan to extend our approach in future work by incorporating channel importance measures alongside distribution shift metrics, allowing for weighting that considers both channel-shift and the importance of channel for classification.

[1] CAFO: Feature-Centric Explanation on Time Series Classification (KDD'24)

R1. [TMLR paper] We will include the provided reference. We also think it will strengthen our work!

W1. Limitation. As the reviewer has noted in Q5, we believe that TransPL may not operate optimally in cases where the heavily shifted channel contains class-discriminative information, as our pseudo-labeling method would weigh less on such channels. We will incorporate this into our Limitation Section in the updated manuscript.

W2. Additional Analysis. We appreciate the reviewer's valuable insight regarding channel alignment analysis. We propose the following experiment to address this concern:

Analyzing Channel-Level Corruption. We plan to inject random noise into high-weight channels ($w_d$) in the adaptation scenario and measure adaptation performance by tracking the reduction in $w_d$ after noise injection. A reduction would demonstrate the model's ability to downweight corrupted channels. Would the reviewer consider this a suitable approach to provide more thorough evidence of our method's capability to identify and ignore channels with large distributional shifts?

We once again thank the reviewer for providing helpful feedback to improve our work. Please let us know if there are any additional uncertainties so that we can address them. Thank you.