Exploiting Representation Curvature for Boundary Detection in Time Series

Thank you so much for acknowledging the impact and intuitiveness of our new boundary detection method. We hope that our responses have addressed your concerns and that you are able to raise your rating.

W1 Theoretical analysis with class-separated representations.

(1) You are right. There is no guarantee of class separation, even though it tends to hold in realistic settings [18, 19]. As you might agree, theoretical formulation often needs some kind of assumptions on data distribution. Without the class confinement assumption, RECURVE is empirically proven to work well with real-world datasets, because point representations are at least densely populated for intra-segment points and sparsely populated for inter-segment points, under the reasonable quality of the learned representations. Figures 5, 6, and 7 show that the curvature-based scores are higher for inter-segment points than for intra-segment points. In particular, the 2-dimensional plots in Figure 5 show the class-separated representations.

(2) In fact, we did try the clustering approach. Our experience indicates that there is no guarantee of one-to-one correspondence between clusters and classes. For example, under the "Walk" class, multiple clusters are formed depending on the speed of walking. Thus, clustering (i.e., proper setting of the number of clusters) was not that straightforward for this purpose.

W2 Normalization of the change metric.

Because change point detection (CPD) methods are typically applied to long, continuing time series (e.g., smartphone sensor recordings for a full day), it is reasonable to think that there are state changes. The same normalization is applied to all samples from an individual dataset by treating them as a single time series. Thus, we believe that all your concerns on the normalization should be resolved now. The details on the normalization will be clarified in the final draft. Similarly, the threshold $\varphi$ is set for each individual dataset, as described in Section 3.3.

W3 Missing baselines.

Thank you for your comment. We already have included such baseline for comparison. In fact, what you refer to as DISTANCE(cosine)+TPC is TS-CP$^2$. Per your comment, we add similar baselines, DISTANCE(cosine)+TNC, DISTANCE(Euclidean)+TPC, and DISTANCE(Euclidean)+TNC. Please refer to Table 5 of the PDF file 🔗. Again, RECURVE is shown to outperform all such baselines. Specifically, in the HAPT dataset with $p=5$, the increase in the AUC from TS-CP$^2$ to RECURVE+TPC for gradual changes is 32%, which is significantly higher than the 21% increase for abrupt changes. This finding indicates that RECURVE is notably more efficient in handling gradual changes.

Q1 Quality of the change metric.

This is a really good question! As long as the score values of inter-segment points are mostly higher than those of intra-segment points, having varying degrees is not a bad thing, as you thought. However, as shown in the left plot of Figure 1, a wide range of the score values of inter-segment points may overlap with the score values of intra-segment points, especially for gradual changes. Thus, the accuracy of change point detection is degraded by this overlap in the distance-based method.

Q2 Figure for the scores over time.

Absolutely. Please see Figure 8 of the PDF file 🔗. The CPD scores of RECURVE indicate the inter-segment points much more clearly than those of TS-CP$^2$.

Q3 Performance results across multiple error margins.

Table 2 already contains the performance results across multiple error margins. Per your suggestion, we present the performance results separately for gradual and abrupt changes across multiple error margins in Table 5 (see the PDF file 🔗). As the error margin $p$ increases, the AUC metric increases slightly more for gradual changes than for abrupt changes, though the difference is not very clear. We conjecture that the transition strength is a significant factor in distinguishing between gradual and abrupt changes, rather than the transition duration. Table 5 will be added to the final draft.

Thank you so much for acknowledging the novelty, evaluation result, and intuitiveness of our new boundary detection method.

W1 Locality assumption on the learned representations.

Thank you for your insightful comment. Yes, RECURVE is built upon class-separated representations. We believe that it is a widely known fact that representation (contrastive) learning produces class-separated representations [18, 19]. According to [18], the augmented positive examples form a connected graph based on augmentation overlap; thus, the contrastive learning will cluster the examples of the same class together and lead to class-separated representations. We use two representation learning techniques, TPC and TNC, for our evaluation. Consecutive classes tend to be well separated on the embedding space, as shown in Figure 5. RECURVE is empirically shown to work well with these two representation learning techniques.

As a topic for future work, we will continue to explore the design of a customized representation learning technique for RECURVE. Your comment is indeed inspiring!

[18] Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap, NeurIPS 2022

[19] InfoNCE Loss Provably Learns Cluster-Preserving Representations, COLT 2023

Q1 Optimality under the right settings.

This is a very good suggestion! The Gaussian mixture model can be applied to a set of point representations. Then, based on the probability distribution of a point, some statistical tests can be done to distinguish between intra-segment points and inter-segment points. It is difficult to implement this procedure during the rebuttal period because of the short time allotted, so we will include this comparison in the final draft. Given this setting, where point representations are well clustered, we anticipate that RECURVE will be very close to the optimal boundary detector.

Thank you so much for acknowledging the novelty and intuitiveness of our new boundary detection method.

W1, Q2 Lack of a strong foundational motivation and interpretation of Figure 5.

(1) We anticipate that our clarification will effectively address your concerns.

• Rationale: In Introduction, Figure 1 empirically shows that the distances between consecutive points largely overlap between intra-segment points and inter-segment points, especially for gradual changes (Up$\leftrightarrow$Down). Figure 2 illustrates the foundational motivation of the curvature-based metric. As long as the point representations are confined into a class ball, the trajectory of intra-segment points make sharp turns more frequently than that of inter-segment points, regardless of gradual and abrupt changes. This benefit of the curvature is proven in Theorem 3.8.
• Evaluation: Furthermore, Figures 6 and 7 clearly demonstrate the advantage of the curvature-based metric over the distance-based metric. For gradual changes, e.g., from walking to descending (Walk$\rightarrow$Down), the embedding distances across these two classes are small (Figure 6(a)). Consequently, the distance-based scores become low for these transitions because the embedding distances between consecutive inter-segment points should be also small (Figure 6(b) and the top row of Figure 7). In contrast, regardless of gradual and abrupt changes, the curvature-based scores are sufficiently high (Figure 6(c) and the bottom row of Figure 7).

(2) Figure 5 is obtained using two principle components out of 32 dimensions. Thus, the exact position of each point representation may not be reflected on the 2-dimensional plot. The color of a point indicates the exact curvature score calculated in the 32-dimensional space. The purpose of Figure 5 is to support our motivation that the intra-segment points are somewhat clustered whereas the inter-segment points are not. In order to minimize confusion, we will identify the plots that most accurately represent the original positions in the 32-dimensional space.

In conclusion, we trust that the curvature-based metric's superiority over the distance-based metric is readily apparent to you. We will endeavor to further improve the presentation in the final draft.

W2 Comparison with more baseline methods.

In fact, we choose the representative method from each of the three categories (statistics-based, pattern-based, and representation-based) mentioned in Related Work. Because the representation-based methods are regarded as the state of the art, we add another representation-based method, which is a variation of TS-CP$^2$ with the TNC representation learning technique. RECURVE also beats this additional method, and please see Table 5 in the PDF file 🔗.

W3 Dependency on the learned representation.

We agree with you. However, as representation learning (or contrastive learning) for time series advances remarkably [a], it is unlikely that the quality of the learned representation is very poor. We will discuss this limitation in Appendix D of the final draft. It is crucial to note that, even if the quality of the learned representation is sufficiently high, the distance-based method is susceptible to gradual changes whereas the curvature-based method is not.

[a] Universal Time-Series Representation Learning: A Survey, arXiv:2401.03717, 2024

Q1 Hyperparameter sensitivity analysis.

We have conducted an analysis of the sensitivity of two hyperparameters $w$ and $d^\prime$, as shown in Section 4.4 and Appendix C, respectively. The sensitivity analysis was done using the test set. It was confirmed that the default setting of the two hyperparameters is suitable for achieving competitive performance across for all datasets, and the sensitivity was not a concern.

Q3 Computational cost.

Given a representation dimensionality $d^\prime$ and a time-series length $T$, the computational complexity of curvature computation in RECURVE is $O(d^{\prime}T)$. This complexity is the same as that of TS-CP$^2$, thereby ensuring a comparable computational burden. It is noteworthy that with a low representation dimensionality, i.e., $d^\prime=32$ as the default, and a linear scaling with respect to the length of the time series, RECURVE demonstrates efficiency in the context of lengthy time series. The detailed cost analysis will be added in the final draft.

Thank you so much for acknowledging the effectiveness and intuitiveness of our new boundary detection method. We hope that you can support our work during the discussion period.

W1, Q1 Categorization of the boundaries and evaluation with the categorization.

(1) Categorization: Thank you for your valuable comment. In our current draft, we categorize boundaries into gradual and abrupt changes depending on the speed of change. Gradual changes (e.g., Walk$\rightarrow$Down) involve slow transitions over time, which can be challenging for traditional distance-based metrics to detect. Abrupt changes (e.g., Stand$\rightarrow$Sit), on the other hand, involve rapid transitions that are typically easier to identify due to significant differences between consecutive points.

One way to explicitly categorize the boundaries is to use the inter-class embedding distance, which is the Euclidean distance between the centroids of point representations of given classes. If the distance is below a certain threshold, the transition between the two classes is considered gradual; otherwise, it is considered abrupt.

(2) Evaluation: Using Figures 6 and 7 in Section 4.3, we have already analyzed our experimental results with respect to the categorization. This analysis enables us to demonstrate how our method, RECURVE, performs in different scenarios and highlights its effectiveness in detecting both types of changes. To further improve the structure and clarity of our paper, we will expand Related Work to include more references on gradual change point detection.

Q2 Capability of RECURVE according to the categorization.

Per your suggestion, we show the performance results separately for gradual and abrupt changes in Table 5 (see the PDF file 🔗). It is obvious that RECURVE can support both types of boundaries, also as demonstrated in Section 4.3. Specifically, in the HAPT dataset with $p=5$, the increase in the AUC from TS-CP$^2$ to RECURVE+TPC for gradual changes is 32%, which is significantly higher than the 21% increase for abrupt changes. The findings presented in Table 5 indicate that RECURVE is notably more efficient in handling gradual changes. A tricky case with very short segments is discussed in Appendix D.

We greatly appreciate your valuable feedback and encouragement. Your comments will be definitely incorporated into our final version.

Thanks for your responses, I will keep my positive score.

Thank you so much for acknowledging the innovation of our new boundary detection method. We really hope that our responses are satisfactory to you.

W1.1 Similarity to the distance-based metrics.

The three points used for calculating the curvature are $t-w$, $t$, and $t+w$. Here, $w$ was introduced for stability. Then, the simplified trajectory (i.e., line segment) between $t-w$ and $t$ and that between $t$ and $t+w$ are considered to measure the turning angle at $t$. Thus, we believe that the trajectory perspective is incorporated into the design of our metric.

W1.2 Unclear necessity of the turning angle.

Thank you very much for your insightful comment. Yes, you are right. The curvature seeks to maximize the combined effect of the turning angle $\theta$ and the distance. In fact, Definition 3.2 is similar to the conventional curvature estimation in the geometry field [17]. The turning angle is truly necessary especially for gradual changes. Please see Figures 6 and 7. Between similar actions, e.g., from walking to descending (Walk$\rightarrow$Down), the embedding distances across these two classes are small (Figure 6(a)). Consequently, the distance-based scores become low for these transitions because the embedding distances between consecutive inter-segment points should be also small (Figure 6(b) and the top row of Figure 7). Nevertheless, these drawbacks are fully cured by adding the turning angle to the change metric (Figure 6(c) and the bottom row of Figure 7). Overall, Figures 6 and 7 clearly show how crucial the turning angle is to the change metric.

[17] Curvature and torsion estimators based on parametric curve fitting. Computers & Graphics, 29(5):641–655, 2005.

W1.3 Roles of the turning angle and the distance.

The relative contribution of the turning angle $\theta$ and the distance varies depending on the scenario. For gradual changes such as the transition from walking to descending in Figure 6, the score increases primarily due to a relatively large $\theta$. In contrast, for abrupt changes such as the shift from walking to standing, the distance term has a greater impact on the score. In order to substantiate our assertion, we present the values of $\theta$ alone in Figure 9(d) (refer to the PDF file 🔗). Here, $\theta$ is greater for the transitions where the curvature-based method succeeds but the distance-based method fails---i.e., for the transitions with smaller inter-class embedding distances.

W2 Implication of Assumption 3.6.

This assumption is made in order to concentrate on the effect of the turning angle especially for gradual changes, where the distances between consecutive points are similar within a local range due to the temporal coherence of time series. This intention will be further elucidated in the final draft. That is, Theorem 3.8 is presented to prove that, because of the effect of the turning angle, the curvature is larger for intra-segment points than for inter-segment points.

We acknowledge that the assumption is not always true for datasets from the real world. However, RECURVE continues to be reliable because it takes advantage of the combined effect of the turning angle and the distance. Moreover, we empirically confirm the validity of Theorem 3.8 on the real-world datasets without the assumption, as shown in Figure 7.

W3 Minor: Term "mean total curvature".

Sorry about the perplexity in the term. We will surely correct the term as the mean curvature.

W4 Other Inquiry: Actual calculation of the curvature in Figure 5.

We are really impressed by your thorough review. Definition 3.2 and Line 165 of our draft state that points spaced $w$ apart are used in the curvature calculation rather than just consecutive points. This calculation makes the curvature insensitive to local noise and enables us to capture broader changes in the trajectory. Your understanding is correct except this point. The discrepancy between actual curvature scores and your calculations is mainly due to dimensionality reduction, where a 32-dimensional representation space is converted to a 2-dimensional space. Thus, unfortunately, the precise location of each point representation may not be accurately depicted on Figure 5.

Additionally, we have removed some redundant points from each trajectory visualization, showing only one point out of every ten, to prevent the figure from becoming too crowded and to make the overall trajectory easier to understand. Our visualization may have resulted in some confusion, and we will surely try to eliminate this confusion.