We express our gratitude to Reviewer ALFQ for insightful feedback. In response to the reviewers’ comments, we prepared a revised version of the manuscript, and we address specific questions below.

1. How do DNNs in RL can identify DTPs? Why do you think DNNs in RL can do that?

   1. As stated in Section 1, when using DNN, it is unconstrained by fixed window sizes or frequencies within the time series data domain. Additionally, deep networks learn in a data-dependent manner, enabling them to capture the characteristics of sub-sequences in time series data more effectively. This capability enables them to perform different smoothness on abrupt changes that are distributed differently for each sub-sequence. For example, as shown in Figure 3 in a revised version of the manuscript, DTF-net extracts trends with varying smoothness for each sub-sequence, in contrast to $\ell_1$. Consequently, DTF-net identified and filtered what was considered noise in the third point, while recognizing the other five points as abrupt changes.

2. What is the motivation behind using RL?

   1. As stated in Section 2.3, our motivation stems from the optimal stock trading strategy, where the goal is to buy at the lowest point and sell at the highest point, aligning with the concept of abrupt changes. Consequently, we draw inspiration from RL stock trading algorithms to effectively capture trend points.
   2. We supplemented our motivation in Sections 1 and 2.3 in a revised version as follows.

   Motivated by the nature of abrupt changes, categorized as extreme values, we introduce a novel algorithm designed to directly identify essential points for trend extraction, departing from approximations. This is inspired by the optimal stock trading strategy aimed at maximizing profit. Attaining the optimal strategy involves buying at the lowest point and selling at the highest point to achieve the maximum return, and these points align with the concept of abrupt changes and extreme values. In this context, Reinforcement Learning (RL) can capture abrupt changes, which are optimal trading points in a single stock trading task, by an agent. Derived from these motivations, we reformulate the Reinforcement Learning (RL) stock trading algorithm as a solution to the trend filtering problem.

3. Why do the authors highlight DTF-net in Table 1 rather than the best performance? (Wavelet overfitting)
   1. In the case of Wavelet, it demonstrates superior performance in terms of simple MSE or MAE metrics when applied to the ground truth of a linear signal added with noise. However, we conclude that it tends to overfit to noisy data for two reasons. First, when compared with the experimental results using only a linear signal as the ground truth, as shown in Table 1 in the revised version, there is a significant discrepancy with the result of the linear signal added noise as the ground truth, implying substantial overfitting. Next, as an additional indication of potential overfitting, we introduce another crucial property of trend filtering, namely, smoothness. To evaluate this smoothness, we make the assumption that the added Gaussian noise, accounting for 10%, should be filtered out to consider it an appropriate level of smoothness. Therefore, Wavelet can be considered incapable of performing effective noise filtering.
   2. We added the assumption of smoothness, “filtering out at least 10% of Gaussian noise”, in a revised version and provided updated results in Table 1.
4. There are a lot of real-world time-series data. Why did the authors not evaluate the methods on them?
   1. Analyzing trend filtering methods poses two challenges. First, defining a ground truth for the trend is challenging. For example, in the case of $\ell_1$, the degree of smoothness varies with $\lambda$, resulting in different trends. Second, labeling abrupt changes is challenging. Obtaining real-world data with such labels is difficult, and arbitrary assignment to real-world data is subjective. Therefore, we performed performance evaluations by setting time series forecasting as a downstream task, indirectly allowing the assessment of trend filtering results on real-world data.
   2. We supplemented why trend filtering analysis on real-world data poses challenges in a revised version.

We would again like to thank Reviewer ALFQ, and we hope that our changes adequately address your concerns. Please let us know if you have any further questions or comments, and we are very happy to follow up!

Thank you for raising the score!

We are grateful to hear that some of your concerns have been addressed and we additionally respond to the first question.

1. I do not think the simple DNN is just able to detect different smoothness and abrupt changes. Most of DNNs are with smoothness assumption and have the property of catastrophic forgetting.
   1. We acknowledge that DNNs may have issues related to smoothness and forgetting, and we also mentioned in Section 2.2 that the smoothness of DNN is approximated to a Gaussian output by the MSE loss function. Therefore, in this paper, rather than directly extracting trend filtering from the output of DNN, which is a regression task, we train the policy of an RL agent to capture trend points through actions. Consequently, we believe that the issues related to DNN's smoothness assumption or forgetting may not be problematic.
   2. Additionally, we conceptualized DTF-net to identify trend points by considering individual sub-sequences in the environment. This is achieved through a forecasting task serving as the reward. Moreover, as illustrated in Figure 3, we demonstrated that DTF-net displays distinct smoothness for each sub-sequence in real-world data.

If you have any further questions or comments regarding the first question, we would be happy to follow up!

6. Experiments) Multivariate time series forecasting

   1. First, we introduce the revised version of the Appendix G.

   DTF-net is applied to multivariate time series data features with the target of univariate trend filtering. This means that RL examines multidimensional aspects to perform trend filtering on the target. However, individual trend filtering for each dimension in the multidimensional case has not yet been addressed in this study. Nevertheless, this is achievable through the multi-discrete action prediction of the PPO algorithm.

   2. We added multivariate time series forecasting as future work in a revised version of Appendix G.

We would again like to thank Reviewer Fg2U, and we hope that our changes adequately address your concerns. Please let us know if you have any further questions or comments, and we are very happy to follow up!

4. Section 3.2) The reason for the choice of MSE loss at the reward function.

   1. First, we introduce the revised version of Section 3.2.

   In the stock trading task, RL optimizes based on cumulative return as the reward. However, in general time series, we cannot specify return as a reward. As a result, similar to traditional trend filtering, DTF-net aims to preserve the `closeness to the original data' property, fundamental to trend filtering. However, employing a sum of squared loss function on current values makes it challenging to reflect appropriate smoothness. Therefore, DTF-net distinguishes itself by optimizing for future values instead of current ones. By employing Time Series Forecasting (TSF) and training RL to minimize its cost function, DTF-net can reflect the characteristics of each sub-sequence and learn temporal dependencies. This allows adjusting smoothness by tuning the prediction window.

   2. In this context, the MSE cost function is not solely used but in conjunction with the downstream task of forecasting. Given that time series forecasting involves learning patterns within time series data, DTF-net is capable of predicting trend points while taking into account the unique characteristics of sub-sequences. Additionally, in forecasting tasks, DTF-net predicts trend points while considering temporal dependencies around abrupt changes (Table 1). This is the reason why we chose forecasting MSE cost function as a reward.

5. Section 3.2) Reward sampling’s operation and how it addresses overfitting.

   1. First, we introduce the revised version of Section 3.2.

   In here, we found that by using a penalty reward as a negative value, irregularly applying penalties through sampling can prevent overfitting rather than penalizing at every moment (Appendix C.2).

   2. Since the reward we use is a penalty with a negative value, RL learns in the direction of reducing the penalty. However, if the penalty is consistently applied, there is a tendency for the model to overfit to the data. Therefore, we intermittently apply the penalty to train DTF-net, aiming to prevent overfitting.

We express our gratitude to Reviewer Fg2U for insightful feedback. In response to the reviewers’ comments, we prepared a revised version of the manuscript, and we address specific questions below.

1. Section 3.1) Explanation or analysis for the superiority of random sampling over traditional sequential sampling (also aligned with Q1), and is dynamic-length sampling necessary or is equal-length sampling would suffice? (also aligned with Q1)

   1. First, we introduce the revised version of Appendix C.1.

   We conduct an ablation study based on the state encoding method. First, the construction of episodes is divided into two approaches: a non-sequential method based on random sampling and a sequential method following the conventional time axis order. Next, the composition of episode length is categorized into a dynamic approach with random sampling and a static approach using a fixed window. Note that static length is set to 1500 for Exchange and ETTh1, and 200 for Illness. As shown in Table 5, it is evident that the proposed method, non-sequential episodes with dynamic length, exhibits the most superior and robust performance, overall. Following this, the traditional approach, sequential episodes with static length, shows second robust performance. This confirms that DTF-net achieves excellent performance through the bidirectional learning it aims for, while the sequential episode with static length method occasionally exhibits overfitting results.

   2. We observed that DTF-net, with its non-sequential episode progression and dynamic length, achieves robust performance through bidirectional learning compared to sequential learning with static length. Additionally, it effectively mitigates overfitting issues through the sampling approach.

   3. We supplemented the detailed ablation study and its results in the revised version of Appendix C.1.

2. Section 3.1) Clarity is needed on how they determine different lengths for various sampling methods.

   1. We set the starting point and length of the episode's sub-sequence through random sampling from a discrete uniform distribution, as follows,

      $s \sim \text{unif}(0, N),$ $l \sim \text{unif}(h+p, H).$

   2. The reason for using a discrete uniform distribution is to ensure that all sequences are considered uniformly.

   3. We supplemented additional detailed explanations with formulas for random sampling in a revised version of Section 3.

3. Section 3.1) Positional Encoding is "specifically designed for time series data" appears overstated.

   1. We acknowledge that the positional encoding statement is overstated and revised it (Section 3.1). Also, we added an ablation study on positional encoding compared to simple zero padding methods in a revised version of Appendix C.1 (Table 5).

We additionally respond to the second question raised by the reviewer.

The selection of the MSE loss function as a reward is motivated by the $\ell_1$ trend filtering, a widely employed method in time series data. Formally, the $\ell_1$ trend filtering minimizes the following objective function,

$\sum (y_t - x_t)^2 + \lambda \sum |x_{t-1} - 2x_t + x_{t+1}|$.

The first term represents the proximity to the original data, while the second term represents smoothness. Therefore, to capture the proximity property of trend filtering, we utilize the first term as a reward function.

Second, regarding smoothness, $\ell_1$ employs proximity to past-current value and current-future value.

$|x_{t-1} - 2x_t + x_{t+1}| = |(x_{t-1} - x_t) + (x_{t+1} - x_t)|$

Contrarily, we utilize the proximity of future values based on the prediction, without the additional regulation term, thereby employing forecasting MSE as a reward. We can adjust smoothness by forecasting the window as a hyperparameter as shown in Figure 6.

We acknowledge the tendency of the forecasting method to lower MSE by generating a smooth output. However, as we leverage the trend extracted through the actions of DTF-net as an additional feature, we take advantage of the characteristic illustrated in Figure 2, where DTF-net performs well in capturing abrupt changes, causing a decrease in MSE. This is further supported by the evidence in Figures 4-(b) and 5, where improved performance in forecasting MSE is observed when the trend effectively reflects abrupt changes.

If you have any further questions or comments regarding the second question, we would be happy to follow up!

Thank you.

We express our gratitude to Reviewer NMhu for positive and insightful feedback. In response to the reviewers’ comments, we prepared a revised version of the manuscript, and we address specific questions below.

1. W1) Computational expensive, overfitting problem, and tuning hyper-parameters.
   1. We acknowledge that the limitations of DTF-net include data and computational expenses, the risk of overfitting, and the need for hyperparameter tuning. However, we have made efforts to address or have addressed these issues in the following ways.
   2. Data-expensive: While deep networks typically require a substantial amount of data, DTF-net demonstrates robust performance even with small synthetic datasets or Illness data, utilizing a simple MLP policy.
   3. Computational-expensive: As discussed in the Appendix, we acknowledge the high computational cost of RL. However, we structured episodes through sampling to optimize DTF-net with a minimal number of steps.
   4. Overfitting: We address the overfitting issue through reward sampling. Given that the reward we employ is a penalty with a negative value, RL learns to minimize this penalty. However, continuous application of the penalty may lead to overfitting. To mitigate this, we intermittently apply the penalty during the training of DTF-net, aiming to prevent overfitting.
   5. Hyper-parameter tuning: To minimize the number of hyperparameters and ensure the robust performance of DTF-net, we have introduced a sampling methodology.
   6. We supplemented these discussions in a revised version of Appendix D.2.
2. W2) Heuristic random sampling algorithm, and W3) Overall framework might be built without RL.
   1. Thank you for your suggestions. Specifically, we found methods that solve MDP without RL, including, Dynamic Programming, Linear Programming, Policy Gradient Methods, Monte Carlo Methods, Q-Learning (Model-Free), and Temporal Difference. We plan to add these methods in Appendix G, Future Work section. If the methods that the reviewer wants to specify are not added, we want to hear from you through discussion.
3. Minors
   1. M1) Figure 1 should be referred to in the main text.
   2. M2) Figure 4 is not straightforward.
   3. M3) Legend of Figure 5 is too small, unreadable.
      1. Thank you for bringing these to our attention. We addressed these minor points in the revised version.

We would again like to thank Reviewer NMhu, and we hope that our changes adequately address your concerns. Please let us know if you have any further questions or comments, and we are very happy to follow up!

We express our gratitude to Reviewer Q1zH for positive and insightful feedback. In response to the reviewers’ comments, we prepared a revised version of the manuscript, and we address specific questions below.

1. Formalize the reinforcement learning process with diagrams to enrich the details.

   1. Thank you for your suggestion and we added a diagram in a revised version of Figure 1.

2. Compared trend filtering methods take abrupt change detection into account.

   1. We compared DTF-net with both methods that consider and do not consider abrupt changes. First, the $\ell_1$ and H-P methods consider abrupt changes using the sum of squared functions. Second, Wavelet is grounded in frequency, and EMD and Median rely on decomposition, which do not prioritize the consideration of abrupt changes. Finally, we also compared with CPD and anomaly detection algorithms, which consider abrupt changes.
   2. We supplemented additional detailed explanations and results in the revised version of Section 2.1 and Table 1 based on the comments.

3. Anomaly detection can be used instead of.

   1. Thank you for your suggestion and we added experiments on anomaly detection methods [1] in the revised version of Table 1.
   2. Additionally, we plan to add details and other various anomaly detection algorithms [1-3] in Appendix F, Discussion section. If the methods that the reviewer wants to specify are not added, we want to hear from you through discussion.

   [1] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.

   [2] Yiyuan Yang, Chaoli Zhang, Tian Zhou, Qingsong Wen, and Liang Sun. 2023. DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '23). Association for Computing Machinery, New York, NY, USA, 3033–3045. https://doi.org/10.1145/3580305.3599295

   [3] Jiehui Xu, Haixu Wu, Jianmin Wang, & Mingsheng Long (2022). Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. In International Conference on Learning Representations.

We would again like to thank Reviewer Q1zH, and we hope that our changes adequately address your concerns. Please let us know if you have any further questions or comments, and we are very happy to follow up!

We extend our gratitude to Reviewer Q1zH and present additional responses regarding anomaly detection experiments.

1. We introduced experiments on recent anomaly detection algorithms mentioned above [1-3] in Appendix F.3.
2. We identified two challenges for trend filtering in anomaly detection:
   1. The sensitivity of the threshold to detect anomalies, particularly in capturing extreme value points.
   2. Despite finding an optimal threshold for trend filtering, anomaly detection often misses peak points in complex sub-sequences.

We would again like to thank Reviewer Q1zH, and we hope that our changes adequately address your concerns. Please let us know if you have any further questions or comments, and we are very happy to follow up!