Thank you for taking the time to review our manuscript. We appreciate your feedback and address each of your comments below.

(W1) RioT relies on costly human feedback: We refer to our global response where we discuss this point, clarifying the amount of feedback required, handling of unknown confounders and confounders in the frequency domain.

(W2) Using Causal learning methods to remove confounders: Knowledge of where the confounder occurs is not necessarily sufficient to apply causal learning methods. Generally, they have additional assumptions about, for example, the data generation process [1]. Additionally, RioT can mitigate confounders occurring within one variate, while typical causal time series deconfounding approaches only consider variates as a whole as confounded or relevant [2]. While knowledge about confounders can, in general, also be used for causal learning methods, that is not a trivial solution to the problem.

Removing confounded areas: In cases where the confounder always occurs at the beginning of the time series, simply removing the time steps from all training and subsequent test examples could also be a solution. However, this strategy fails if the confounder’s position varies. In such scenarios, simply removing the confounder would also destroy the alignment of time steps between samples and make learning quite difficult. Applying RioT to a model enables it to ignore confounders without these alignment issues. Similar difficulties arise when attempting to remove confounders from multivariate data. In the case of a frequency confounder, this approach is generally not possible anymore, as there is no specific time step that corresponds to the confounder and could be removed.

(W3) Frequency explanations: While the frequency explanation is "only" a transformed version of the spatial explanation, this change in representation is quite important. Regularizing models in the spatial domain is limited to spatially separable confounders. When a confounder is, for example, a sine wave frequency that overlaps with the data, spatial feedback is not helpful anymore, as the confounder occurs at all points in time. It has been shown that models are prone to frequency shortcuts/confounders even when only confronted with spatial input [3], and using the presented frequency explanation is a suitable tool to mitigate this. Further, transforming the input to the frequency domain and then applying IG is essentially equivalent, as the FFT is a linear transformation that a single matrix multiplication can represent. Nevertheless, we agree that directly learning frequency explanations is a very interesting direction for future research, and such explanations could then directly be incorporated in RioT as well.

(W4) Presentation: Thanks for pointing these typos out. We checked the paper and improved the writing. We kindly ask whether the reviewer has any other specific points where the presentation of the paper should be improved, particularly in the light that the other reviewers found the paper to be clearly structured and well-written.

(Q1) See answer to W2

(Q2) Reapplying Annotations to other samples: In many cases, the same annotation can be reapplied to other samples as well. When examining P2S, the domain expert identified the run speed as a potential confounder for the outcome of the experiments and then annotated a few (5 to 10) samples with feedback masks. As the confounder always occurs at a specific phase of the run, the feedback masks could then be used for all samples, requiring no further annotations.

(Q3) Alternative Approaches for P2S So far, other methods to mitigate confounders (for example, through causal learning) on time series data rely on a setting where a full variate of the time series can be considered confounded and the other variates contain the relevant information. As the confounder and the relevant information for the classification are part of the same variate in P2S, they cannot be applied easily. Other confounder mitigation work has so far been concentrated on the visual and textual domain without considering time series. The scarcity of alternative methods to tackle this problem is one of the important motivations for introducing RioT.

We thank the reviewer for the response and acknowledge the remaining concerns.

1. Removal of confounding data: We agree that in certain scenarios, removing confounded data can increase the robustness of time series models. As the reviewer rightly points out, this is a common preprocessing step, extending beyond time series data. However, removing (or imputing) confounded data has still to be done carefully, as this process can inadvertently introduce new confounders, which a model may similarly exploit.

To illustrate this, we conducted two additional experiments where we compare a model trained without any confounder mitigation, one using imputation of annotated data, and RioT leveraging the same annotations. The results demonstrate that simply imputing the confounder (which even only occurs in one class) does not improve model performance. Instead, the model tends to use the imputed data as a shortcut to solve the task. Similarly, in the frequency setting, we remove the annotated frequencies from the data. However, this also introduces new shortcuts in the data, as the removal of said frequencies impacts the data from both classes differently. In contrast, RioT effectively addresses the confounder in both scenarios by training the model to disregard the confounder in a more generalized manner.

In summary, while we agree with the reviewer that removing confounded data is a crucial preprocessing step, RioT does not aim to replace this process. Rather, RioT complements it by teaching the model to generalize and robustly ignore confounders, particularly in more complex cases.

2. Other frequency representations: We agree that the FFT is not a one-size-fits-all solution for analyzing the frequency domain. Since the Mel-Spectrogram is only approximately invertible, using this transformation can introduce additional challenges. While it is possible to generate Mel-Spectrogram-based explanations in a manner similar to FFT-based explanations, applying them for model optimization is less straightforward. So far, RioT can easily incorporate transformations that are invertible, like STFT or Wavelets alongside FFT, but extending this approach to other types of transformation or exploring alternate methods to generate frequency explanations are important steps for future work.

3. Causal learning methods: Generally, knowledge about confounders does not imply (full) knowledge about the data generation process, as confounders represent only a subset of the overall data generation mechanism. However, when experts can provide the necessary information, causal learning, and deconfounding methods can be applied, provided the confounder is represented as a separate variate.

Alternatively, it is also possible to employ causal discovery to obtain this information (e.g., with https://jakobrunge.github.io/tigramite/). These methods, however, rely on their own assumptions, such as Causal Sufficiency or Faithfulness [1], which have to be considered when applying them in real-world scenarios.

Overall, we agree that when the required assumptions are satisfied, applying causal learning methods to mitigate confounders is a good approach. Importantly, we emphasize that RioT can be used without relying on these specific assumptions.

Hopefully, this could clarify the remaining concerns. If not, we are happy to answer any follow-up questions.

[1] Jakob Runge et al., Detecting and quantifying causal associations in large nonlinear time series datasets.Sci. Adv.5,eaau4996(2019).DOI:10.1126/sciadv.aau4996

Thank you for your thoughtful review and constructive feedback on our paper. We address each of your comments and clarify points in the paper accordingly.

(W1) Validation Set Usage: We apologize for the lack of clarity regarding our data split procedure. For forecasting we utilized a train/validation/test split of 56%/14%/30%, where the "train" portion was further divided into training and validation sets using an 80/20 split. We relied on the official test splits provided for datasets from the UCR repository (i.e., for classification tasks). In addition, we also provide train/test splits for P2S, where we also use 20% of the original training set as validation.

Our optimization process is, of course, based on the validation set and not the test set. We have clarified this point in the manuscript and provided additional details about this setup.

(W2.1) Requirement of human feedback: We provide a detailed discussion of this aspect in the general response above.

(W2.2) Adversarial Attacks: We agree that adversarial attacks present an important challenge in human-in-the-loop systems. However, the scope of this work primarily focuses on establishing the feasibility of human feedback to mitigate confounders rather than performing a comprehensive adversarial analysis. Addressing adversarial attacks in these interactive methods, particularly in time series contexts, is indeed a critical area for future research. We clarified this now in more detail in our limitations section.

(W3.1) Extension to Multivariate Time Series: Extending RioT to multivariate time series is both feasible and straightforward within our proposed framework. For brevity and to maintain controlled experimental conditions, this paper focuses on univariate series. To apply RioT to multivariate time series, an explanation method such as integrated gradients can be employed to provide insights across multiple variables. The feedback mechanism extends naturally to multivariate data by expanding the feedback mask dimensions to match the number of variables. Alternatively, it is also possible to mask a variate entirely if it is a potential confounder. This straightforward adjustment allows RioT to adapt seamlessly to multivariate settings without substantially modifying the core methodology.

(W3.2) Dataset Constraints and Generalizability: While we selected specific datasets to ensure controlled experimental conditions, especially for adding synthetic confounders, this does not imply RioT is limited to these scenarios. As long as model explanations and feedback masks can be given for a time series, RioT can also be applied to that time series. This can also include settings with missing values or varying lengths, as IG can be applied in both scenarios. However, we agree that training in settings with very small amounts of data containing confounders can generally be challenging due to the tendency of deep learning models to overfit when training data is scarce [1]. However, this is a general problem in these circumstances and is not limited to RioT.

(W4) Integrated Gradients for Forecasting: Per se, there is no technical difference between applying integrated gradients for classification or regression tasks, instead of using the class output neuron as a target, the regression output neuron is used (cf. [2]). What we add as a further adaption to enable IG for forecasting is the aggregation over the different neurons in the forecasting window. We believe that such an aggregation is reasonable, as we are more interested in the overall global features of the time series rather than the exact timestep details. Thus, despite its simplicity, aggregation serves as a reasonable proxy. However, we acknowledge that explanation and attribution methods for time series forecasting are still open research topics and encourage continued exploration in this field.

(W5) Clarification on Methodology and Notation: Thank you for your remark, we apologize for this oversight. $\bar{{x}}$ is the baseline for integrated gradients. The method integrates the gradient along the path from baseline to the input value. Thereby, $\alpha$ is the integration variable, and $\tilde{{x}}$ is the temporary point on the path used for integration. We now introduce all missing variables in Equation (1).

(W6) Reproducibility and Model Tuning: We tuned our hyperparameters using Optuna, and all training and setup scripts are included in the provided code repository to support reproducibility. Details are given in Appendix A1

Thank you very much for the rebuttal, I have carefully read the authors responses and still have some concerns.

---

(W1) Why not follow the splitting procedures established in time series analysis [1][2][3][4][5][6][7]?

(W2.2) Unfortunately, a discussion on adversarial attacks is not included in the manuscript.

(W3.1) I am sure that the methodology can be extended to a multivariate setting, however, I am uncertain whether it is feasible to provide human feedback on time series with a large number of variates?

(Q6) It is unfortunate that the manuscript does not include self-supervised baselines pre-trained on large time series corpora, such as [7][9][10][11][12][13][14], as I believe a thorough investigation whether such pre-trained models are more robust against confounders would improve the quality of the study.

(Q9) The authors provide t-SNE plots, which are known to be sensitive to parameter tuning. Therefore, a latent space analysis using principal component analysis would be the fairer approach.

---

[1] Zhou, H. et al. "Informer: Beyond efficient transformer for long sequence time-series forecasting." AAAI Conference on Artificial Intelligence (AAAI). 2021.

[2] Wu, H. et al. "TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis." International Conference on Learning Representations (ICLR). 2022.

[3] Zhang, X. et al. “Self-supervised contrastive pre-training for time series via time-frequency consistency.” Advances in Neural Information Processing Systems (NeurIPS). 2022.

[4] Liu, Y. et al. "iTransformer: Inverted Transformers Are Effective for Time Series Forecasting." International Conference on Learning Representations (ICLR). 2023.

[5] Dong, J. et al. “SimMTM: A simple pre-training framework for masked time-series modeling.” Advances in Neural Information Processing Systems (NeurIPS). 2023.

[6] Nie, Y. et al. “A time series is worth 64 words: Long-term forecasting with transformers.” International Conference on Learning Representations (ICLR). 2023.

[7] Gao, S. et al. "UniTS: A unified multi-task time series model." Advances in Neural Information Processing Systems (NeurIPS). 2024.

[8] Max Planck Institute for Biogeochemistry. “Weather station.” 2024. URL: https://www.bgc-jena.mpg.de/wetter/.

[9] Jin, M. et al. "Time-LLM: Time Series Forecasting by Reprogramming Large Language Models." International Conference on Learning Representations (ICLR). 2023.

[10] Zhou, T. et al. "One fits all: Power general time series analysis by pretrained lm." Advances in Neural Information Processing Systems (NeurIPS). 2024.

[11] Goswami, M. et al. "MOMENT: A Family of Open Time-series Foundation Models." International Conference on Machine Learning (ICML). 2024.

[12] Woo, G. et al. "Unified Training of Universal Time Series Forecasting Transformers." International Conference on Machine Learning (ICML). 2024.

[13] Yang, C. et al. “Biot: Biosignal transformer for cross-data learning in the wild.” Advances in Neural Information Processing Systems (NeurIPS). 2024.

[14] Jiang, W. et al. “Large brain model for learning generic representations with tremendous EEG data in BCI.” International Conference on Learning Representations (ICLR). 2024.

Thank you for your review and constructive feedback on our work. We are encouraged by your positive assessment of the practical relevance of RioT, the dual feedback approach, and its empirical robustness. We appreciate the opportunity to address your concerns and clarify some key points.

(W1) Dependency on Human Feedback: We provide a detailed discussion of this aspect in the general response above.

(W2) Training Cost: While RioT's gradient-based feedback does introduce computational overhead, the actual computational effort remains manageable. In particular, even a naive implementation of RioT in PyTorch scales to large, pretrained models like OFA.

(W3) Limited Exploration of Feedback Noise: We agree that evaluations with noisy and potentially incorrect feedback are important for all interactive learning methods. To start addressing this concern, we performed robustness tests regarding noisy feedback, where RioT achieved promising results. Nevertheless, we agree that more detailed evaluations on the robustness of RioT and XIL methods in general to noisy feedback are essential steps in future research in this field.

(Q1) The incorporation of Feedback: Figure 1 illustrates the application of RioT to a confounded time series model. First, the model undergoes standard training, followed by an inspection of its explanations. Relevant annotations are collected from these explanations, and the model is subsequently retrained, this time optimized using both the target labels and the annotated explanation masks. While we do not explicitly consider pretrained models in this work, previous work has shown that explanation-based regularization can also be applied to already confounded models to correct them without the need for full retraining [1]

(Q2) Negative Impact of Frequency Domain Feedback: In Fig. 6, we investigated how noisy feedback influences the model's performance, this included noisy feedback on the frequency domain and highlights that RioT is quite robust to noisy frequency feedback. In our experiments, we did not encounter situations where the frequency feedback did contradict the spatial feedback or vice versa. While such cases are, in principle, imaginable, they are not that likely to occur, as feedback in the frequency domain has a different impact on the model than feedback in the spatial domain due to the different representations.

(Q3) Balancing Feedback Quality and Computational Efficiency: In cases where training costs are of high interest, the "switch XIL on" [1] strategy could be employed. At first, training happens without RioT, and then at a later part of the training process, RioT is used.

[1] Felix Friedrich, Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. A typology for exploring the mitigation of shortcut behaviour. Nature Machine Intelligence, 5(3):319–330, 2023a.

Thank you for your review and valuable feedback on our paper. We are pleased to see that you found the application of Right for the Right Reason (RRR) methods to time series data, as well as the introduction of dual loss terms for time and frequency domains, to be novel contributions. We would like to address your comments and clarify a few points regarding novelty, clarity, and the questions raised.

(W1) Novelty: While we understand your observation that RioT leverages RRR methods applied to time series data, we respectfully highlight that our work introduces significant innovations in both methodology and application context. RioT extends RRR beyond vision-based applications by tailoring each of its components to time series data. In particular, adapting feedback from the frequency domain is essential in this context. Further, taking the step to include feedback from transformed representations overcomes one of the major limitations of RRR and previous XIL methods in general: Employing this technique enables feedback on confounding factors that are not separable in the original input domain. Previously, giving such feedback was not possible, but RioT enables this.

(Q1) Equation 1 $\bar{x}$: The variable $\bar{x}$ is indeed the baseline for IG. We are sorry that this was not clear, and we added a clarification to the paper. In the experiments, we chose a zero baseline, following previous work [1, 2].

(Q2) Calculation of $e'_i(x)$: The explanation $e'_i(x)$ is the model explanation for time step $i$ of the forecasting window. We follow the typical explanation computation for classification to obtain them but replace the target class with the forecasting output at time step $i$. Concretely, this corresponds to using IG by modifying Eq. 1 and replacing the target for the gradient computation with the forecasting output.

We made this more clear in the paper

(Q3) Requirement of the 2nd order derivative: As RioT includes an optimization "through" the IG explanation, we need to compute the second-order derivative. In fact, we need the computation of the second-order mixed-partial derivative, where we first compute the explanation (first-order gradient w.r.t. the input) and then backpropagate through this w.r.t. the model parameters. In A.3 in the appendix, we discuss this derivative and its computational costs in more detail.

(Q4) Improving Figure 4 Visualization: We apologize for any difficulty interpreting the color attributions in Figure 4. We enhanced the figure's contrast to improve visual clarity and updated it in the paper.

(Q5) Confounder Masks in Both Domains: In the "Removing Multiple Confounders at Once" experiment, we indeed provide separate masks for spatial and frequency domains to capture different confounding factors across each domain. (Although time and frequency domains are related, unique confounding features may only appear prominently in one domain. Giving separate feedback in the respective domain helps the model learn to ignore spurious patterns in either domain.)

[1] Dominique Mercier, Jwalin Bhatt, Andreas Dengel, and Sheraz Ahmed. Time to Focus: A Comprehensive Benchmark Using Time Series Attribution Methods. ArXiv:2202.03759, 2022.

[2] Manjunatha Veerappa, Mathias Anneken, Nadia Burkart, and Marco F. Huber. Validation of XAI explanations for multivariate time series classification in the maritime domain. Journal of Computational Science, 58:101539, 2022.

Since multiple reviewers raised concerns regarding the reliance on human feedback, we would like to address this point comprehensively. Our work introduces a novel framework that enables the revision of time series models based on their explanations. Notably, the approach to obtaining feedback is flexible and can be tailored to the specific domain and resources available.

Effective with Minimal Feedback Our experiments demonstrate that RioT performs effectively even when feedback is provided for only a small fraction of the data. As shown in Fig. 5, annotations on as little as 5% of the samples significantly mitigate confounders, yielding substantial improvement compared to scenarios with no feedback. This efficiency ensures that the feedback requirement is manageable.

Leveraging Generalizable Feedback to Reduce Effort In many cases, annotations can be generalized across samples, further reducing the interaction cost. For example, in the P2S experiment, the domain expert provided feedback on only 5–10 samples, and this information was successfully extrapolated to all data points. Confounders, such as those with physical origins identified in P2S, often exhibit systematic patterns across data [1]. Once such patterns are identified in a subset of samples, they can be applied broadly without needing individual review. This systematic nature of confounders is a practical advantage that can be leveraged to minimize the overall annotation effort. Additionally, alternative strategies such as semi-automated feedback, transfer learning, or the use of large language models (LLMs) could further reduce the overhead associated with human annotations.

Domain-Specific Advantages of Expert Feedback It is also important to consider the domain-specific costs associated with feedback. Reviewer concerns seem to assume that obtaining expert annotations is inherently expensive. However, in many real-world domains, expert feedback may actually be more accessible and cost-effective than other strategies to reduce confounding via data collection. For instance, privacy constraints and limited patient availability in the medical field make data collection challenging. Similarly, in mechanical engineering, experimental data acquisition can involve significant costs and complex setups. In such scenarios, leveraging domain expertise can be a practical and valuable alternative to extensive data collection efforts to mitigate confounding.

Strategies for Managing Unknown Confounders We acknowledge that providing feedback on unknown confounders can be challenging. Nonetheless, existing research on confounder detection in machine learning [2, 3] could be adapted for time series data, facilitating the identification of such confounders. Even when a confounder is only partially identified, our experiments (cf. Fig. 4) show that RioT still achieves performance improvements. We agree that mitigating unknown confounders remains a broader challenge requiring further investigation and development in future work.

Advantages of Feedback in the Frequency Domain While feedback in the frequency domain may initially appear non-intuitive, it can offer distinct advantages depending on the data type. For example, in audio data, the frequency representation often provides clearer insights, making it easier to annotate confounders. As demonstrated in [4], this perspective underscores that the frequency domain can sometimes be more intuitive and effective than the spatial domain, depending on the application context.

In conclusion, we believe these points demonstrate that the reliance on human feedback is not a limitation of our framework but rather a flexible feature that can be adapted to different domains. Our method is designed to work effectively with minimal feedback, and the generalizability of annotations, along with domain-specific considerations, makes RioT a practical and scalable solution.

[1] Lapuschkin, Sebastian, et al. "Unmasking Clever Hans predictors and assessing what machines really learn." Nature communications 10.1 (2019): 1096.

[2] Yu Yang, Eric Gan, Gintare Karolina Dziugaite, and Baharan Mirzasoleiman. 2024. Identifying spurious biases early in training through the lens of simplicity bias. In International Conference on Artificial Intelligence and Statistics. PMLR, 2953–2961.

[3] Shunxin Wang, Raymond Veldhuis, Christoph Brune, and Nicola Strisciuglio. 2023. What do neural networks learn in image classification? a frequency shortcut perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1433–1442

[4] Johanna Vielhaben, Sebastian Lapuschkin, Grégoire Montavon, and Wojciech Samek. Explainable AI for Time Series via Virtual Inspection Layers. ArXiv:2303.06365, 2023

Early Stopping on forecasting datasets

Early Stopping on classification datasets