Capturing the Temporal Dependence of Training Data Influence

Q [What would be the data influence scores for outliers in the training data?] “How would the proposed data influence handle outliers in training data? ... It will be helpful if the authors could a). Conduct an experiment specifically examining the behavior of their method on datasets with injected outliers; b). Discuss potential mitigation strategies if outliers are indeed found to have outsized influence; c). Compare how their method handles outliers versus existing influence estimation techniques”

A We appreciate the reviewer's important question about handling outliers. We would like to clarify that our method naturally accounts for outliers through validation-based influence scoring. While outliers may indeed have large gradient magnitudes during training, their influence scores are determined by their impact on validation loss. Detrimental outliers typically result in negative influence scores, as they harm model performance on the validation set. This allows our method to naturally distinguish between beneficial high-influence points and harmful outliers.

In Appendix E.2.1, we directly evaluated Data Value Embedding’s ability to detect outliers through extensive experiments on mislabeled data detection. In these experiments, we compared our approach against multiple baselines. Our method achieved strong performance in detecting mislabeled examples while demonstrating more stable results compared to retraining-based methods, as evidenced by the lower standard deviation in performance. Unlike retraining-based methods that can be unstable due to the stochastic nature of model training, our approach provides more consistent results. Moreover, by considering the entire training trajectory rather than just the final model state, our method offers more reliable outlier detection than influence functions.

We have revised the main paper to make the mislabeled data detection experiment more noticeable to the readers. Thanks for the great question!

Q [Clarification on Figure 1(b)] “Using numerical examples, the authors show that early and late regions have high influence and can achieve similar performance improvements in comparison to using data from the entire process. This is quite interesting but also raises a lot of questions. Intuitively, the high influence of the early region is because one starts with nothing and the information on the early region is consequently high. After that, the added information slows down but builds up for the improvements for late region. Thus, it appears to this reviewer that one cannot simply use the early and late regions for the learning process without the middle region. Thus, more clarification and discussions on this is necessary.”

A We thank the reviewer for the question regarding our Figure 1. We would like to clarify an important point about Figure 1(b): we are not training the model using only data from early and late regions while skipping those from the middle region. What differs between the columns shown in Figure 1(b) is the training phases where we apply a computationally expensive online data selection algorithm.

​​Specifically, we adapt the online data selection algorithm from Fan et al. (2024). For each training iteration where selection is applied, we: (1) sample a candidate pool of training points of size $2B$ (where $B$ is the desired batch size), (2) compute gradient cosine similarity between each candidate point and a randomly sampled validation batch, and (3) select the $B$ points with highest similarity scores to form the next training batch. This process is computationally expensive, requiring additional computation at each selection step. Our results demonstrate that we can achieve comparable performance improvements by strategically applying this selection process only during the early (first 2000 iterations) and late (after iteration 20000) phases of training, while using simple random batch sampling during the middle phase.

Your intuition about the importance of different training phases aligns well with our findings! The early phase is indeed critical for establishing good initialization, while the late phase is important for final model refinement. Our results suggest that during the middle phase, when the model is in a relatively stable learning regime, the selection of high-quality training batches may not be necessary - random sampling suffices.

This insight has significant practical implications: we can allocate computationally expensive data selection efforts on the phases where they matter most, substantially reducing computational overhead without compromising model performance. We have revised the manuscript and expanded the details and discussion in Section 5.3 and Appendix E.3.2 to make our message more clear to the readers.

Fan, Simin, Matteo Pagliardini, and Martin Jaggi. "DOGE: Domain Reweighting with Generalization Estimation." ICML 2024.

Thanks for the positive assessment!

Q [Terminology of LOO & Comparison between LOO influence and LOO CV] “... it is important to make it clear regarding the type of CV considered in the paper ... Explicitly compare and contrast their LOO error definition with traditional CV error”

A Thank you for this insightful comment about terminology. We agree that "LOO influence" more precisely describes our metric than "LOO error," and we have revised the terminology throughout the paper. We sometimes use “LOO score” in the paper.

Traditional LOOCV and our LOO influence serve fundamentally different purposes. LOOCV estimates model generalization by measuring prediction performance on held-out data, serving as a model selection and evaluation technique. Lower CV errors indicate better generalization. In contrast, LOO influence quantifies how individual training points impact the model's behavior on validation data, measuring the contribution of specific training examples. Higher absolute LOO influence values indicate more impactful data points.

We have added an explicit discussion in Appendix A.1 to prevent confusion. The key distinctions we will highlight include: (1) their different objectives (model evaluation vs. data importance), (2) their interpretation, and (3) their computational approaches (N separate models vs. counterfactual analysis).

Thank you for helping us improve the clarity of our paper!

Q [Loss vs other performance metrics] “However, large change in the loss does ... (1) Discuss the implications of using loss change rather than model performance metrics (2) Provide examples or experiments showing how their LOO error relates to changes in model performance”

A Thank you for this thoughtful comment about the relationship between loss changes and model performance! We agree that this deserves more discussion and will address both points.

Why use validation loss instead of other metrics?

• (1) Loss is one of the most widely used performance metrics in deep learning and serves as the direct optimization objective during ML training, making it a natural choice in the computation of data influence quantification. This is a common choice in most of the papers in the field. In this work, we also focus on computing the LOO influence for validation loss as it is a widely accepted proxy for language model performance.

• (2) Loss values often provide richer information compared to more human-interpretable metrics like accuracy. Consider, for example, measuring influence through classification correctness on validation data, where outcomes are binary (0 for incorrect, 1 for correct). The resulting LOO influence scores would be limited to {-1, 0, 1} for every training data point, providing far less information than the continuous values obtained from loss calculations.

• (3) The framework of Data Value Embedding supports performance metrics beyond validation loss. The derivation in Appendix C.1 shows we can approximate the parameter difference $\theta_T - \theta_T' \approx \frac{\partial \theta_T(\varepsilon)}{\partial \varepsilon}|_{\varepsilon=0}$, allowing us to estimate $\theta_T' = \theta_T + \text{DVEmb}(z^*)$. This means that for any differentiable performance metric $f(z,\theta)$, we can approximate its LOO influence through $\nabla f(z,\theta_T)^\top\text{DVEmb}(z^*)$. While we can also evaluate non-differentiable metrics like classification accuracy through $f(z,\theta_T) - f(z,\theta_T+\text{DVEmb}(z^*))$, this approach is less efficient when computing LOO influence for all $z^*$, as it requires forward prediction on large models $N$ times where $N$ is the training data size, much slower than simply taking the dot product against each $\text{DVEmb}(z^*)$. Therefore, in this work, we focus on approximating the LOO influence for the validation loss.

Additional experiment: We have conducted an additional fidelity check experiment under the same setting as Section 5.1, where we assess the correlation between LOO influence scores computed from validation loss and the ground-truth LOO score for classification accuracy. In this figure, we observe strong positive correlations (Spearman correlation of 0.763 and 0.735) between our computed influence scores and ground-truth LOO accuracy changes in both (a) single-epoch removal and (b) all-epochs removal settings. Looking at the scatter plots, we can observe a key advantage of using loss over accuracy: while multiple data points often share the same LOO accuracy scores due to the discrete nature of accuracy metrics, our loss-based influence scores provide more fine-grained distinctions between these points. Despite this discretization effect in the ground-truth accuracy measurements, the strong correlation coefficients and clear monotonic trend in both settings demonstrate that our loss-based influence scores effectively capture changes in model performance measured by accuracy.

Dear Reviewer tZN9,

Thank you again for your positive assessment and valuable feedback on our work. Regarding your questions about Data Value Embedding's outlier detection capabilities, we have conducted additional analyses to provide a more comprehensive response. In addition to our quantitative experiments on mislabeled data detection and data selection (Appendix E.2.1), we have also examined real-world examples from the Pile dataset that receive the most negative influence scores from Data Value Embedding (Appendix E.5). These examples demonstrate our method's ability to detect subtle forms of problematic data. For instance, we identified code snippets containing mainly variable declarations or configuration settings that, while syntactically valid, provide minimal learning value and potentially introduce noise. We also found examples of text with poor formatting (missing punctuation and paragraph breaks) and mathematical problems with repetitive structures (frequently beginning with "What is...") that could bias the model's generation patterns.

These findings demonstrate that our method can identify not just mislabeled data, but also more diverse cases that could subtly degrade model performance. This complements our quantitative evaluation in Appendix E.2.1 and provides concrete evidence of how our method handles outliers in real-world, large-scale training settings. We believe these examples provide valuable insights into the types of training data that might impede model learning.

We would like to thank the reviewer for the positive assessment!

Q [Robustness and scalability when applied to industrial-scale dataset] “While the paper’s experiments demonstrate high fidelity on small datasets like MNIST and reduced subsets of larger datasets (e.g., 1% of the Pile), ... when applied to diverse, large-scale data used in production.”

A We appreciate the reviewer's comment on the scalability and robustness of Data Value Embedding on large-scale datasets.

Scalability: Our method is designed to scale efficiently to large models with minimal computational overhead. As detailed in Section 4.3 and Appendix C.9, the additional computation required is significantly lower than regular training costs ($O(BT \sqrt{p \tilde{p}}/L)$ for random projection and $O(BT \tilde{p}^2/L)$ for data value embedding computation, compared to $O(BTp)$ for standard training). Moreover, our method offers substantial efficiency improvements over existing approaches: the most efficient implementation for the influence function still requires extra compute equivalent to an additional full training run to recompute gradients on the final model. While our experimental scale was constrained by academic computing resources, our theoretical complexity analysis demonstrates that Data Value Embedding can efficiently scale to any model size that fits within available computational resources.

Robustness: We acknowledge the challenge of extending the groundtruth comparison experiments in Section 5.1 to other datasets and models, especially for larger-scale settings. Computing ground-truth LOO scores for large models is computationally infeasible, as it requires multiple complete training runs. This is a fundamental challenge faced by all research in this field. While direct ground-truth comparison is infeasible for larger models, we provide multiple indirect validations of our method's effectiveness at scale. In our mislabeled data detection experiments and data selection experiments (Appendix E.2.1) using ResNet18 on CIFAR10, our method achieves competitive or superior performance compared to existing baselines. Furthermore, our qualitative analysis (Section 5.4) shows that Data Value Embedding successfully identifies semantically relevant training examples for GPT2 on Wikitext-103, while baseline methods like influence functions sometimes select irrelevant data. While these experiments cannot definitively prove that our method perfectly approximates TSLOO in large-scale settings, they demonstrate that our approach can reliably distinguish data quality and influence for large-scale settings, which is the primary goal of data attribution.

Q [Storage requirements] “Potential Overhead in Implementation: ... its practical utility without further optimizations.”

A We sincerely appreciate the reviewer's thoughtful comments on the storage requirements. While our method does require storage for gradient information and embeddings, we would like to emphasize two key points regarding its practical efficiency:

(1) Storage requirements are balanced by computational benefits: Our method trades increased storage requirements for substantial computational savings. In modern computing environments, disk storage is generally more cost-effective compared to high-performance GPU computing resources. While storage costs are non-trivial, they represent a one-time investment that enables real-time data attribution capabilities - allowing immediate computation of influence scores for new test points without requiring model retraining or access to the original training data. This is particularly valuable in production environments where computational efficiency and quick response times are crucial.

(2) Dataset-level attribution aligns with practical needs while reducing storage: In many practical applications, stakeholders are primarily interested in dataset-level rather than individual-level attribution (e.g., valuing contributions from different data providers). For these cases, as discussed in Appendix C.10, a simple extension to the derivation in Appendix C.1 shows that we can aggregate embeddings by data source, computing a single embedding that captures the collective influence of all data points from the same source. This approach not only aligns with real-world use cases but also significantly reduces storage requirements, as we only need to store one embedding per data source rather than per data point.

Dear Reviewer Sfva,

Thank you again for your positive assessment and valuable feedback on our work!

To follow up on your question about Data Value Embedding's use cases: while the method cannot be directly applied to closed-source models (just as other existing data attribution techniques), we believe its value for understanding training dynamics, data quality, and data influence remains significant. Our comprehensive experiments demonstrate this value through multiple analyses: studying training dynamics (Section 5.3), identifying influential training examples (Section 5.4), and detecting mislabeled data for targeted data selection (Appendix E.2.1). To further validate our method's practical usefulness, we have examined real-world examples from the Pile dataset that received the most negative influence scores from Data Value Embedding (examples are shown here and have been added to Appendix E.5). For instance, we identified code snippets containing mainly variable declarations or configuration settings that, while syntactically valid, provide minimal learning value and potentially introduce noise. We also found examples of text with poor formatting (missing punctuation and paragraph breaks) and mathematical problems with repetitive structures (frequently beginning with "What is...") that could bias the model's generation patterns.

These findings demonstrate that our method can identify not just mislabeled data, but also more diverse cases that could subtly degrade model performance. This complements our quantitative evaluation in Appendix E.2.1 and provides concrete evidence of how our method can identify bad data. We believe these examples provide valuable insights into the types of training data that might impede model learning.

Thanks for the very positive comments about our work!

Q [Clarification on conceptual and technical contributions] “Probably due to the lack of a dedicated related work section, ... when introducing TSLOO at the beginning of Section 2. Similarly, the approximation in Section 3.1 was also used by Hara et al., 2019 and is "well-established" in the literature. ... If everything before Data Value Embedding (DVE) is a part of literature review, the authors should make it clear.”

A For “has TSLOO been studied before or is it a novel concept proposed by the authors? …”

The basic mathematical formulation of trajectory-specific LOO was indeed first introduced as 'SGD-influence' by Hara et al. (2019). However, our work significantly extends this concept by providing the first formal treatment of trajectory-specific LOO as a fundamental framework for understanding data influence in modern deep learning. While Hara et al. (2019) focused primarily on data cleansing applications, we motivate TSLOO by highlighting why traditional permutation-invariant LOO assumptions fundamentally break down in the context of large-scale neural networks and multi-stage training curricula. Section 2 thus serves dual purposes - providing necessary background while discussing why TSLOO is more suitable for data attribution in the era of foundation models. We thank the reviewer for this observation and have revised Section 2 to better distinguish the background material from our own insights. We have cited Hara et al. (2019) when introducing TSLOO.

For “... the approximation in Section 3.1 was also used by Hara et al., 2019 and is "well-established" in the literature.”

The reviewer is correct that Section 3.1 provides preliminary (highlighted in the section title), and our core technical contribution begins in Section 3.2. While "unrolling differentiation" was first developed by Hara et al. (2019) for data cleansing, similar mathematical expressions have emerged independently in various domains, particularly in continual learning theory. However, despite its theoretical elegance, this approach has seen limited practical adoption compared to influence functions, primarily due to its substantial computational demands. For instance, Hara et al. (2019) could only analyze the final epoch of SGD training and were restricted to small model architectures. Our key contribution begins in Section 3.2, where we introduce data value embedding, a novel framework that makes the approximation from Section 3.1 computationally feasible for large-scale models. This enables, for the first time, trajectory-specific LOO analysis at the scale of foundation models.

Q [Related work section] “I suggest a dedicated related work section to clearly delineate authors' novel contributions from existing concepts in the literature (e.g., Hara et al., 2019).”

A We thank the reviewer for the great suggestion! We have an extended related work section in Appendix A, but we completely agree that incorporating a focused discussion in the main text would enhance readability and help readers better understand the context and novelty of our contributions. Following your suggestion, we have added the following paragraph to the end of Section 2.

Data attribution methods primarily fall into two categories: LOO-based methods and Shapley value-based methods. While Shapley value-based methods (Ghorbani and Zou, 2019) offer elegant theoretical interpretation, they typically require expensive model retraining, which limits their practical applicability. As a result, LOO-based methods such as influence functions (Koh and Liang, 2017) have gained more attention due to their computational efficiency. However, many studies have demonstrated that influence functions can be highly unreliable when applied to deep learning models (Basu et al., 2020; Bae et al., 2022; Epifano et al., 2023). In this work, we argue that TSLOO provides a more appropriate attribution framework for deep learning, particularly in the context of foundation models. Various research communities have independently explored Taylor expansion-based technique for approximating TSLOO for different purposes (Hara et al., 2019; Zou et al., 2021; Evron et al., 2022; Wu et al., 2022; Wu et al., 2024; Ding et al., 2024). However, practical adoption has been hindered by computational demands. In this work, we propose a new method that overcomes the computational bottlenecks in approximating TSLOO for large-scale models.

Thanks for the reply to my comments and for clarifing my confusions on the distinctions between this work and existing works. I will keep my score.

Q [Groundtruth comparison experiment only on specific datasets and model types?] “The evaluation with ground truth focuses on specific datasets and model types ... generalizability of the findings.”

A We acknowledge the challenge of extending the groundtruth comparison experiments in Section 5.1 to other datasets and models, especially for larger-scale settings. Computing ground-truth LOO scores for large models is computationally infeasible, as it requires multiple complete training runs. This is a fundamental challenge faced by all research in this field. Additional experiments: During the rebuttal time, we conducted an additional experiment using a small CNN architecture (2 convolutional layers followed by a linear layer) trained on MNIST to demonstrate our method's effectiveness beyond simple MLPs. As we can see in this figure, we observe strong correlations between our estimated influence scores and ground-truth LOO scores in both (a) single-epoch removal (Spearman correlation: 0.818) and (b) all-epochs removal (Spearman correlation: 0.682) settings. This validates our method's effectiveness across different architectures. We sincerely thank the reviewer and this additional result has been added to Appendix E.2.

Indirect validations for large-scale settings. While direct ground-truth comparison is infeasible for larger models, we provide multiple indirect validations of our method's effectiveness at scale. In our mislabeled data detection experiments and data selection experiments (Appendix E.2.1) using ResNet18 on CIFAR10, our method achieves competitive or superior performance compared to existing baselines. Furthermore, our qualitative analysis (Section 5.4) shows that Data Value Embedding successfully identifies semantically relevant training examples for GPT2 on Wikitext-103, while baseline methods like influence functions sometimes select irrelevant data.

Q “In Line 169, the loss change is estimated by the first-order Taylor expansion, ... How this approximation can be valid in such a case?”

A We agree with the reviewer that in hypothetical scenarios where some data points are extremely influential (e.g., having exceptionally large gradients), the first-order Taylor approximation could break down as $\theta'_T$ and $\theta_T$ would become significantly different. However, in modern deep learning practice, particularly for foundation models, such scenarios are unlikely to happen as the learning rates are generally very small. In addition, gradient clipping is also a commonly used technique in foundation model pretraining. These methods effectively prevent any single data point from causing dramatic parameter changes.

Furthermore, our empirical validation supports the effectiveness of this approximation in practical ML training scenarios: (1) In addition to the high correlation with ground-truth TSLOO scores in Section 5.1 and Appendix E.2, Data Value Embedding achieves strong performance on the standard benchmarks of mislabeled data detection and data selection in Appendix E.2.1. (2) In the qualitative evaluation in Section 5.4, our method successfully identifies training data that are semantically correlated with the test data point.

Overall, while we acknowledge the theoretical possibility raised by the reviewer, in training setups typical of modern deep learning, our approximation provides a practical and reliable way to assess data influence.

We thank the reviewer for the very positive assessment of our paper!

Q [Assumption of SGD] “The method is explicitly tailored for SGD ...”

A We appreciate the reviewer raising this important point. While our theoretical framework is derived from SGD, we would like to emphasize two key aspects:

First, the assumption about SGD is significantly less restrictive than traditional methods like influence functions, which require strong convexity of the loss function and convergence to a global minimum. Our empirical results strongly support this: despite training models with Adam, our method successfully identifies training samples that are most relevant to the test data (Section 5.4) and achieves strong performance in mislabeled data detection and data selection benchmarks (Appendix E.2.1). These results demonstrate that our approach remains effective even when applied to models trained with SGD's variants.

Second, using SGD-based analysis as a proxy for SGD’s variants is a well-established approach in the literature, particularly in data attribution [1,4], data selection [2,3], and data difficulty estimation [5]. This widespread adoption is because SGD's simpler update rules make it more amenable to theoretical analysis while still capturing the essential dynamics of optimization. The empirical success of this approach across various domains suggests its validity as an analytical tool.

That said, we acknowledge that extending the framework of Data Value Embedding to Adam would be valuable future work. The current formulation strikes a balance between theoretical rigor and empirical feasibility, as demonstrated by our experimental results.

[1] Pruthi, Garima, et al. "Estimating training data influence by tracing gradient descent." NeurIPS 2020

[2] Fan, Simin et al. "DOGE: Domain Reweighting with Generalization Estimation." ICML 2024

[3] Yang, Yu, et al. "Towards sustainable learning: Coresets for data-efficient deep learning." ICML 2023

[4] Nguyen, Elisa, et al. "A Bayesian perspective on training data attribution." NeurIPS 2023.

[5] Paul, Mansheej, et al."Deep learning on a data diet: Finding important examples early in training." NeurIPS 2021

Q “Several assumptions ... such as model layer independency, learning rate scheduling, which might not be satisfied ...”

A We appreciate the reviewer's attention to the assumptions in our work.

Layer Independence: This assumption originates from the natural gradient descent literature [1] and has been widely adopted in influence function research [2, 3, 4], consistently demonstrating its effectiveness in practice. The assumption enables tractable analysis while preserving the essential characteristics of deep neural networks, as intra-layer interactions typically dominate cross-layer effects. This approximation has proven particularly valuable in developing practical algorithms for large-scale models.

Learning Rate Scheduling: Our theoretical analysis derives error bounds for learning rate schedules of the form $O(1/\sqrt{t})$ with maximum rate $O(1/\sqrt{T})$. Though foundation model pretraining may use different schedules, these schedules share key properties with our theoretical assumptions: small magnitudes and systematic decay. Our empirical results confirm that the method's effectiveness persists even when theoretical conditions are relaxed.

While these assumptions may not perfectly align with all real-world scenarios, they effectively approximate the conditions observed in our experiments. To validate this, we provide multiple empirical evaluations beyond theoretical analysis. For instance, our method demonstrates strong performance in detecting mislabeled data using ResNet18 on CIFAR10 (Appendix E.2.1), achieving comparable or better results than established baselines. The qualitative analysis (Section 5.4) shows that our method successfully identifies semantically relevant training examples, while baseline approaches sometimes select irrelevant data.

We acknowledge that these assumptions could be further relaxed in future work. However, our current framework provides valuable theoretical insights while maintaining practical effectiveness, as evidenced by our comprehensive empirical results. The success of our method in real-world applications, despite potential deviations from theoretical assumptions, suggests that these approximations capture the essential aspects of data influence in deep learning systems.

[1] Martens, James, and Roger Grosse. "Optimizing neural networks with kronecker-factored approximate curvature." ICML 2015

[2] Grosse, Roger, et al. "Studying large language model generalization with influence functions." arXiv 2023

[3] Kwon, Yongchan, et al. "DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models." ICLR 2024

[4] Choe, Sang Keun, et al. "What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions." arXiv 2024