Diff-In: Data Influence Estimation with Differential Approximation

Thank you for your time and effort during the review process, as well as for your recognition and thoughtful suggestions regarding our paper. Your feedback is highly valuable and has significantly contributed to improving the quality of our paper.

Weakness-1. Concerns about the polynomial growth of estimation error.

Thanks for your constructive comments!

(a) This phenomenon is intuitive. The reason that increasing the training steps leads to larger theoretical errors is that the optimized parameters diverge further from the initialized parameters as training progresses. This divergence makes accurate estimation more challenging and contributes to the accumulation of approximation errors.

It is also worth noting that some methods, such as those in [1, 2], exhibit a faster increase in error as $T$ grows. In contrast, our approach, with its polynomial error bound, demonstrates a significantly smaller growth rate compared to other methods, such as the exponential growth observed with SGD-Inf [2]. This indicates that our method effectively mitigates error accumulation over time, even as $T$ increases.

(b) It is important to emphasize that this bound reflects a worst-case scenario for error. In practice, Diff-In performs robustly even with larger $T$ values. As training progresses and the model approaches convergence, parameter changes become minimal, meaning larger $T$ typically has a negligible impact on errors. This highlights the effectiveness of our approach in maintaining stable performance.

(c) We present results comparing performance across different values of $T$. The table below illustrates how Diff-In performs relative to the maximum training time $T$ in coreset selection experiments on ImageNet. Notably, increasing $T$ does not degrade Diff-In's practical performance.

[1]. Theoretical and practical perspectives on what influence functions do. NeurIPS 2024.

[2]. Data cleansing for models trained with sgd. NeurIPS 2019

W-2. Concerns about the generalization ability of the samples selected by Diff-In.

Thank you for your helpful comments!

We believe that the coreset selected by Diff-In can be used for different capabilities of models and its generalization on downstream tasks.

As demonstrated in Section 5.3, "Coreset Selection for Vision-Language Pretraining," the subsets chosen by Diff-In enable pre-trained models to achieve superior performance across various tasks, including zero-shot classification, linear probing classification, and image/text retrieval. This improvement stems from the fact that the utility of data often depends on the inherent quality of the samples, which is relatively independent of the model's capabilities.

We would be happy to discuss this further if you have any concerns.

Thank you for your time and effort in reviewing our paper. We sincerely appreciate your recognition and thoughtful suggestions. Your feedback is valuable and has greatly enhanced the quality of our work.

Response to Weaknesses

W-1. Diff-In closely follows TraceIn’s approach of accumulating influence over successive training steps.

Thank you for bringing this up! While both TracIn and Diff-In accumulate influence over successive training steps, they differ significantly in their objectives, formulations, and outcomes, as summarized below.

(a) Objectives: Diff-In is designed to estimate the impact of removing a sample on the model's performance, known as leave-one-out influence. This measure can be used to assess a data sample's influence on both the loss and model parameters (see Sec.3.2). In contrast, TracIn focuses on estimating changes in the validation loss when a sample is excluded from training during specific iterations (referred to as TracInIdeal, see Sec. 2.1). However, TracIn does not guarantee an accurate estimation of leave-one-out influence.

(b) Formulation: Their distinct objectives lead to differences in their mathematical formulations. Diff-In defines sample-wise influence on model parameters as the cumulative sum of the parameter changes across successive training iterations. This difference term is approximated using second-order derivative information with high accuracy, eliminating the need for the model convexity assumptions required by existing methods. In contrast, TracIn estimates a sample's influence on loss values for a given dataset by summing its impact on the loss across various training iterations. This estimation relies on first-order gradient information for approximation. These differences become particularly evident when advanced optimizers, such as Adam or SGD with momentum, are used, as elaborated in Appendix B.

(c) Performance: These differences result in distinct outcomes for TracIn and Diff-In. TracIn is limited to estimating a sample's influence on loss values, whereas Diff-In can estimate the influence on both parameters and loss. Consequently, Diff-In supports a broader range of tasks, such as coreset, data cleaning, and data deletion/unlearning, while TracIn is restricted to coreset and data cleaning tasks. Furthermore, Diff-In consistently outperforms TracIn on data cleaning on GSM8K and coreset selection across ImageNet-1K, CC12M.

W-2 and W-3. Performance and speed comparison with some new works on large-scale datasets and large models.

Thank you for the constructive comment!

During the rebuttal period, we conducted coreset experiments to compare Diff-In with EK-FAC [1] and TRAK [2]. These experiments utilized the Llama-3-8B-SFT model with a selection rate of 10% on the OpenMathInstruct dataset, which contains 1.8 million math problems and solutions. Additional details are provided in Appendix D.4.

Diff-In outperformed TracIn and TRAK by 3.1% and 1.7%, respectively. Although EK-FAC proved to be the most computationally efficient method, its performance was significantly lower due to its reliance on the convexity assumption, which is not well-suited for deep models. Overall, the computation time across all methods was comparable, efficient, and scalable to large-scale models.

[1] Studying Large Language Model Generalization with Influence Functions. ArXiv 2308.03296.

[2] Trak: Attributing Model Behavior at Scale. ICML 2023.

Response to Questions

Q-1. More results with the Linear Data Modeling Score (LDS) metric.

Thank you for the question! LDS is a valuable metric for evaluating the effectiveness of data attribution methods by measuring how accurately they can predict outcomes. Here, we also use LDS to assess Diff-In, TracIn, Arnoldi-IF [1], TRAK [2], and IF on the ImageNet-1K dataset. The results demonstrate that Diff-In and TRAK consistently outperform the other methods.

[1]. Scaling Up Influence Functions. AAAI 2022.

[2]. TRAK: Attributing Model Behavior at Scale. ICML 2023.

[3]. Estimating Training Data Influence by Tracing Gradient Descent. NeurIPS 2020.

Q-2. Could the authors provide similar speed comparisons for large-scale models?

Thanks! Please refer to the response to W-2 and W-3.

Thank you for your thoughtful review and response. We appreciate your feedback and recognize the value of your insights in enhancing our work.

If there are any aspects of the paper that require clarification or if you have suggestions for improvement, we would be eager to address them. Your input is valuable to us, and we are committed to making any necessary adjustments to strengthen our submission.

Additionally, if you feel that our revisions adequately address your concerns, we would be grateful if you could consider raising the score.

Thank you once again for your time and consideration!

We have already included those new experimental results in the paper along with those discussions of general interest to the article during the rebuttal phase. Specifically, we have made the following changes:

1. Fixed typos (main paper). 

2. Added new references [1-13] in the related work section (appendix, Sec.C). 

3. Added clarification of Data deletion and new baseline [14, 15] (main paper). 

4. Added speed and performance comparison with SOTA methods on experiments for Llama-3-8B on OpenMathInstruct (appendix, Sec.D.4.1 and Table 11). 

5. Added further ablation study on the choice of the number of selected checkpoints (appendix, Sec.D.4.2 and Table.11 and Table 13). 

6. Added further performance comparison with IP and OGI on the cleaning experiments for GSM8K  (appendix, Sec.D.4.1 and Table.12). 

7. Added guidelines for checkpoint selection (appendix, Sec.D.4.3). 

8. Added experiments and discussion about the growth of the bound (appendix, Sec.D.5.1). 

9. Added discussion about the assumption of the bound (appendix, Sec.D.5.2). 

10. Moved the discussion of limitations to the appendix.  

Reference:

[1]. Studying Large Language Model Generalization with Influence Functions. Arxiv 2308.03296. 

[2]. TRAK: Attributing Model Behavior at Scale. ICML 2023. 

[3]. Scaling Up Influence Functions. AAAI 2022. 

[4]. Revisit, Extend, and Enhance Hessian-Free Influence Functions. Arxiv 2405.17490

[5]. Outlier Gradient Analysis: Efficiently Improving Deep Learning Model Performance via Hessian-Free Influence Functions. Arxiv 2405.03869. 

[6]. DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models. ICLR 2024.

[7]. A Survey of Machine Unlearning. Arxiv 2209.02299.

[8]. Datamodels: Predicting Predictions from Training Data. ICML 2022. 

[9]. What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions. ArXiv 2405.13954

[10]. "What Data Benefits My Classifier?" Enhancing Model Performance and Interpretability through Influence-Based Data Selection. ICLR 2024.

[11]. HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks. AAAI 2021. 

[12]. Achieving Fairness at No Utility Cost via Data Reweighing with Influence. ICML 2022. 

[13]. Revisiting inverse Hessian vector products for calculating influence functions. Arxiv 2409.17357

[14]. Efficiently Erasing Samples from Trained Machine Learning Models. NeurIPS 2021 Workshop.

[15]. Machine Unlearning Method Based On Projection Residual. DSSA 2022.

Thank you very much again!

[1]. Estimating training data influence by tracing gradient descent. NeurIPS-2020.

[2]. Data cleansing for models trained with SGD. NeurIPS-2019.

[3]. Understanding black-box predictions via influence functions. ICML-2017.

[4]. Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models. ICLR 2024.

[5]. Learning transferable visual models from natural language supervision. ICML-2021.

[6]. How Does Batch Normalization Help Optimization? NeurIPS-2018.

[7]. Visualizing the Loss Landscape of Neural Nets. NeurIPS-2018.

Thank you for your time and effort during the review process, as well as for your recognition and thoughtful suggestions regarding our paper. Your feedback is highly valuable and has significantly contributed to improving the quality of our paper.

Weakness.1 The implementation relies heavily on saved checkpoints during training, with unclear guidelines on optimal checkpoint selection.

Thanks!

(a) The checkpoint selection process involves determining the number of checkpoints, $m$, and the sampling method. These parameters were optimized through ablation experiments conducted on the CIFAR-10 dataset using the ResNet-18 backbone, as detailed in Sec. 5.4.

(b) For the selection of $m$, as shown in Figures 3(a) and 3(b), increasing $m$ improves approximation precision and reduces variance but also increases computation time. To balance computational cost and precision, we set $m=5$ as performance tends to grow slowly after this point while maintaining high efficiency. During the rebuttal, we have further conducted coreset selection experiments on the OpenMathInstruct dataset for fine-tuning Llama-3-8B (see Appendix.D.4 for details). OpenMathInstruct is a large math instruction tuning dataset with 1.8M problem-solution. Here are the results. We can observe a similar trend, that is, the increase of m will improve the performance and stability, but this improvement will be saturated at m greater than 5.

(c) For the sampling guideline, on the coreset experiments of ResNet-50 and ImageNet-1K at a selection rate of 20%, we experimented with fixed-step selection (mode-0) and uniform random sampling (mode-1), both of which achieved similar performance. We also compare the other two selection modes, namely, mode-2 selecting checkpoints from the first 5 epochs; and mode-3 selecting checkpoints from the last 5 epochs. These two modes perform inferiorly. Consequently, we adopted uniform random sampling in our experiments.

(d) In all experiments presented in the main paper, we applied the above checkpoint selection strategy. The promising results are shown in Tables 1, 2, 3, and 4 across various tasks and datasets, such as coreset for ImageNet-1K and CC12M, and data clean for GSM8K, demonstrating the effectiveness and generalizability of this strategy to a wide range of datasets.

Weakness.2 Limited Generalizability: focusing on sample-influence, lacking broader application to hyperparameters.

Thank you for the thoughtful suggestion! Existing methods for influence estimation, including ours, rely on calculating derivatives, which require the function to be differentiable. As a result, the non-differentiable nature of the loss function with respect to model hyperparameters makes it difficult to directly apply such methods to estimate the influence of hyperparameters on model learning.

In the future, leveraging estimation methods (e.g., straight-through estimation) to estimate gradients could be a promising solution for incorporating hyperparameter influence into the existing framework, which is beyond the scope of this paper, focusing on data influence. We consider it an exciting direction for future exploration.

Weakness.3 Theoretical Assumptions: Analysis assumes Lipschitz continuous gradients and bounded gradient norms, with limited discussion of assumption violations.

Thank you for your comments!

Lipschitz continuity and gradient norms are commonly used to characterize the smoothness of a neural network's loss landscape. Modern deep learning models incorporate techniques such as normalization layers (e.g., batch normalization and layer normalization) and shortcut connections to enhance smoothness and continuity [6,7], facilitating optimization. These techniques make the assumption of smoothness generally valid in practice. This stands in contrast to many studies [3,4] that rely heavily on the convexity of neural networks. Our assumption, by comparison, is much easier to satisfy in real-world scenarios.

If the conditions for $\ell$-Lipschitz continuous gradients are not met, the gradient norms $g$, the values of $g$ and $\ell$ can become very large. In such cases, the error bound derived from these parameters may lose its practical relevance. However, this does not necessarily imply that the algorithm will fail in practice.

We include the discussion in Appendix.D.5.

Thank the authors for the detailed responses! I read the responses and the updated PDF. My concerns have been mostly addressed.

Thank you for your time and effort during the review process, as well as for your recognition and thoughtful suggestions regarding our paper. Your feedback is highly valuable and has significantly contributed to improving the quality of our paper.

Weakness.1-1. Questions on the hyperparameter (the number of selected time steps and the sampling method): generalization across different datasets.

(a). The number of checkpoints and the sampling method was optimized using the CIFAR-10 dataset and a ResNet-18 backbone, with results presented in Figures 3(a) and 3(b) in Sec. 5.4. These results demonstrate that setting the number of checkpoints $m = 5$ provides the best trade-off between accuracy and efficiency, while uniform sampling across all learning stages achieves the optimal performance. Consequently, we adopt $m = 5$ and uniform sampling for checkpoint selection in all our experiments.

Our model's strong performance across three tasks (e.g., coreset selection, data cleaning, data deletion) and eight datasets (e.g., ImageNet-1K/22K, CC12M, GSM8K, OpenMathInstruct) highlights the effectiveness of this configuration. Notably, we achieve these results without requiring hyperparameter tuning for specific datasets. This demonstrates that the hyperparameters of our method are robust and not sensitive to model size, dataset size, or task type, enabling excellent generalization across various scenarios.

(b). During the rebuttal period, we conducted an additional ablation study for corset selection to validate our claim regarding the number of checkpoints. This study was performed on two large-scale scenarios: vision-language pre-training (VLP) using CLIP-ViT-Base and supervised fine-tuning (SFT) of a large language model on the OpenMathInstruct dataset.

First, we present the VLP results at a selection rate of 10% in the table below. The results indicate that as $m$ increases, both performance and stability improve. However, this improvement plateaus once $m$ exceeds 5.

Next, we present the coreset experimental results for supervised finetuning Llama-3-8B-Instruct at a selection rate of 10% on the OpenMathInstruct dataset. This dataset comprises 1.8 million problem-solution pairs designed for math instruction. As detailed in Appendix D.4, the results reveal a similar trend: increasing $m$ enhances performance and stability, but the improvement saturates when $m$ exceeds 5.

Further, for the sampling method, on the coreset experiments of ResNet-50 and ImageNet-1K at a selection rate of 20%, we validate that uniform random sampling is a good choice. Here, we experimented with fixed-step selection (mode-0) and uniform random sampling (mode-1), both of which achieved similar performance. We also compare the other two selection modes, namely, mode-2 selecting checkpoints from the first 5 epochs; and mode-3 selecting checkpoints from the last 5 epochs. These two modes perform inferiorly. Consequently, we adopted uniform random sampling in our experiments.

In summary, these findings further validate that the hyperparameters optimized on the CIFAR-10 dataset with a ResNet-18 backbone generalize well across different datasets and models. This demonstrates that hyperparameter tuning is unnecessary for new tasks and models, ensuring broad applicability with high efficiency.

Weakness.2-3. For experiments on large models with large datasets, the selection of m should be considered.

Please refer to our responses to 1-1 and 1-2 for details on why selecting $m=5$ is generalizable across different model sizes and tasks.

Here, we present additional experiments in LLM SFT scenarios to evaluate Diff-In's performance and runtime efficiency with varying values of $m$. More details are provided in Appendix D.4. Diff-In consistently delivers the best performance. As $m$ increases, both performance and stability improve, with diminishing returns once $\geq 5$. These findings align with the ablation study in the main paper and our responses to 1-1 and 1-2.

We observed that with 5 checkpoints, Diff-In incurred only about 2 additional GPU hours compared to TracIn while improving performance by over 3%. This checkpoint configuration is robust across different settings and does not require re-validation on new datasets and tasks.

Weakness.3. Concerns about the experiment and title on the data deletion subsection

Thank you for the great suggestion! We’ve added clarifications in Section 5.2 to emphasize that our focus is on unlearning data points without retraining. Additionally, we included two unlearning baselines, SSSE [a] and PR [b], from [7] for comparison. Despite our approach being broadly applicable to various data-centric tasks, it still outperforms methods specifically designed for the unlearning task.

[a]. Efficiently Erasing Samples from Trained Machine Learning Models. NeurIPS 2021 Workshop.

[b]. Machine Unlearning Method Based On Projection Residual. DSSA 2022.

Weakness.4. Some related works need to be discussed [8,9,10,11,12,13,14].

Thanks! We have already added them in the related work section in the revision.

Weakness.5. The reviewer pointed out some typos.

Thanks! We have revised them all!

[1]. Studying Large Language Model Generalization with Influence Functions. Arxiv 2308.03296.

[2]. TRAK: Attributing Model Behavior at Scale. ICML 2023.

[3]. Scaling Up Influence Functions. AAAI 2022.

[4]. Revisit, Extend, and Enhance Hessian-Free Influence Functions. Arxiv 2405.17490

[5]. Outlier Gradient Analysis: Efficiently Improving Deep Learning Model Performance via Hessian-Free Influence Functions. Arxiv 2405.03869.

[6]. DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models. ICLR 2024.

[7]. A Survey of Machine Unlearning. Arxiv 2209.02299.

[8]. Datamodels: Predicting Predictions from Training Data. ICML 2022.

[9]. What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions. ArXiv 2405.13954

[10]. "What Data Benefits My Classifier?" Enhancing Model Performance and Interpretability through Influence-Based Data Selection. ICLR 2024.

[11]. HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks. AAAI 2021.

[12]. Achieving Fairness at No Utility Cost via Data Reweighing with Influence. ICML 2022.

[13]. Revisiting inverse Hessian vector products for calculating influence functions. Arxiv 2409.17357.

Weakness.1-2. How the randomness of the selection affects the performance

Based on the above experiments and the ablation results presented in Figures 3(a) and 3(b) in Sec. 5.4, the small standard deviation (less than 0.05 in the approximation accuracy test) across multiple experimental runs on CIFAR-10 suggests that the randomness introduced by random sampling becomes negligible when $m\geq5$. We attribute this to the fact that uniform sampling ensures the selected checkpoints are evenly distributed across different training stages, capturing diverse model behaviors throughout learning.

This observation also highlights a key guideline for checkpoint selection: sample evenly to ensure that time steps span across all training stages, providing comprehensive coverage of the model's evolution during learning.

To validate this selection strategy, we conducted additional experiments on the coreset experiments of ResNet-50 and ImageNet-1K at a selection rate of 20%, comparing fixed-step selection (Mode-0) and uniform random sampling (Mode-1). As shown in the table below both methods achieved comparable performance. We also evaluated two alternative selection modes: Mode-2, which selects checkpoints from the first 5 epochs, and Mode-3, which selects checkpoints from the last 5 epochs. These two modes performed noticeably worse.

Notably, for Mode-1, the performance variance was within 0.12% across 5 runs, further confirming that the randomness introduced by sampling is negligible as long as the sampling spans different learning stages.

Weakness.2-1: Discussion and comparison with [1,2,3,4,5]

We sincerely thank the reviewers for providing these valuable references. EK-FAC [1], TRAK [2], and Arnoldi-IF [3] leverage Kronecker factorization, kernel estimation techniques, and Arnoldi acceleration, respectively, to improve the scalability of influence estimations (IF). Despite these advancements, [1] and [2] rely on the convexity assumption of loss functions, which is often violated in practical scenarios. While Arnoldi-IF [3] improves scalability through the Arnoldi method, however, it still requires the assumption of the network's convexity. OGI [5] connects the identification of harmful training samples with outlier detection by applying influence functions in the gradient space. Both [4] and [5] perform well in handling noisy and outlier data.

In response to the reviewers’ suggestions, we have conducted additional experiments comparing our approach with [1], [2], and [3] on the coreset selection task, as well as with [4] and [5] on the data cleaning task.

(a) First, we present the coreset experiment results. Specifically, we evaluated Llama-3-8B-SFT at a selection rate of 10% using the OpenMathInstruct dataset, which includes 1.8 million problem-solution pairs. For additional details and discussions, please refer to Appendix D.4. The results demonstrate that Diff-In outperforms all other methods, highlighting its effectiveness in measuring sample importance for large datasets and models.

(b) Next, we conducted data-cleaning experiments on the GSM8K dataset, comparing Diff-In with IP [4] and OGI [5]. The experimental setup aligns with the description in Sec.5.1. In these experiments, Diff-In consistently achieved significantly superior performance, further demonstrating its effectiveness.

Weakness.2-2. Data cleaning using validation influence.

The table below presents the results of the data-cleaning task after replacing the self-influence estimated by Diff-In with validation influence. A slight drop in performance was observed, suggesting that self-influence may be a more effective method for identifying incorrect or outlier data. Additionally, self-influence is computationally more efficient, as it requires only the sample itself to calculate influence, rather than relying on a separate validation dataset.

A high self-influence value indicates that the model lacks sufficient training samples to make predictions consistent with the labeled information. This often serves as a reliable signal for detecting incorrect data or outliers.

We appreciate your recognition and your reply!

We have added the experimental content of this Rebuttal and the discussion content of general interest to the article, and the specific updates are as follows. Please let us know if you have any further interest or concerns.

Specifically, we have made the following changes:

1. Fixed typos (main paper). 

2. Added new references [1-13] in the related work section (appendix, Sec.C). 

3. Added clarification of Data deletion and new baseline [14, 15] (main paper). 

4. Added speed and performance comparison with SOTA methods on experiments for Llama-3-8B on OpenMathInstruct (appendix, Sec.D.4.1 and Table 11). 

5. Added further ablation study on the choice of the number of selected checkpoints (appendix, Sec.D.4.2 and Table.11 and Table 13). 

6. Added further performance comparison with IP and OGI on the cleaning experiments for GSM8K  (appendix, Sec.D.4.1 and Table.12). 

7. Added guidelines for checkpoint selection (appendix, Sec.D.4.3). 

8. Added experiments and discussion about the growth of the bound (appendix, Sec.D.5.1). 

9. Added discussion about the assumption of the bound (appendix, Sec.D.5.2). 

10. Moved the discussion of limitations to the appendix.  

Reference:

[1]. Studying Large Language Model Generalization with Influence Functions. Arxiv 2308.03296. 

[2]. TRAK: Attributing Model Behavior at Scale. ICML 2023. 

[3]. Scaling Up Influence Functions. AAAI 2022. 

[4]. Revisit, Extend, and Enhance Hessian-Free Influence Functions. Arxiv 2405.17490

[5]. Outlier Gradient Analysis: Efficiently Improving Deep Learning Model Performance via Hessian-Free Influence Functions. Arxiv 2405.03869. 

[6]. DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models. ICLR 2024.

[7]. A Survey of Machine Unlearning. Arxiv 2209.02299.

[8]. Datamodels: Predicting Predictions from Training Data. ICML 2022. 

[9]. What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions. ArXiv 2405.13954

[10]. "What Data Benefits My Classifier?" Enhancing Model Performance and Interpretability through Influence-Based Data Selection. ICLR 2024.

[11]. HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks. AAAI 2021. 

[12]. Achieving Fairness at No Utility Cost via Data Reweighing with Influence. ICML 2022. 

[13]. Revisiting inverse Hessian vector products for calculating influence functions. Arxiv 2409.17357

[14]. Efficiently Erasing Samples from Trained Machine Learning Models. NeurIPS 2021 Workshop.

[15]. Machine Unlearning Method Based On Projection Residual. DSSA 2022.

Thank you very much again!