Do Influence Functions Work on Large Language Models?

• The paper presents only negative results without offering alternative approaches. For example, it would be beneficial to discuss alternative definitions that do not rely on gradients. While RepSim does not fit the traditional definition of gradient-based influence functions, it does show promising results and could be considered as an alternative. Thus, on one hand, the authors are calling for alternative methods to quantify influence, and on the other they are showing that RepSim, which is such an alternative, performs great on all their benchmarks. This seems a contradiction.
• The experiments are conducted on small datasets which are much smaller than typical LLM training data. It is unclear whether RepSim would still outperform other methods on a larger scale. This could also be a fourth reason why influence functions struggle with LLMs, as identifying the most influential samples might be more challenging with larger datasets. Unfortunately, this aspect is not discussed.
• DataInf, LiSSA, and GradSim are not the only methods for estimating influence functions. A quick search reveals two other methods ([1] and [2]) that seem to perform well. In fact, [1] suggests they outperform RepSim. However, no comparison is made with these methods, making it unclear if the conclusions generalize beyond the three tested methods.
• The experiments use only a single, outdated model, Llama-2, making it unclear whether the findings are specific to this model or if they generalize to other LLMs.

Overall, I am concerned that the conclusions may not generalize beyond the three methods tested, the single model used, and the small datasets employed. Additionally, I question whether RepSim should be considered something that "should be outperformed," as it could be seen as a different type of influence estimator. Of course, similarity is not the same as influence, but given that the evaluated applications of influence estimation all seem to be covered by RepSim, there does not seem to be a reason to make this distinction based on the current experiments.

My current score is based on an expectation that the authors are able to address some of my concerns to the expected degree. However, if they cannot, I will reduce my score. In contrast, if they can additionally clarify some aspects, especially with regards to RepSim and the missing baselines, I would also increase my score.

[1] Choe, Sang Keun, et al. "What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions." arXiv preprint arXiv:2405.13954 (2024).

[2] Grosse, Roger, et al. "Studying large language model generalization with influence functions." arXiv preprint arXiv:2308.03296 (2023).

Major Concerns:

1. Lack of Critical Details: The paper is missing essential details, particularly in the experimental setup described in Section 3. The authors only mention the dataset and model used but do not explain how they compute the value of the influence function. This omission raises several questions:

• Gradient Calculation: How did the authors compute the gradients? Specifically, did they treat all parameters across different layers as a single $\theta$, or did they calculate gradients selectively for specific layers? If they used the former approach, how did they manage the complexities of LoRA and fit the massive gradient matrix into a 40GB A100 for computation?
• Loss Function Choice: How did the authors choose an appropriate loss function given the sequence generation nature of large language models (LLMs)? Additionally, how did they define the (x,y) pairs for use in the influence function? I also attempted to access the GitHub repository provided by the authors, but it appears to be unavailable.

2. Unreasonable Experimental Logic: The experimental design lacks coherence, especially in the harmfulness experiment. The influence function is supposed to measure the impact of individual training samples on the loss function, which the loss function in this context, should represent some measure of harmfulness. This ties back to the first concern: without a clear definition of the loss function, the entire basis of the harmfulness experiment is unclear. The same issue arises in the other two experiments.

3. Unconvincing Explanations for Failures: The explanations offered for potential reasons behind the failures are not compelling.

• Uncertain Convergence State: The authors could refer to [1], where similar convergence violations are discussed but with different conclusions.
• Effect of Parameter Changes: The authors only demonstrate that the L2 norm of $\Delta \theta$ does not correlate with the attack success rate (ASR). However, this merely shows that the L2 norm is ineffective and does not provide evidence against the validity of the influence function. The utility of the influence function lies in its problem-specific nature and the associated loss landscape, which differentiates it from simple metrics like the L2 norm of $\Delta \theta$.

Minor Concern:

• The definition of "Acc" on the y-axis of Figure 8 needs to be clarified.

Reference: [1] Schioppa, Andrea, et al. "Theoretical and practical perspectives on what influence functions do." Advances in Neural Information Processing Systems 36 (2024).

• I have concerns with the paper presentation. I find this paper confusing at various points.
  • Background information and experiment settings are not explained clearly.
  • Tables and figures are not self-contained or explained clearly. The connection between the tables/figures and the main text is weak and may be strengthened.
  • See my questions below.

(Please be aware that influence function is not my direct research area and I am only familiar with the basic concepts; I gave a confidence score of 2)

• Limited Motivation:
• • The complexity and computational aspects of iHVP approximation are not fully discussed
• • Benefits compared to existing data filtering methods (such as those in Dolma paper: https://arxiv.org/pdf/2402.00159) are not well justified
• Methodological Clarity:
• • AdvBench task specifications could benefit from example illustrations
• • RepSim's methodology and its strong performance lack detailed explanation
• • The rationale for using different datasets for performance evaluation and causality analysis is unclear (e.g., MNIST)
• • No evaluation on downstream tasks such as in https://arxiv.org/pdf/2104.12567
• Analysis Scope:
• • Study is limited to Llama as the backbone model
• • Section 4.2's findings raise questions about influence function behavior in smaller language models
• • Relationship between uncertainty and factors like data, model size, or fine-tuning could be explored further
• • Potential connections to pretraining effects not discussed