We thank the reviewer for the insightful feedback and questions!

Although down-scaling and pruning are the main topic of the paper, the technical details on methods used is very limited (even in the Appendix too). If space is an issue, I would suggest to cut down the paper motivation which is repeated multiple times throughout the paper.

We are working on including more details about the pruning techniques to make our work self-contained. Will update once we apply the edits.

Relatedly, there is very little discussion regarding down-scaling vs. pruning. For general readers it would be helpful to understand what are the difference between the two, and is one a specific version of the other?

We can view dense down-scaling as a form of structured pruning. While SparseGPT prunes at the granularity of individual weight, we can view dense-downscaling as pruning at the granularity of entire neurons and entire layers. Some important differences between dense-scaling and pruning include:

• The order of weights removal and training – viewing dense downscaling as structured pruning, weights removal happens before training. However, for pruning, weight removal happens after training (and is followed by repair).
• Granularity of weights removal – dense downscaling removes entire neurons & layers, whereas pruning removes individual weight.

For learning tasks evaluation, why only consider task with scalar values as labels? I understand this needs to be something that model can generalize through the examples, but if we focus on language capability of the model, I would expect that a natural language task is used instead.

Please refer to our global response, additional task evaluation section. We provide additional results on the LAMBADA benchmark, a natural language based ICL task. Our focus on synthetic ICL tasks (thus scalar/synthetic labels) is driven by our goal to disentangle fact recall from ICL tasks. Many existing natural language based ICL tasks run the risk of relying on the model's parametric knowledge. For example, one of the ICL examples in GPT3 paper is language translation, which clearly heavily depends on pre-trained knowledge of source and target language. We already know that such pre-trained knowledge is impaired in our QA experiments. Synthetic tasks, due to their synthetic nature, are largely free of such issues.

For ICL results, it seems the performance drops significantly (not gradually) from 70% above, do you have intuition why?

We conjecture that ICL learning ability depends on a small set of weights that act as the in-context gradient descent module [1]. At 70% and above, SparseGPT may eventually decide to remove this small set of weights, causing sudden drop of accuracy.

As the paper only use some particular pruning methods, do you have any opinion on whether the same findings will hold for other pruning methods? It would be good to have a short discussion on the difference between them.

We believe our findings will generalize to other pruning methods. This is because our findings not only apply to pruning (SparseGPT, Wanda), but also to dense down-scaling. The consistency of our results between sparse and dense down-scaling provides evidence that our observation is fundamentally about model size reduction in general, measured in the number of parameters.

Section 5 is hard to follow without examples, there are multiple notations without explanation, e.g. K (page 7), x, D=4, N=32, etc

We will make sure to improve our writing and let you know once we finish updating the paper!

Would be good to release the version of datasets that are used for benchmarking

Yes we will release the datasets we use!

We appreciate the efforts that go into providing us with so much feedback! We strive to address them fully, and we look forward to hearing back!

The major weakness of the paper is the effective testing of LLM parametric and ICL knowledge. In particular, how do the authors verify that the in-context evidence contradicts a fact present in training data? Although LLama was trained on publicly available data, it is not a simple matter to verify that the answers in the Q&A datasets align or misalign with the massive dataset used to train LLamA (note that, the dataset itself was never released, so all public datasets would have to be evaluated in their entirety for verification). In this case, it is not clear how Verifying that "answers do not exist in the pre-training corpus" is possible. While the authors discuss previous work which has explored LLMs' abilities to override in-memory/parametric knowledge, such works set up measurement guardrails to do so through fine-tuning

We agree with your point and took actions to address your concern. It is indeed nearly impossible to verify that counter-factual answers do not exist in pre-training corpus. We amended the phrasing of our experiment design to avoid making absolute statement about dataset contamination. Furthermore, we quantify the extent to which they exist in the pre-training corpus, by running the counterfactual QA evaluation using dense models without the counterfactual context – we count the model’s answer as correct if and only if it matches the counterfactual answer. If the training dataset of LLaMA is contaminated with the counterfactual question/answers, we should expect to see reasonably high accuracy under this setup. Instead, we observe near-zero accuracy for both LLaMA-13 and LLaMA-33B model (0.1% for both LLaMA-13B and LLaMA-33B).

Furthermore, the exact measurement of accuracy used in the paper is potentially incorrect and too conservative for recently release instruction-tuned LLMs like LLaMA and OPT. From the text: "Answers are the model’s prompt completions produced by greedy decoding. We report the percentage of answers that exactly match ground truth." <- Two important remarks: greedy is known to be extremely suboptimal for recent instruction-tuned LLMs, and an exact match is not necessarily a fair metric. Such chat models are known to be extremely wordy, so if the model produces some lead up text followed by the correct answer, this metric discounts such a correct response. For the former, it makes sense as a fair, reproducible benchmark across different sparsity percentages per model (e.g., nucleus sampling would produce differing results between runs), please include in the text why greedy is used. However, note that the latter is an extremely important problem which biases all related results.

To begin with, both LLaMA[4] and Pythia[5] models (which we study) use greedy decoding and exact match during evaluation. So our experimental design is standard as is. Furthermore, we do not use instruction-tuned models, so verbosity should not be a concern either. Nevertheless, we collected extra results to address your concern about decoding and wordiness; and show that our conclusion holds up to additional scrutiny. We re-run key subset of our experiments using a more advanced sampling scheme and more tolerant matching criteria according to your proposal. Specifically, we repeat our LLaMA-13B and LLaMA-33B experiments on TriviaQA benchmark with and without context. We use beam search with beam size of 3 (some paper in the past, e.g., OPT[6], GPT3[7] used beam search during evaluation). To get an answer from the model, we generate 32 tokens from the model, and check whether the answer appears anywhere within the generated tokens. The average number of tokens for TriviaQA answer is 4.7, so a model can generate extra tokens irrelevant to the answer whilst still correctly answering the question. Here’re our results:

• Our conclusion still holds in this setting: using our current experimental design, averaging over two LLaMA models, one can prune 30% and 40% weights on Closebook and Openbook TriviaQA task, respectively. Using beam search, one can prune 30% and 50% weights on Closebook and Openbook TriviaQA tasks, respectively. The ability to retrieve answers from provided context remains more resilient to pruning than the ability to recall information learnt during pre-training.
• Additionally, we observe a noticeable accuracy boost likely due to better decoding strategy using beam search (about 10-20% accuracy improvement). This is expected, as beam search with beam size of 3 costs ~3x the compute of greedy decoding.
• Please see full pruning curves and comparison with greedy decoding in Appendix L of our newly updated paper draft.

"From work on image classification, however, we know that down-scaling neural networks affects more than just top-line metrics or task accuracy. Pruning, for example, can introduce biases (Hooker et al., 2019) or disproportionate affects on certain subsets of the data (Jin et al., 2022)." <- This claim is too strong, it makes it seem as though it is a certainty that such effects occur given down-scaling. However, a significant amount of work has shown that pruning is an effective tool for vision models.

We agree. Our use of the word “can” intends to convey the uncertainty. If you think this is insufficient, we are happy to be more precise about the precondition of the said phenomenon – “Pruning to very high sparsity, for example, can introduce biases…”

"It is difficult to assess these abilities in isolation, as a standard downstream task needs to process the information provided in context as well as access the information stored in weights." <- Please contrast related work which has previously explored zero-to-many shot ICL performance (across different target applications); see the following for an extensive overview: Dong et al, "A Survey on In-context Learning", https://arxiv.org/pdf/2301.00234.pdf

Thank you for sharing the survey on ICL. Our definition of ICL is consistent with the one provided in the survey. The survey summarizes work on various aspects of ICL (algorithmic mechanisms behind it, evaluation of ICL, benchmarks, improving ICL, ICL in smaller models via distillation, etc.), but does not mention any work attempting to disentangle ICL and fact recall, or evaluating how pruning affects ICL, thus providing further evidence on the novelty of our contribution.

"Improve inference efficiency. Our work reveals that scaling down model size alone has little impact on tasks demanding processing information in the LLM’s context. Practitioners may thus use our findings to identify scenarios where decisions could be routed to a smaller model instead of a larger one without hurting task performance (Chen et al., 2023; Dohan et al., 2022)." <- The latter work already explores how the parameter size affects performance. In particular, the Wanda paper already tackles the question of how pruning affects ICL performance (and compares to SparseGPT)

Please also refer to our global response for a discussion of our contribution within the context of existing pruning work.

FrugalGPT[8] (Chen et al.,), SparseGPT[1] and Wanda[2] indeed all investigated the task performance-size/cost trade-off of LLMs. However, when a new task emerges outside of those examined by these prior studies, how can a practitioner know if it is a good idea to route it to a smaller model? Our work provides insight in this scenario: practitioners can assess how much the said task relies on parametric knowledge versus context processing capability. The more a task relies on context processing, the more likely such a task can be routed to a smaller model.

"Our work differs from prior scaling studies in two ways: while prior work (Kaplan et al., 2020b) studies joint scaling of both pre-training corpus size and model size, we focus on scaling model size alone. Furthermore, instead of measuring task performance, we focus on foundational capabilities of LLMs–fact recall and ICL. These capabilities drive the success for many real world applications of LLMs" <- This is wrong for a number of reasons. Firstly, "we focus on scaling model size along" is not a valid contribution, as this would, by definition, be provided in the study of "joint scaling of both pre-training corpus size and model size."

We use this statement to highlight a difference in methodological choice – we want to be upfront about the more focused and thus narrower scope of our study. To be clear, we use this statement to disambiguate, not to present a contribution.

Secondly, the work of Kaplan does not study pruning, but rather LLM model size->training->resulting performance. It is necessary to demarcate the difference between these two paradigms.

We agree. The difference is important – 1). In addition to dense scaling (Sec 6), which follows the paradigm of Kaplan et al, we also studied pruning. 2). Instead of perplexity, we assess LLMs on its ability to recall information from pretraining vs process contextual information. Both differences are also our key contributions.

"In-weight versus in-context learning" <- Please explain how the considered work differs from Longpre et al 2022, which extensively explores In-weight versus in-context learning.

This paper focuses on constructing experiments that contradict existing knowledge in a variety of ways, including changing the answer to an alias, changing the type of the answer. Alongside DisentQA they provide key methodological foundations to our work (specifically in the counterfactual QA setup). Our work differs from this in that 1). Our work studies pruning, and 2). Our work investigates context-processing abilities beyond answering counter-factual questions – we also construct in-context learning tasks to learn parametric functions to stress test a model’s ability to process information in context.

"the versatility of LLMs calls for a different approach to assessing pruned models. Our work begins to fill this gap, proposing to evaluate pruning’s effect on fact recall and ICL." <- As previously mentioned, what the authors define as fact recall is equivalent to the task of zero-shot question answering; the effect of pruning on various tasks has been explored, e.g., within the papers of the pruners specifically used within this work (SparseGPT and Wanda), as well as in the LLM-Pruner paper. Furthermore, the effect of pruning an LLM to various sparsity levels on ICL was extensively explored in the Wanda paper. Please revise your contributions, and position them within the context of previous works.

we focus on foundational capabilities of LLMs – fact recall and ICL" <- Fact recall is zero-shot Q&A, which may be thought of as a specific task. Please adjust this claim.

We emphasize that fact recall is not equivalent to zero-shot closebook question answering (correct us if we’re wrong, by zero-shot QA we believe you mean zero-shot closebook QA). The model may rely on the ability to recall parametric knowledge even in openbook QA settings. Hence we argue that no single task exclusively test the model’s ability to recall information from pretraining versus its ability to process information in context. Hence a primary contribution of ours is to disentangle the assessment of both abilities to the best of our abilities. Please refer to our global response for the discussion of our contribution in the context of prior pruning work.

Nevertheless, we are happy to better acknowledge the work done by prior pruning research, and revised our phrasing to the following: … approach to assessing pruned models. Prior work does a commendable job to assess model size and task accuracy trade-off. Our work continues to expand our toolkits for empirical assessment, proposing to evaluate pruning’s effect on fact recall and ICL.

" In all the above settings, as a simple point of comparison, we measure the effect of downscaling on perplexity" <- Please note in the paper that this was previously considered in both the SparseGPT and Wanda papers. -"

Agree. Revised to credit this to prior pruning papers.

Thank you for your helpful feedback! Here's our response to your comments.

Fact Recall and In Context Learning are some reasonable aspects, but the authors could have considered more. Detailed Instruction Following, and Heavy Reasoning feel like other key aspects, as well as the ability to learn from Few Shot inline. I would have loved to see some more details. Are there a few more settings that one could use for evaluating model performance? The set of tasks seems rather small.

Please see the additional task evaluation section of our global response. Specifically, we present additional ICL task evaluation to learn algorithmic solutions to problems in context. The solution thus requires some algorithmic reasoning. For the ability to learn from few-shots inline, we emphasize that our ICL task evaluation setup examines exactly this ability. Model learns to perform novel tasks based on few-shot examples we provide in-context.

As for detailed instruction following, since all six models we examine are pre-trained models without instruction-tuning, we feel like this may be out of scope for our current work.

Are all In Context Learning tasks equally difficult? Could a few more gradations be helpful here?

In Appendix E, we varied the difficulty of in-context learning tasks by changing the input dimensionality, and observed that ICL tasks nevertheless remain highly resilient to pruning. This observation corroborates our main conclusion.

I'm assuming that pruning is primarily used to increase inference speed, right? If that's the case, I'd like to see tradeoffs between accuracy and inference speed be presented here.

It is true that the ultimate goal of pruning is to increase inference speed. However, pruning studies usually appear before the relevant hardware support is available. For example, seminal work done by Han et al. showed the first promising results of applying pruning techniques to deep neural network models in 2015. However it is not until 2020 that nVidia rolled out the first GPU with hardware support for sparsity. Our work focuses on unstructured sparsity, which is not supported natively by all current generations of GPUs and TPUs; nonetheless, we believe our results will facilitate the real-world adoption of pruning techniques like SparseGPT and Wanda.

Thank you for your helpful feedback!

The study focuses on evaluating two pruning algorithms that are unstructured pruning, evaluation on structured pruning methods are expected. How the structured pruning methods, e.g., LLM-Pruner, performs on these two LLM capabilities?

We are working on providing some structured pruning results. We will get back to you once results become available.

The study could include other types of tasks, like NLI, classification, summarization, to make the study more solid.

Please see additional task evaluation in our global rebuttal response. We present 4 additional task evaluations including lambada, translation and additional ICL classification tasks.

[1] SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot, Frantar et al., ICML 2023.

[2] A Simple and Effective Pruning Approach for Large Language Models, Sun et al., arXiv 2023.

[3] LLM-Pruner: On the Structural Pruning of Large Language Models, Ma et al., NeurIPS 2023.

[4] LLaMA: Open and Efficient Foundation Language Models, Touvron et al., arXiv 2023.

[5] Pythia eval code: https://github.com/EleutherAI/lm-evaluation-harness/blob/master/lm_eval/tasks/triviaqa.py https://raw.githubusercontent.com/p-lambda/incontext-learning/main/data/GINC_trans0.1_start10.0_nsymbols50_nvalues10_nslots10_vic0.9_nhmms10/id_prompts_randomsample_3.json

[6] A Survey on In-context Learning, Dong et al., arXiv 2022.

[7] OpenICL: An Open-Source Framework for In-context Learning, Wu et al., arXiv 2023.

[8] Entity-Based Knowledge Conflicts in Question Answering, Longpre et al., ACL 2021.

[9] DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering, Neeman et al., arXiv 2022.

[10] The LAMBADA dataset: Word prediction requiring a broad discourse context., Paperno et al., ACL 2016.