Bayesian scaling laws for in-context learning

[continued from above]

• Q5: What we observed was complete degradation at the end of the context window, i.e. task accuracy suddenly dropping to 0. We think this was an issue with the inference provider or the model’s advertised context window being slightly longer than its true context window, not a clean trend towards the end of the window. We don’t think this behaviour was systematic enough to be fit with our law; we can provide the full ICL curves we observed for all the models in a follow-up message soon.
• Q6: We definitely find the clear practical application to be extrapolating the # of examples needed to achieve some desired (or undesired) task accuracy from a much shorter test window, with clear efficiency gains since attention has quadratic time complexity. From a scientific angle, it seems that our Bayesian law is a better analysis tool for understanding how different post-training approaches or inference-time scaling techniques change ICL behaviour; ideally, we want methods to change task knowledge rather than just the prior, and this tool enables separating those two.
  Your second suggestion seems not doable by our current method since it requires measuring $K$ for each example, whereas we assume fixed $K$ averaged over many ICL curves. However, we find this idea intriguing, and perhaps in the future one could design a Bayesian law that estimates per-example $K$. This could be useful to grade example difficulty (for curriculum learning or RL) or to come up with a short but highly effective set of examples for ICL prompting.

Thanks again for your very useful questions and comments! We will provide the materials mentioned in Q4 and Q5 in a follow-up comment within a day.

Thank you for your extensive review, we appreciate how comprehensive your comments are! We will address your concerns/questions point by point.

Re: reasons to reject:

• W1: See our response to Q1 below.
• W2: We agree; e.g. in section 6 we say “we do not claim to have proven that LLMs are Bayesian learners”. We wanted an interpretable law describing model behaviour, which previously did not exist (logistic/power law fits). However, we don’t necessarily find this a weakness; a behavioural law is highly practically useful since it can be easily fit to the behaviour of new models and does not require access to weights (only logprobs). Mechanistic analysis is complementary to, but not better than, our approach in this paper.
• W3: We did refine our setup to ensure convergence and numerical stability in all laws we tested, not just the Bayesian one, in order to ensure fair comparison. Likely due to the small learned parameter count, Newton methods like L-BFGS are much more stable than e.g. AdamW in our tests. L-BFGS is a standard solver for fitting scaling laws in general, e.g. see section 3.3 of Hoffmann et al. (2022).

Re: questions:

• Q1: The efficiency parameter $K$ is indeed additional to the Bayesian law and makes the fit more expressive. We justify this in section 3.2 (essentially, this parameter enables us to not make assumptions about the frequency of Bayesian updates) and show that it is interpretable in Figure 2b. The efficiency parameter improves the fit substantially. As for parameter-tying, this addition actually makes our law less expressive by reducing the number of learned parameters, i.e. we are not using the full expressivity of Bayes’ theorem. We are therefore matched in parameters with the baseline laws. In fact, without tying, we find much lower interpolation error rates for our law, but it extrapolates much worse and the parameters are not interpretable (indicating overfitting, we believe); we can report these ablation results in an appendix.
• Q2: This is an extremely interesting and relevant question. In our view, outside of this work there are a few pieces of evidence in prior work (listed below) suggesting that the shape of the ICL curve does reveal something about model internals. However, to our knowledge there is no work that clearly and directly establishes a link between the shape of the ICL curve and some mechanistic explanation for ICL. So we do believe our Bayesian scaling law is somehow founded in an underlying mechanism, but in this work we did not do the interpretability work that would be necessary to prove this to be true (and so we do not claim to have found a mechanism).
  • Schaeffer et al. (2025) show a relationship between the shape of the inference-time scaling curve to model performance on individual examples in a dataset and the distribution of question difficulties in that dataset. An analogous relationship might be found for the ICL curve.
  • Anil et al. (2024) which proposed the (bounded) power law as a fit for the ICL curve in appendix I of their paper propose a mechanistic explanation for their law, related to the “function vector” mechanism in ICL which has been reported in some other works.
• Q3: In figures 16-19, we observed that DPO makes the ICL curve non-monotonic which renders it indescribable by all of the scaling laws we consider (which assume ICL accuracy is non-decreasing with more examples). Therefore, in the Bayesian framework it is not clear how to describe what is happening under DPO. Our best interpretation is that DPO makes the model no longer (approximately) a Bayesian learner, and that this is due to its contrastive objective which doesn’t simply match an expected distribution (unlike SFT, which in our experiments will converge to a model that only knows HMM 0) and thus has unpredictable effects on what the model ends up learning.
• Q4: We have not closely examined the non-prior parameters between the base and instruct Llama models. We will look into this and provide the findings in a follow-up message soon.

[continued below]

Q4

There are some interesting differences in the Bayesian fits for the base and instruct Llama 3.1 8B models, besides the prior. For the efficiency parameter $K$ we observe that the base model almost always (except on logiqa) has higher ICL efficiency than the instruct model. This suggests that post-training hurts ICL ability; we weren’t able to find any other work in the literature claiming this.

For the in-distribution probabilities $P_{i, i}$ we find that (except on harmbench) the instruct model consistently has higher values than the base model, indicating ICL converges to a higher task accuracy.

The off-diagonal probabilities (which are tied in our Bayesian law) don’t reveal much of interest; they are very low in all tasks for both models.

We will add this analysis in our appendix. Overall, it seems that post-training affected all parameters of our Bayesian fits; not only is the prior on harmful tasks lower, but overall ICL efficiency is decreased and ICL converges to a higher performance. These trends seem to hold up across tasks.

Q5

Below is the plot of the complete ICL curves. Note how on some datasets the performance suddenly craters at the very end. This is clearly some kind of outlier, so we excluded the end of the ICL curves from our analyses.

Link (if the above doesn't render)

Thanks for the reply, it solved most of my questions. I have changed my rating.

Thank you for your questions and comments! Re: whether this law is useful/holds across architecture in general, you are correct that we did not study non-transformer autoregressive sequence modelling architectures in the paper. We are not sure if alternative architectures (e.g. SSMs, or hybrid SSM-Transformers) exhibit the same ICL curve. We can run our section 5 experiments on one of these pretrained non-Transformer models if time permits (e.g. the mamba/mamba2 models on HuggingFace). Let us know if this would be interesting in your view!

Thank you for your comments!

We want to clarify the aims of this paper: we present Bayesian scaling laws as an analysis tool for understanding the behaviour of language models. Having such a scaling law enables extrapolation of performance from few-shot ICL (e.g. section 4.1), allows for better understanding of model behaviour under post-training (section 4.2 and 4.3), and we showed these benefits hold in real-world LLMs as well (section 5). Our intention is not to “demonstrate improvements on downstream tasks” directly; instead, one could use this tool to e.g. figure out how many ICL examples to include in the context to achieve some desired performance. Additionally, our experiments in section 5 contain ample evidence of the practical applicability of our findings, and largely agree with our synthetic findings.

Are there particular experiments or pieces of evidence that would convince you of the generality of our findings, or other specific shortcomings that we could address? We appreciate your feedback.