How Do Large Language Monkeys Get Their Power (Laws)?

Thank you for your positive and thorough review of our work. We appreciate your thoughtful assessment of our theoretical and numerical analyses, as well as your recognition of our contribution in applying the mathematical insight about power laws emerging from weighted sums of exponential functions to this specific domain. We will certainly fix the identified typo in Line 413, changing "contributes is a new hypothesis" to "contributes a new hypothesis."

We are committed to making this paper as strong as possible and would value your guidance on what specific improvements would strengthen the manuscript further. If you have any additional suggestions that would elevate your assessment to a 'Strong Accept,' we would be grateful for that feedback and would make every effort to address those points in our revision.

Thank you again for your constructive engagement with our work.

We appreciate your constructive feedback. We address your points below.

Improvements to Related Work

Could they provide a parametric form for that distribution? Polo et al. (2024) likely also deserve more detailed discussion due to the pass@k experiments they describe in Section 4.5.

We will incorporate a more thorough discussion of Polo et al. (2024)'s Sloth and the related papers Ruan et al. (2024)'s Observational Scaling Laws, and Owen (2024)'s predictability analysis. Their latent variable regression models offer complementary perspectives to our approach. Interestingly, Polo's Section 4.5 pass@k experiments show relatively poor fits (their Figure 7), possibly because (as best as we can tell) they use the biased estimator of $\operatorname{pass_i@k}$ that Chen et al. (2021) caution against. That said, while Polo et al. (2024) don’t define a functional form for scaling, our mathematical analysis could be combined with their estimated per-problem single-attempt success probabilities $\operatorname{pass_i@1}$. It’s possible that their cross-benchmark fitting method gives better estimates of these per-problem single-attempt success probabilities, which would improve our method.

We will expand on this connection and highlight how per-problem analyses could potentially combine with such cross-benchmark approaches to further improve predictability.

What I expect to find in a related work section is insight about how these works relate to the paper. Here specifically, I am missing a discussion about prior work attempting to explain the origin of scaling laws currently listed in lines 390 to 395 - do any of these models predict a power law tail for the distribution of difficulties of individual test data?

We will revise the related work section to discuss contributions from key works beyond just listing scaling law analyses. To the best of our knowledge, no prior work has specifically attempted to explain the power law emergence with repeat sampling in the manner we propose, perhaps due to the recency of works like Brown et al. (2024) and Hughes et al. (2024).

Quantification of Measurement Error

This is an excellent suggestion. For Large Language Monkeys, each problem had 10,000 attempts sampled, making the per-problem single-attempt success rate $\operatorname{pass_i@1}$ a Bernoulli estimator with well-understood standard error. The Best-of-N jailbreak case is more nuanced due to varying sample sizes across problems (as detailed in Appendix A).

We will add this measurement precision information to the main text and try to develop an appropriate visualization to represent the uncertainty in Figure 4.

Optimality of Discretized ML Estimator

Is the discretize+ML described in lines 370-379 optimal in any sense? More specifically, is it the true maximum likelihood estimator of the pass@1 distribution parameters given the observations?

Regarding whether our discretize+ML approach is optimal: we cannot make such a strong claim. While our empirical results demonstrate its effectiveness for the specific task of power law exponent estimation, a formal proof of optimality would require additional theoretical analysis.

The approach may be particularly well-suited for estimating parameters that best describe the distribution's left tail, which is crucial for our application. However, as you correctly suggest, this is a potentially complex statistical estimation question that warrants dedicated investigation. We will clarify these limitations in our revision and position this as an opportunity for future research.

Invitation for Ways to Improve

We believe addressing these points will substantially improve the paper's clarity and impact.

We are committed to making this paper as strong as possible and would value your guidance on what specific improvements would strengthen the manuscript further. If you have any additional suggestions that would elevate your assessment to a 'Strong Accept,' we would be grateful for that feedback and would make every effort to address those points in our revision.

Thank you for your thoughtful review. We appreciate your recognition of our work's strengths, particularly that our paper "presents a novel theoretical framework explaining why per-problem success rates follow exponential decay, while aggregate success rates exhibit power law behavior" and that our distributional estimator "significantly improves compute efficiency, making it a practical contribution for LLM evaluation."

Models and Tasks Are Diverse

However, the study is somewhat limited in scope, as the sample sizes for both benchmarks (128 math problems, 159 jailbreaking prompts) may not fully capture model behavior across diverse tasks.

We believe our study offers substantial diversity.

1. Our analysis spans leading frontier models from four major AI companies (OpenAI, Google, Anthropic, Meta), open-parameter models ranging from 17M to 12B parameters, and fundamentally different tasks (mathematical problem solving and multimodal jailbreaking). This diversity strengthens our confidence in the generalizability of our findings.

2. We agree that verification across an even wider range of models and tasks would further strengthen generalizability. We relied on existing datasets from Brown et al. (2024) and Hughes et al. (2024) because generating 10,000+ attempts per model per problem involves substantial computational costs. For perspective, an experiment with 10 models across 5 benchmarks with 100 problems each would require 50 million sampled outputs.

3. From a statistical perspective, sample sizes matter in order to make precise statistical statements, e.g., determining confidence intervals. If you feel like 128 and 159 problems with >=10k samples per problem are inadequate for specific claims, could you please tell us which claims you find inadequately justified so we can better assess?

Real-World Applications

The proposed estimator for power law exponents is validated on synthetic data and limited benchmarks, but its performance on real-world applications (e.g., machine translation, conversational AI) remains uncertain.

This is a valid concern. While our current validation on both synthetic and real-world data demonstrates the estimator's effectiveness, we acknowledge the need for broader validation across diverse applications. Our method's foundations in statistical theory provide confidence in its generalizability, but we agree that testing across additional domains would be valuable. We view this as an important direction for future work and are exploring partnerships to apply our estimator to machine translation, conversational AI, and other practical applications.

Deviations from Power Law Scaling

The study notes that some models (e.g., LLaMA 3 8B IT) do not follow power law scaling. Do you have hypotheses on why

Our theoretical framework provides a clear explanation: power law scaling emerges only when the distribution of single-attempt success probabilities has a heavy left tail, which Llama 3 8B IT lacks when tested on jailbreaking (as shown in Figure 4).

What this means practically is that Llama 3 8B IT has lower robustness against adversarial attacks than the other models Hughes et al. 2024 tested. This could be attributable to many reasons. This could stem from several factors, including its smaller size, potentially less extensive safety training, or absence of defense mechanisms likely present in API-based models like GPT, Claude, and Gemini. Unfortunately, the proprietary nature of these other models limits our ability to investigate these hypotheses further.

Deeper Theoretical Analysis of Why Heavy Left Tails Appear

While the authors demonstrate empirically that single-attempt success rates follow a heavy-tailed distribution, they do not provide a deeper theoretical justification for why this occurs in practice. They speculate that benchmark design and selection bias may contribute, but these points are not rigorously analyzed.

The best answer we can think of is that power law scaling emerges in a "Goldilocks zone" of problem difficulty. For heavy left-tails to appear, we need problems that are challenging but not impossible—difficult enough to require many attempts yet still solvable. This explains why we wouldn't observe power law scaling when applying state-of-the-art models like GPT-4.5 or Claude 3.7 Sonnet to relatively simple benchmarks like GLUE (too easy), nor when applying these same models to extremely difficult tasks like Millennium Prize problems (effectively impossible). The power law phenomenon manifests precisely in this intermediate difficulty range.

It is not clear to us what a more compelling or more rigorous investigation would look like. If you have suggestions, we would greatly appreciate them!

Thank you again for your insightful feedback, which will help strengthen both this work and our future research directions.

Thank you for your thorough and thoughtful review of our work. We will correct the typo you identified in line 255, changing "Kuamraswamy" to "Kumaraswamy." We address other points below:

Origins of Heavy Left Tailed Distributions

The brief discussion of benchmark design and selection bias could be expanded.

Your explanation focuses on the statistical properties of problem distributions that lead to power laws. Could you elaborate on potential causal factors that might create these heavy-tailed distributions in natural language tasks?

If our paper is accepted, we will use the additional page in our camera-ready version to expand on benchmark design and selection bias as factors leading to heavy-tailed distributions. Your question about causal factors creating these distributions touches on an important insight: power law scaling emerges in a "Goldilocks zone" of problem difficulty. For heavy left-tails to appear, we need problems that are challenging but not impossible—difficult enough to require many attempts yet still solvable. This explains why we wouldn't observe power law scaling when applying state-of-the-art models like GPT-4.5 or Claude 3.7 Sonnet to relatively simple benchmarks like GLUE (too easy), nor when applying these same models to extremely difficult tasks like Millennium Prize problems (effectively impossible). The power law phenomenon manifests precisely in this intermediate difficulty range.

If you can think of a more rigorous way to investigate causal factors, we would welcome your suggestions!

The empirical analyses are limited to specific model families and benchmarks. Verification across a broader range of models and tasks would strengthen generalizability.

We agree that verification across a wider range of models and tasks would strengthen generalizability. We relied on existing datasets from Brown et al. (2024) and Hughes et al. (2024) because generating 10,000+ attempts per model per problem involves substantial computational costs, e.g., drawing 10k attempts from 10 models across 5 benchmarks with 100 problems each would require 50 million samples. While computational constraints limited the scope of our current study, we view this as an important direction for future work and are exploring more efficient experimental designs to validate our theoretical framework more broadly.

Dark Matter of Neural Scaling Laws

Regarding your question about the "dark matter" of neural scaling laws, this is indeed the focus of our ongoing follow-up work! The experimental approach involves training numerous small models on scaling ladders and running scaling predictions in reverse to identify deviations from expected power law functional fits. This allows us to fit more complex functional forms and better understand deviations. We're particularly excited about this direction because experimenting with small models enables cheaper and faster iteration, but is currently not useful because extremely small models are poorly predictive of massive models. If we can figure out the appropriate scaling corrections, this would accelerate experimentation with larger models.

Thank you again for your insightful comments and strong support for our work.