Predicting Emergent Abilities with Infinite Resolution Evaluation

Thank you very much for recognizing the significance of our research. Given its theoretical and fundamental nature, which diverages from the typical hot research topics of LLM, we are deeply appreciative of those who can understand the importance of predicting task performance where emergence exists.

Weakness 1

Fortunately, the computational cost remains acceptable across model sizes. For large models, we sample a few times, with each generation being slow, and for small language model, we sample a lot of times while each generation is fast. For instance, in our HumanEval experiments, we used a batch size of 200 across 8 GPUs for smaller models, resulting in a processing time of approximately 200 seconds per test example (with a sample number of 10,000). For 0.03b model, the whole test time for human eval is 40 min. For 2.4b model, it is 2h. Both run in parallelization on 8 A100 GPUs.

Weakness 2

We think that as long as we do not define ''pass'' as generate a sequence exactly the same as the ground truth answer, the smoothness of PassUntil won't be very hard for long generations. Suppose we are now predict the performance on AlpacaEval, who answer is known to be long (> 1000 words). We define ''pass'' as the generation from our model is better than GPT-4, judged by GPT-4. Thus, the probability of generating a ``pass'' answer is not exponentially decrease with the length.

Weakness 3

Thank you for highlighting this issue! A significant concern of ours is the common misunderstanding among readers, especially those without access to much pre-training experience, about the difference between 'low perplexity/loss' and 'good task performance.' We are addressing this in our revised paper by adding a detailed discussion on why tracking direct task performance, rather than perplexity/loss, is crucial:

1. PPL (Perplexity) evaluates aspects different from task metrics. It measures the likelihood of an answer being a natural continuation of a sentence, which is distinct from 'answering a question or do one task successfully'
2. PPL cannot accurately inform us of the concrete value of task performance, particularly in practical evaluations.
3. While we use loss on ground-truth of the small model to aid in predicting PassUntil, PPL alone is inadequate for predicting performance. This is because, for each sample, the accuracy is binary (0 or 1), which complicates establishing a linear relationship between model performance and PPL

Question 3

In our experiments, we set the top-p value to 0.95. Your suggestion to improve the smoothness of the PassUntil curve by further adjusting top-p or top-k is valid. A concern, however, is setting the top-k or top-p value too low, potentially causing the correct answer to be permanently excluded from the candidate pool for small models.

Thank you for your insightful review! We believe there may be some misunderstandings that have led to the questions raised.

Weakness 1

Sorry for the confusion, reviewer 5dKH weakness-3 indicates that it's our fault that we do not make the point clear. Let us make the point clearer.

• It's important to distinguish between 'loss is not predictive of task performance' and 'loss can help predict task performance.' The former suggests that loss is a not sufficient statistic for estimating task performance without other measurement, while the latter indicates that loss is one of useful factors in improving prediction accuracy. In our paper, we clearly verify both statements. Without utilizing the PassUntil method, one cannot deduce actual performance (accuracy) solely from loss values. For example, a loss of 1.0 does not directly translate to an accuracy of 0.2 for a task. And actual performance must be empirically measured. Furthermore, as shown in our pre-revision Figure 4, the loss of an individual sample does not have a one-to-one correlation with PassUntil results, much less with discrete accuracy.
• However, loss does provide useful information. Once we measure PassUntil across a large sample set, we can establish a statistical relationship between loss and PassUntil (not possible if we only rely on loss data). This relationship can enhance our prediction accuracy.
• The incorporation of loss for improved predictions is driven by practical considerations, such as limited computational resources, rather than being a necessity. Figure 3 demonstrates that even without loss data, we can accurately predict task performance. Imagine a scenario where we can measure every sample with sufficient resolution to ensure each is passed at least once; in such a case, loss data would not be necessary.

Weakness 2

As the first open-sourced project attempting to predict the scaling curve of task performance, we are not aware of any other existing methods capable of such prediction. If the reviewer is aware of related methods, we would appreciate references to enhance our understanding.

Weakness 3

We have addressed the confusion regarding the Definition of Emergence in [our global response]（https://openreview.net/forum?id=lDbjooxLkD&noteId=qUjpV9mmJv）. Please let us know if there are any further points of confusion.

Question 1

• Our methodology currently applies only to generative tasks where the success rate is zero for random generation. This limitation is outlined in Section 4.1's comments and the Limitations section in the Appendix. However, it appears that most tasks inherently fit this generative model with some pass criteria if the test format is adapted accordingly. We will expand on these limitations in the body of our revised manuscript.
• You mentioned 'Token Edit Distance,' which is predominantly used in multiple-choice tasks. While this metric offers more continuity than accuracy for such tasks, establishing a direct link between 'Token Edit Distance' and accuracy remains challenging. Therefore, we have deferred the exploration of these types of tasks to subsequent work.

First and foremost, we want to express our heartfelt thanks to Reviewer v7YM. You are not just a professional, meticulous, and thoughtful reviewer, but also a sage brimming with insight and patience. Your comments are truly inspiring. Thank you reviewing our paper and we wish you every success and happiness in life. Even though eventually we might not convince you with our following answers, we would like to thank you for improving the quality of our paper.

The updated version will be uploaded after we answer all reviewers' questions.

Weakness 1

• We appreciate your observation regarding the misattribution, and we apologize for any confusion caused. Our emphasis on distinguishing our work from Schaeffer et al. (2023) arose from our initial feedback after share our work with colleagues, where some perceived our approach as another version of ‘smooth metrics’, which led to doubts about the practicality of our method for predicting actual metrics of interest. Our contrast was specifically directed only at this aspect of their work, not their other conclusions, which we believe may have led to misunderstandings among readers.

• We also acknowledge our oversight in not sufficiently recognizing Schaeffer et al.'s discussion on resolution. In our revised paper, we will duly credit their insights on this aspect and state that our approach delves deeper into enhancing resolution based on the observation of Schaeffer et al. In fact, unlike Schaeffer et al., who increase resolution by expanding test data (an impractical approach for real test sets), our method improves resolution without the need for additional test data. This innovation allows for the prediction of more challenging tasks and represents the first open-sourced success to use this approach for predictable scaling.

Weakness 2, 3

Regarding the second and third questions about the PassUntil method, we will address them together:

• We apologize for the confusion caused by our erroneous use of the term 'unbiased' estimate. During our method's design, we were aware of the 'biased' statement on Wikipedia. Our initial solution was to move the expectation to the denominator. With this remedy, the math itself is correct: $E[K] - r = E[f] = r(1-P(s))/P(s) = r/P(s)$, thus $P(s) = r/E[K]$. Practically, if we repeat experiments, we first take expection of $K$, then get $P(s)$. However, upon further reflection, we realize that this make $r/E[K]$ as a constant, rather than a random variable for estimating $P(s)$, makes 'unbiased estimate' an inappropriate term.
• Regarding the MVUE estimate you proposed, we find it theoretically sound but encountered an issue when setting $r$ to 1, resulting in an MVUE of $0$ for $P(s)$ as calculated by $(r−1)/(K−1)$. We seek your guidance for a suitable MVUE when r=1. (Negative Binomial Distribution itself doesn’t require $r>1$). Once that is solved, we would be happy to migrate the whole method to (r-1)/(K-1).
• Our current solution for the statement in the paper involves shifting from 'unbiased estimate' to 'maximum likelihood estimate' (MLE). The MLE, particularly $r/K$, suffices for most scenarios, including ours.
• Notably, fixing the sample number $K$ to a large constant value and make $r$ a random variable also renders $r/K$ an effective MLE. This might partly resolves your concern about the case of PU score when computation limit exists. i.e., if we set all runs to sample for a fixed times $K$, and observe the number of success $r$, the MLE we get is still $r/K$. I wonder if this explanation might be satisfying to you.

Weakness 4

• Please see the changes in updated version.

Weakness 5

• In our equation $c$ is set to $c = \sum_i c_i$ (we make that clear in our revision). So the equation is not incorrect. We add an acknowledge of Schaeffer et al.’s contribution besides the original referenced OpenAI citation.

Weakness 6

• Please see the changes in updated version.

[To be continued]

Question 1

• We do need to train our own model, since the prior goal of this work is to make precise prediction on the actual performance of our trained model. For Pythia and OPT, they do not faithfully obey the loss-scaling law. For example, you can see the loss visualization of OPT checkpoints in this paper ’s Figure 1. Clearly they do not obey loss scaling law. Therefore, we only use them for the qualitative study in Table 1 (before revision). For llama and mistral, they only open opensourced less or equal to four checkpoints of different sizes, which is not enough for fitting a scaling curve for making predictions about the largest size checkpoint. And their fit to scaling law is not satisfying as well. Using their checkpoints, we can make valuable observation, but we can not make predictions. We verify our models follow loss scaling law strictly in our Figure 2, which provides a solid foundation to task performance prediction. We will open-source all our checkpoints to facilitate prediction scaling in the community.

Question 2

It is not contradict to our theoreom 2. The misunderstanding is indeed caused by our attempt to overwrite an exstablished concept. Multiple step reasoning, or complex skills for solving a task, will make the task scaling law's slope steeper in the log space, which results in a steeper ''exponential form'' in the task performance. However, our defined ''accelerated emergence'' (originally named as emergence), is not about the speed, but about the acceleration. The acceleration is only distinguishable after the mathematical form has been established. And we actually means that ''multistep reasoning'' does not lead to super-scaling law behavior, a.k.a., accelerated emergence. We will make it cristal clear in the revision.

To sum up, we owe many thanks to Reviewer v7YM, who has made concrete contribution to the quality improvment of our work. His/Her professional knowledge, commitment, carefulness, and patience deeply touches us, making us believe ICLR's review quality is at the apex of AI conference. We will add Reviewer v7YM to our acknowledgements despite that we don't have the reviewer's name.

Weakness 7

We do not cherry-pick the results. As this research represents an initial attempt into predictable scaling, we selected tasks based on specific criteria for simplicity:

1. The tasks are not of a multiple-choice grade. This exclusion eliminated 153 tasks. Our investigation into the predictable scaling of multiple-choice tasks is under-progress for a future publication.
2. The answers are longer than a single word, such as ASCII Word Recognition, as they do not align with our sampling procedure.
3. The tasks are solvable by our largest model, the 2.4B, under natural metrics (for the sake of verification of prediction). This criterion ruled out tasks like chess_state_tracking, hindi_question_answering etc.

Based on these criteria, we identified the 'Unnatural In-Context Learning' task from BigBench as most suitable for our current stage of research. As for the task has 8 tasks, we will add the rest 4 subtasks in our revision.

It's important to clarify that our primary goal is on making precise predictions based on our scaling experiments. Hence, the breadth of task variety is not our immediate priority. However, we are committed to continuing this line of research and aim to achieve predictable scaling across a broader range of tasks in future studies.

Weakness 8

Yes, we highly agree. After we have submitted our paper, we also thought that the word ''emergent'' has been used to describe the sudden rise in performance and it's not very appropriate to re-define it and causing misunderstandings. We consider to name the phenominon of ''super-scaling law behavior'' as ''accelerated emergence'' in the revised version. With that definition, emergence can still be empirically understood as ''exponential-like growth'' in task performance. And among which, a part of emergent task still obeys the task scaling law. While we are discovering the new case of emergence that grows faster than ``scaling law-induced emergence''.

Weakness 9

We respectfully question your assertion that subscaling behavior does not resemble multi-step success task curves. Prior to our mathematical categorization, the performance of all emergent tasks exhibited characteristics akin to 'exponential growth', especially when the slope in logarithmic space was significant. They are not easy to distinguish unless you view it from a log space and with mathematical characterization. Additionally, we observed that LLM performance does indeed decrease geometrically. This observation aligns with Schaeffer et al.'s statement `... as the target string length increases, ... geometrically ... ''. And this geometrical growth emerges when each step is governed by a consistent scaling law coefficient, resulting in an expression like $\exp(|y|N^\alpha) = \exp(N^\alpha)^{|y|}$. We wonder whether the reviewer could back the interesting observation with some reference? Such insights could be invaluable for our further investigation.

Weakness 10

• Yes, it’s an assumption, we will make this clear as you suggested.

Weakness 11

• Actually, the trade-off between actual sampling numbers and time cost for each sample is interesting. For small models, they need more sampling iterations, but each sample is super fast. For large models, their generation is slow, but they success within a few samples. For 0.03b model, the whole test time for HumanEval is 40 min. For 2.4b model, it is 2h. All evaluations run parallelly on 8 A100 GPUs.

Limitation 1

We have mentioned it in limitation section. We will make it clearer in revision.

Title

We buy it! Thanks a lot for the excellent title!

[To be continued]

Question 1

• A1). The example provided does not account for the discrepancies noted. In our methodology, we estimate the probability of generating the correct answer (otherwise, why should we predict an answer’s probability which contains error itself?). Since the answer is correct, we assume it to be a natural continuation consistent with the training distribution. Therefore, each token is conditioned on the preceding correct (or natural) token, without any internal errors. This allows us to apply the scaling law to each token to calculate the total probability of the answer.
• A2). While scaling laws are typically applied in an average sense, this does not preclude their adaptation to individual token levels. Theoretically, a scaling law implies linearity in the logarithmic space of model parameters/compute and the logarithmic space of loss. If single-token losses were not linear in this log space (There must be more random noise at instance-level, though), it would be challenging to understand the observed linearity on average. This is because numerous non-linear structures would dominate the scaling process, leading to non-linearity in aggregate. To provide a more rigorous analysis, prompted by your question, we are examining the scaling relationship of single-token loss. We do linear regression log(loss) ~ log(modelsize) on each token of HumanEval's ground-truth solution. And we count the R^2's distribution. which is

So, at token level, randomness indeed dominate the loss, but there are still a considerable tokens obeying scaling law. Therefore, we think the assumption is reasonable at the deduction stage. A careful dive into the R^2 interval provide us the observationn that R^2=0-0.2 contains the tokens that are either too easy or too hard to predict.

Question2

Our choice of evaluation tasks can be attributed to two primary considerations:

• 1. Our method is currently designed for tasks where a random guess would typically result in a zero score, particularly in smaller models. This limitation is detailed in Section 4.1, Comment 2. However this does not compromise the general applicability of our approach. Generative tasks are inherently versatile, making our method suitable for a variety of applications. Moreover, other metrics could be build upon our method. For example, a possible solution for multiple choice grade metric is to track each choice’s PassUntil with task scaling law or just track the Brier Scores.
• 2. Another consideration of our task selection from practical perspective is that the task can be evaluated on a simple pass/fail basis. While our method is capable of processing generative tasks that requiring complex human evaluation, these would necessitate the support of an answer-judging model, which will complicate our initial attempt for predictable scaling.

We must point out that observing scaling behavior (what Wei et al.2022a do) is easier than predicting it. Despite that, we are currently expanding our scope of prediction!

Question3

Equation (4) was build upon very strong assumption (each token’s scaling speed is the same, and there is one circuit to get the answer passed in terms of neural circuit), without which we can not achieve any meaningful formulation of the scaling properties of task performance. We can see there are practical tasks (HumanEval, Date Understanding, etc.) do show such scaling properties. However, since the assumption is so strong, we are not surprised to see the cases (in figure 6) that does not obey the assumption. In fact, the whole section 6 is to discuss the more general case of task scaling law. We can see that if each token’s scaling speed is not the same, it must induce a sub-scaling law growth instead of super-scaling law growth. Which is not what we observed in experiments. So we propose another hypothesis based on neural circuits. The methodology we have adopted is not arbitrary or ad-hoc. Instead, it represents a systematic and gradual approach to addressing a complex scientific problem.

Question4

That ``emergent abilities are induced by multi-step reasoning'' is not an assumption made by this work, it is made by Wei et al.2022a in Section 5.1. We are refuting such assumption by formalizing it into a theorem to check it correctness. The non-reasoning tasks in Appendix E of Wei et al. indeed support our view against multi-step reasoning assumption.

Question 5 Temperature affects the performance. However, currently we didn’t take into account the variation of temperature during model scaling, we use the same temperature for the small LLMs as the large LLMs, which is a natural choice considering the current prevalent use of random sampling in LLMs' generation.

Instead, we propose the term 'accelerated emergence' from a quantitative perspective

That sounds very good. Thank you for thinking this through carefully!