Many thanks for taking the time to review our paper in such detail, and raising several points we had not considered. We are pleased to see the value of our contributions has been recognized. Below, we respond to your main questions, followed by your ‘other comments’ and finally several points raised elsewhere in the review.

• Q1. Discrete vs continuous encoders for BC.

Thanks for this comment – we appreciate the reviewer’s insight into trying to reconcile the two plots, which is something we agree is worth understanding in more depth. First, as discussed in Other comment 1. caution should be applied as the two BC losses are in fact subtly different, e.g. one should not say the BC-CNN loss is ~10x higher than BC-Token’s. However, the fact BC-CNN asymptotes quicker, means those models are approaching the natural entropy in the data ($E$ in Eq. 7) at modest compute budgets, while BC-Token does not show signs of this. One hypothesis is that BC-CNN models are therefore more compute efficient, though an alternative hypothesis is that differences in loss modelling cause this. We propose to investigate this further when training the new smaller BC-Token models requested by Reviewer TxDt, by computing the equivalent BC-CNN loss for the new models, to better understand how these two losses align. Again, thanks for the suggestion.

• Q2. Parametric fit distorting results

We assume the reviewer is referring to the BC-Token experiment? We expect that following Reviewer TxDt’s recommendation to train a set of smaller models to encourage saturation, we will be able to provide coefficients for the frontier fit method resolving this issue.

• Q3. Inference speed and model size

We can confirm all BC-Token and BC-CNN models considered in our experiments can be run at >10Hz on a V100 GPU. We do not currently have numbers to hand on how inference speed changes with model sizes.

• Q4. Figure 11 correlation

The computed correlation comes out as 0.61, which is considered a moderate strength relationship. We will add this to the figure caption. The high variance comes since a set of models are trained for each point – one model in the family failing to converge properly affects the coefficient entirely. This was something that occurred more often for models trained on larger tokens-per-image. We believe with further optimizing of training schedule these models could converge more consistently and the pattern would emerge even stronger.

• Q5. Investigating effect of tokenization of actions on coefficients

This is a nice suggestion we had not thought of! Does the action tokenizer effect coefficients in similar ways to image tokenizers? Whilst most action dims are natively discretized (buttons), the natural lever to play with here would be number of bins for the continuous joystick dims. But we could also consider grouping together various button combinations to require less tokens per action prediction. We feel this goes a little beyond the current scope of the paper but would be excited to see investigation of this in follow up work.

• Other comment 1. Figure 1 with differing loss scales

We apologize for not making this clearer. In fact, the loss magnitudes are not directly comparable between BC-Token-540 and BC-CNN. Each model optimizes a subtly different losses; BC-Token-540 predicts action dimension-by-dimension, while BC-CNN produces predictions for each action dimension independently (Section 3.2). Our implementations also aggregate the losses in different ways. We will edit the figure caption to make this clearer.

• Other comment 2. Small figures

Apologies, we will expand figure sizes in the next version.

• Other comment 3. Line 370 unclear

Apologies for this, you were correct in your guess. We will clarify.

• Comment 1. Re-analysis of BC ... unsure about infinite data assumption

To confirm, Tuyls et al. used pre-trained policies so could generate unlimited data and remain in the infinite data regime.

• Comment 2. Character prediction experiments used parametric fit

We agree frontier loss would be preferable. While we could have used frontier fit for the Dense loss, this did not appear possible for spare loss super-classed (Figure 10). We favored consistency in our fitting method across these experiments.

• Comment 3. Conclusions about scaling hard to understand Thanks for this comment. We have focused on presenting nuanced details from the scaling analysis and have so far not offered clear conclusions for practitioners. Would the reviewer agree with the below concrete recommendations?

1. When training BC-Token models, model sizes should be substantially smaller than for BC-CNN.
2. For WM-Token models, increasing tokens-per-image of the tokenizer should be done in parallel with increasing model sizes.
3. As per response to Q1. we may be able to advise a preference for BC-CNN models, but await our analysis contrasting BC-CNN and BC-Token losses.

Thank you for taking the time to review our paper. Please see below our responses.

• Comment 1. Infinite data regime assumption

Thank you for drawing attention to this important detail. To clarify, all experiments in our paper for both domains (video games and robotics) are conducted in the infinite data regime (Section A.3.1 computes the amount of FLOPs that would violate this assumption in our set up for the main experiments).

Investigation into scaling laws outside the infinite regime is an active research area in LLM research (Scaling Data-Constrained Language Models). As we understand, the current thinking is that scaling laws do still exist in this setting, but are influenced in adverse ways by repeated epochs.

We believe the reviewer is correct to call out this as a line of research very relevant to embodied AI, which typically falls within the data-constrained regime. We believe it falls outside the scope of a first investigation into scaling laws in embodied AI, for which we have chosen to focus our investigation on the effect of tokenizer, task and architecture.

• Comment 2. Pre-training loss and downstream performance

We have done our best to frame all claims in the paper as optimizing for loss in the pre-training phase of agents, rather than for downstream performance, as these are the concrete results we have. We have further provided conceptual arguments and experimental evidence on the link between pre-training loss and performance. Note the Nethack and Atari experiments, while being of limited complexity, are not necessarily small-scale (up to 10^18 FLOPs).

Providing further evidence effectively for our dataset, environment and setup, we feel would require resources beyond the scope of this paper. We would require implementing a form of post-training phase to maximize some reward signal, and engineer an automated, distributed solution to enable rollouts in this game at large scale (something Bleeding Edge does not natively do).

In general, we agree with the reviewer that the link between pre-training loss and online performance is important enough to deserve further study, but leave this to a separate investigation.

• Other comments

Thank you for these suggestions. Should we be fortunate enough to have the paper accepted, we will take your suggestion and use the allowed extra page to enlarge figures for clarity. Regarding the paper organization, please confirm whether other reviewers support your proposed changes. We would then be happy to reorder the paper as you suggest in the next version.

Thank you for your review. We are delighted to have successfully communicated the value of our study. We agree that extending principles from LLMs to embodied AI tasks is a timely and important avenue of research. Allow us to respond to your comments below.

• Comment 1. RT-1 dataset has limited diversity and is small scale. How do conclusions apply to tasks where generalization is important, and larger scale datasets.

We recognize that the RT-1 tasks, policies and visuals contain only limited diversity. On the other hand, the Bleeding Edge dataset contains a huge amount of diversity, with a pool of seven maps, tens of thousands of demonstrators of differing skill levels. While these two datasets are from differing domains, we did not observe contradictions between the two sets of experiments. We hope that analyzing the results in combination will give researchers enough evidence that similar patterns would also emerge in a large diverse robotics dataset.

• Comment 2. Effects of tokenizer compression rates and target granularity lack theoretical grounding

Thank you for this comment. This is one of the directions for future work we are also most excited about, and have started very initial experiments in this direction. We sketch our current thinking on this issue in case of interest.

We have begun considering the case of lossless compression, where more-tokens-per observation (lower compression) should make next-token-prediction easier, both in the sense of the pattern being simpler (less parameters required) and less stochastic noise (less data required). We suspect this leads to lower values of both alpha and beta. But the key question is which shrinks faster (since their ratio is the thing that matters). Which might be harder to reason about without further assumptions.

One thing that makes analysis tricky, is that losses may not be consistent across tokenizers even for the lossless case, and in reality, tokenizers become more lossy at higher compression rates.

Regarding this paper submission, we believe that a deeper analysis of compression rates is best left to a separate paper, and perhaps domains more straightforward than embodied AI. Should you be interested in exploring this direction further, please reach out following the review period.

Thank you for your review, we are pleased your judgement of the paper comes out on the side of acceptance and agree that your suggestions would further improve the paper. Due to logistical constraints, we are not able to complete all of these requests, but commit to completing the smaller scales BC-Token experiments (which we agree is an excellent idea), before any camera-ready deadline. Below we respond to your three primary questions, and several further key comments we noted.

• Q1. Additional evidence … to show the correlation between the training loss and game performance

We have done our best to frame all claims in the paper as optimizing for loss in the pre-training phase of agents, rather than for downstream performance, as these are the concrete results we have. We have further provided conceptual arguments and experimental evidence on the link between pre-training loss and performance but agree this should continue to be explored through future work. Doing this effectively for our dataset, environment and setup, we feel we would require resources beyond the scope of this paper. We would require implementing a form of post-training phase to maximize some reward signal, and engineer an automated, distributed solution to enable rollouts in this game at large scale (something Bleeding Edge does not natively do). As such, similar to the LLM reference noted by the reviewer (their [2]), we believe this is best left to a separate paper.

• Q2. Smaller models in the BC-Token experiments

Thank you for this very sensible suggestion! Our original protocol had aimed for consistent model sizes across tasks and tokenizers, but on reflection, we agree that smaller models could be used for the BC-Token experiments. Logistically this will be difficult to complete within the rebuttal period (very small models do not fully utilize GPU FLOPs and hence are very time inefficient), but we are happy to commit to completing this before any camera-ready deadline (should we be fortunate enough to have the paper accepted).

• Q3. Additional extrapolation experiments

We agree with the spirit of this request – the gold standard test for scaling laws is how well they predict orders of magnitude out. As the reviewer mentions, this would require more compute than used in our current paper, which is a hard for us to commit to. We hope our current results, which span around three orders of magnitude, are sufficiently interesting for much of the embodied AI community working which has not yet advanced to the model sizes in LLMs.

• Comment 1. Claims feel stronger than results … since the scaling laws aren’t really general and are influenced by the specific task, tokenizer and architecture

Please let us know if there are specific wording changes you’d recommend. We aimed to write the paper in a way that would engage researchers across embodied AI, whilst avoiding overclaim or hype. Power laws did consistently emerge across the two domains tested, and one of our main contributions is that task, tokenizer and architecture do impact coefficients. This is an important departure from LLM research, where datasets, tokenizers and architectures usually only see minor variations.

• Comment 2. Dataset is proprietary, reproducing these results are substantial, limiting accessibility for researchers with fewer resources

The Bleeding Edge dataset was accessed under a data sharing agreement and unfortunately, we do not have agency to share ourselves. On the other hand, the RT-1 experiments are fully reproducible. This follows recent high-impact embodied AI work (Genie: Generative Interactive Environments) which described a lightweight environment for other researchers to experiment in. Note our LLM experiments are also designed to help researchers capture the essence of challenges we identified in a more accessible modality.