We are thankful for your time and helpful feedback. We were glad to hear that you liked the hypotheses proposed!

The biggest limitation is that the paper is mostly descriptive. It doesn’t test its main hypotheses.

We agree this would be beneficial. We did not explore more hypotheses in detail earlier because we thought the primary impact of this paper to be its relevance to chain-of-thought monitoring, for which a demonstration of an increasingly salient problem seemed useful even if we couldn’t explain its mechanism with sufficient rigor. Chain of thought monitoring is increasingly prominent in frontier AI safety work (e.g. DeepMind and OpenAI both consider it to be a major part of their safety agenda), so we consider it especially salient right now. Given more time since the submission, we have been working on more experiments now for understanding the mechanism here.

To specify our current understanding, we currently think that this is a consequence of token importance in chains-of-thought being very heavy-tailed. There is some recent work supporting this: for example, a paper showing that the tokens with highest entropy in a CoT are responsible for most of the gains from RL such that masking gradient updates to all but those tokens often outperforms models trained in the standard way, and mechanistic interpretability work showing a small pool of sentences receiving disproportionate attention from future sentences. This is related to the vestigial reasoning hypothesis we mention in the paper, and we are working on analyzing entropy distributions in legible and illegible CoTs.

The term “illegibility” is used a lot but is never clearly defined. Is it syntactic noise? Semantic incoherence? Non-English tokens? A more intuitive or formal breakdown would help.

We intend the term to be used pragmatically, in reference to how easy it is for a monitor to understand the contents of the text. This could take the form of syntactic noise, semantic incoherence, language mixing, and more. We will add a clearer definition in the final version of the paper.

The meaning of each score on the 1–9 scale is never fully explained. What makes a 3 vs. a 5? Are there scoring rules or rubrics? This makes the interpretability of their figures difficult.

We few-shot prompt the autograder with examples of human-graded text, along with a description of how to judge the text. The full prompt is in the code in the supplementary material, but we will add it to the appendix of the paper. There are examples of text scoring 3 and 5 in Table 1.

While prompts are included in the supplementary code, it would help to show one or two in the paper for clarity and reproducibility.

We fully agree; as mentioned we will add this as an appendix to the final version of the paper.

The model comparison would be more robust if an open model (e.g., LLaMA or Mistral) were included.

We will add results from more models to the paper, as detailed in response to reviewer Y9nr above.

The correlation between illegibility and correctness is interesting but not necessarily meaningful—is illegibility actually helping reasoning? Or is it just correlated because of some third factor like model uncertainty or longer outputs?

We agree this is hard to rigorously establish. We plan to add some new analysis based on what the upstream factor could be (token importance being heavy-tailed), as mentioned above. That said, the main claim is primarily that outcome-based RL can lead to more illegibility, not necessarily that it’s a straightforward causal relationship. The implications for AI safety and monitoring are similar in both cases, though we agree plans for mitigation would look quite different.

The authors mention that Claude 3.7 exhibits more legible CoTs compared to DeepSeek R1, despite also being trained with outcome-based RL. Can you please elaborate on what differentiates Claude’s training or architecture that might explain this behavior?

We’re unsure. Claude’s model card doesn’t reveal training details like that. The paper has some speculation on potential interventions that could have caused this, based on other work Anthropic has done on preserving reasoning oversight (e.g. paraphrasing and re-training).

The 1–9 legibility scoring system is central to your evaluation, but its criteria remain somewhat vague. Could you provide a clearer definition or rubric that distinguishes between scores?

Our current strategy is primarily few-shot prompting with human-graded text, along with a general guide for rating text based on how legible it appears to the model. We considered a more precise rubric, but didn’t want to bias the models too much toward one kind of illegibility—for example, at the start the focus was primarily on language switching since that was mentioned prominently in the R1 paper, but there are many instances of illegible text without language switching. We agree that this should be clearer, and will update the paper with more details about the prompting setup (including an appendix with the prompts themselves).

Since temperature affects generation randomness, it would be helpful to see a more systematic analysis of how varying temperature impacts CoT legibility and correctness. This is critical for understanding robustness.

We agree that this would be useful. Our testing shows that most temperatures below 0.9 result in behavior similar to temperature 0 for R1, but not for R1-Zero. This could suggest that the primary effect from the added legibility training for R1 was on lower temperatures. We will update the paper with these results.

Have you considered removing illegible segments from CoTs and observing whether the final answer remains unchanged? This could help determine whether these segments are actually contributing to reasoning or are merely artifacts.

This was an experiment we were very excited to test, but no API provider seems to support this kind of pre-filling with a forced answer. In our tests, the model would ignore such a pre-fill and generate a new one from scratch. We couldn’t test a locally hosted version of the model because of compute constraints.

The authors mentions some disagreement between Claude and human annotators in legibility grading. A more detailed analysis of these discrepancies (e.g., inter-rater reliability or specific examples) would strengthen the case for relying on model-based grading.

Late into writing this paper, we noticed that our autograder wasn’t well-designed to pick up on whether models were deciding on their final answer within the legible portion of the CoT. Owing to time constraints in setting up and testing a better autograder, we instead had human contractors judge a random 5% of samples, and added it into the paper at the last minute.

We agree that this is poorly explained. We also think that the number of samples tested were too low, so we will improve the autograder and update the paper with revised results.

Thank you again for taking the time to review the paper and providing helpful feedback! Do the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?

We thank you for your thoughtful feedback.

The evaluation is considerably limited. Using only the GPQA diamond dataset, which contains just 198 questions, is insufficient to draw persuasive generalizations about model behavior.

We will add results from more datasets to the paper, as detailed in response to reviewer Y9nr above.

The paper restricts its analysis to only two families of models (DeepSeek and Claude). A more comprehensive evaluation across a broader set of reasoning models, such as QwQ and Qwen3, and DeepSeek distilled models, would strengthen the study and enhance generalizability.

We will add results from more models to the paper, as detailed in response to reviewer Y9nr above.

While the paper emphasizes evaluating the impact of temperature, it considers only two extreme temperature values (0 and 1), neglecting other practically relevant settings. For example, DeepSeek-R1’s default generation temperature is 0.6 [1], which is notably absent from the study.

We studied various intermediate temperature settings, but found the results to be very similar to the setting with temperature 0. We will include this in the final version of the paper (along with other generation settings such as top_p, which we also found variance on).

We note though that the DeepSeek API docs for R1 recommend using a temperature of 1 by default.

The analysis provided is largely shallow and observational, lacking systematic, hypothesis-driven experimentation. Given the expectations for top venues such as NeurIPS, it would be beneficial to clearly formulate hypotheses and conduct controlled experiments to rigorously test and validate them.

We agree this would be beneficial. We did not explore more hypotheses in detail earlier because we thought the primary impact of this paper to be its relevance to chain-of-thought monitoring, for which a demonstration of an increasingly salient problem seemed useful even if we couldn’t explain its mechanism with sufficient rigor. Chain of thought monitoring is increasingly prominent in frontier AI safety work (e.g. DeepMind and OpenAI both consider it to be a major part of their safety agenda), so we consider it especially salient right now. Given more time since the submission, we have been working on more experiments now for understanding the mechanism here.

To specify our current understanding, we currently think that this is a consequence of token importance in chains-of-thought being very heavy-tailed. There is some recent work supporting this: for example, a paper showing that the tokens with highest entropy in a CoT are responsible for most of the gains from RL such that masking gradient updates to all but those tokens often outperforms models trained in the standard way, and mechanistic interpretability work showing a small pool of sentences receiving disproportionate attention from future sentences. This is related to the vestigial reasoning hypothesis we mention in the paper, and we are working on analyzing entropy distributions in legible and illegible CoTs.

Experimental details are inadequately documented. Crucial information such as the exact prompts used for the Claude-3.5 judging process is not provided, limiting reproducibility and clarity.

The prompts and code for replicating the results are provided in the supplementary material. We agree that it would be much better to include the prompts as an appendix in the paper, and will add that to the final version of the paper.

In Figure 3, why was temperature = 0 (greedy decoding, although with potential minor variations in practice due to API implementation and hardware-specific issues) chosen, along with multiple generations, to determine question hardness?

Setting the temperature to 0 was primarily just to account for variance in model response. “Multiple generations” was poor phrasing—it’s meant to indicate that the question responses were sampled separately from R1 at temperature 0. We will edit that to be clearer.

We thank you again for taking the time to review the paper and providing helpful feedback! Do the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?

Thank you for the thoughtful feedback, especially the detailed comments! We’d like to address the weaknesses and questions you brought up.

The use of a limited data-set, one that is also particularly difficult in its use of scientific jargon, makes it hard to generalize the conclusions

We plan to add results from more datasets to the paper, as detailed in response to reviewer Y9nr above.

The authors used a limited set of reasoning models, and even for those, 2 of the 4 didn't follow the conclusions they draw

We are testing more reasoning models, such as the Qwen and R1-distill models, and will add those results to the paper.

However, we note that DeepSeek-V3, one of the models tested in the paper, is not a reasoning model, and was included to compare against R1 and R1-Zero, models trained with RL from V3. Claude 3.7 Sonnet does not have illegible CoTs, but we don’t believe this contradicts our conclusions—we mention that it suggests that what we observe in R1 and R1-Zero is not a necessary consequence of RL, but at most a likely consequence. New results from other reasoning models (detailed in our response to reviewer Y9nr above) suggest this to be the case as well. Our conclusions are that RL poses risk to monitoring ability in some cases, not that it always does so.

There is a lack of causal mechanism or clear explanation for why and when we observe this illegibility. [...] I mostly just agree with the authors' own assessment.

We agree that this is a very valuable addition; for more information, refer to our response to reviewer Cvn4 below.

"we study whether outcome-based RL causes reasoning models to do meaningful reasoning in illegible CoT" -- this is a weird sentence, and it is unclear what you mean. Especially as this is THE most important sentence in the opening ("we study X"). What is 'meaningful' reasoning? How do you do reasoning "IN" CoT? Based on your conclusions, you also don't study that.

By ‘meaningful’ reasoning, we meant whether a model is using an illegible CoT in forming its answer. For example, if the model were to output a series of random words after deciding on its final answer verbally, we would consider those words to not have been used for deciding the answer. We tried to study this by looking at whether models decide on their answer before or after illegible tokens arise, and whether they’re more likely to get correct answers with illegible CoTs, but we agree this is correlational and not causal evidence. We will edit this sentence (and other mentions of this clause) to be clearer.

Claude 3.5 as the auto-grader: I think the authors have a reasonable justification for this, but it would still be helpful to see what the actual prompting for Claude here was. [...] I don't think any of this will change the conclusions, but still.

Prompts were included in the supplementary code, but we agree it would have been a better choice to include examples in an appendix to the paper itself.

We did take mean(sum(score(Claude) - score(GPT-4o))) (our comparison model was GPT-4o and not o1), which we agree is limiting. We get a Cohen’s weighted kappa of 0.782, which we will edit the paper to mention instead.

"we find a positive correlation between CoT illegibility and accuracy of model answers." -- that's not technically true. You have a binary split between correct and incorrect answers, and your statistical test (not detailed, see below) finds that the legibility for these two partitions is statistically different. That is not the same as a correlation of illegibility and accuracy.

We agree that this was imprecise phrasing. We will edit it to say that correct answers are associated with more illegible CoTs.

"These results suggest that illegible reasoning is likely a consequence of outcome-based RL" -- why does it suggest that, just based on this graph? And what is each dot representing here?

This is suggested because of the difference between the scores of V3 and R1 / R1-Zero. V3 was the base model that the latter two models were trained from using outcome-based RL (and in R1-Zero’s case, only with outcome-based RL and no supervised fine-tuning). Since V3 has functionally no illegible outputs, this suggests that the illegible reasoning in R1 and R1-Zero arose as a consequence of outcome-based RL. We will update the description to be clearer about this.

The dots represent outlier scores in the distribution. We will update the description to mention this as well.

Figure 6: ""We then plot the distribution of these scores [...] a specific claim like that requires an actual test to back it up.

We agree that a quantitative metric here would be useful. When computing Spearman’s rank correlation, we get a mean correlation of 0.54 (p <0.0001), and a median correlation of 0.74. We will update the paper to mention this instead.

"This suggests that lengthy CoTs may be detrimental to CoT monitoring, especially as medium legibility scores are more clear condensed reasoning and replicate across models, before progressing to stronger illegibility." -- minor issue, but the middle part of that sentence makes no sense.

This was a typo, and will be edited. The intended point was that CoTs with medium illegibility scores have stylistic patterns that occur in many other reasoning models (such as o1, or more recently Grok 4), and the fact that we see a continuous shift from fully legible text to this kind of moderately compressed text to fully illegible text might imply that other frontier models may end up with fully illegible text.

"This stark contrast with DeepSeek and OpenAI models [...] possible they DID encounter this (that is, that it is an inevitable consequence) but found additional ways around it once it cropped up, no?

This was also a typo and will be edited, we did mean Anthropic models.

We agree it’s possible they did encounter it and managed to patch it. We mention some prior work by Anthropic (such as paraphrasing of CoTs) that could explain this behavior in another section of the paper.

"For CoT legibility evaluation, we think it’s useful to be pretty lenient — in realistic monitoring setups, we will likely assist the monitor" -- This whole sentence is odd. Lenient in what sense? Who is the 'we' that will be assisting the monitor? Assisting them how?

That sentence was intended to explain why we didn’t follow the method in Kirchner et al which uses a time-constrained human contractor for measuring legibility, to simulate a realistic situation with a limited human auditing budget. In practice for CoT monitoring, we think it’s plausible that having some monitoring overhead (in terms of letting the monitor reason at length) and a lack of general knowledge gaps (since model graders have more broad science knowledge than a human contractor and can better evaluate model reasoning) are reasonable affordances AI labs or external auditors will give monitors.

We will update this sentence to be clearer.

"In our tests, Claude’s responses often differed from human evaluators on this specific question." -- what human evaluators? what tests? The paper is written as though there are no human evaluators, and no further information is provided about this.

" For these cases, we stick with human evaluation on a smaller set of samples (5%) instead of the automated scoring. - which cases? how do you know which cases these are? Who are the humans doing this evaluation, and what is their metric? What is their agreement?

Since submitting the paper, we’ve decided that relying on human evaluation was a poor choice and will be using an upgraded autograder instead. Refer to the response to reviewer sJFd for more details.

"Text scoring 3 or higher often exhibits condensed reasoning, with articles and common words being dropped and sentences being cut short." -- is there an actual analysis to go with this statement? Right now it is extremely subjective and qualitative.

We found that samples in this category often have different ways of exhibiting properties of condensed reasoning, such that more specific metrics than the autograder, like words dropped, often doesn’t meaningfully capture what we want. Table 1 shows one excerpt of text with this score; we are planning on adding an appendix with the autograder’s comments on each chunk from this transcript, which have similar analysis.

" the Claude autograder sometimes becomes confused by the content of such CoTs. -- very inexact, what does it mean 'become confused' and 'sometimes'?

We meant that Claude often describes the text in such CoTs as confusing for it to understand. We will update the description to be clearer about this. Precisely judging the frequency of this is difficult as when asked to explicitly quantify its level of understanding of the text, it often over-claims—e.g. giving high scores even for nonsensical text because the presence of a particular phrase makes it believe it understands the general content.

"We find that R1 and R1-Zero both have significantly more [...] I can't accept the statement or the conclusions that follow from it.

23% and 29% represents the increase in the average score on easy questions to hard questions for R1 and R1-Zero. The p-value for the difference between scores for easy and hard questions is 0.0005 for R1’s reasoning, and <0.0001 for R1-Zero’s reasoning.

We will update the paper to include these numbers to justify the claim.

Figure 4: What is 'point density' meant to be?

Point density is how many samples are close to each other. This was purely for visual clarity since it looked difficult to decipher before. We’ll update the description to better explain this.

Thank you again for the very detailed feedback! It was very useful for improving our paper. Do the above actions address your concerns with the paper? If not, what further clarification or modifications would help to improve it?

We are thankful for your time and help. We were glad to hear that you found the paper relevant and convincing!

The tested models are limited to Deepseek and Claude, which do not represent the state-of-the-art. How about GPT-o1 and GPT-o3?

We agree that it’d be useful to see results from more reasoning models, especially state-of-the-art models such as o1, or o3. Unfortunately, none of the o-series of models provide access to the raw chains of thought; instead, they provide summaries of model reasoning, which likely won’t reveal illegibility and isn’t relevant to our focus. The same applies to the frontier Gemini models.

That said, we acknowledge that testing more models is necessary for understanding how well our results generalize. Since the paper’s submission, we have tested other reasoning models such as the Qwen and R1-distilled models. Preliminary results suggest that QwQ has similar behavior, while the distilled models are noticeably more legible (which would make sense if this is caused by reinforcement learning in particular, since they are solely trained with supervised fine-tuning). We plan to test the new Mistral and Llama-Nemotron reasoning models as well. We will add these results to the final version of the paper.

The focus on a single dataset (GPQA-Diamond) and specific models limits generalizability. It is unclear whether similar results would hold in more naturalistic multi-turn or open-ended reasoning tasks.

We agree with the importance of testing more settings. We have now tested R1 on more datasets, specifically MMLU-Pro and ChemBench. We find a high variance of scores on MMLU-Pro; on further investigation we find that this is due to many questions being easy and requiring very short reasoning traces. For reasoning traces of length comparable to those from GPQA, we see high illegibility scores. On ChemBench, we find higher illegibility scores than GPQA, partly owing to the questions inducing long reasoning.

We will add these results to the paper, as well as results from testing other models on these new datasets. Are there any datasets you would recommend we consider as well? For example, to better test settings such as multi-turn conversations.

Thank you again for taking the time to review the paper and providing helpful feedback! Do the above actions address your concerns with the paper? If not, what further clarification or modifications could we make to improve it?