Thank you very much for recognizing the Quality, Clarity, Significance, and Originality of our work. Regarding your two questions (Q1-Q2), we address them one by one as follows:

Q1: Regarding the 1x Cost of ST-BoN.

It is evident that the computational cost of ST-BoN cannot be identical to that of greedy decoding. As you noted, it requires at least a brief $N$ sampling.

We would like to clarify this potential misunderstanding. Due to the limited space in Figure 5, the computational costs for $N = 3$ and $N = 5$ appear visually close to the 1x baseline, but actually not the same. The precise computational costs for the four objective tasks are as follows:

We will include these numerical values in the revised appendix to enhance clarity and transparency.

Q2: Regarding the Applicability to Reasoning Models

Thanks for your suggestion regarding the broader applicability of ST-BoN. To evaluate the applicability of ST-BoN on long-CoT reasoning models, we report results on Qwen3-8B-Thinking and Deepseek-R1-Distill-Qwen-7B, using the AIME24+25 dataset. Both the models and dataset are representative of long-CoT paradigm.
(Due to computational resource constraints, Full-BoN was only evaluated up to $N \leq 40$.)

Qwen3-8B-Thinking (Accuracy / Cost):

Deepseek-R1-Distill-Qwen-7B (Accuracy / Cost):

Key insights:

• First, due to the much longer generation lengths of reasoning models (often >10k tokens) compared to non-reasoning tasks (often <1k), the memory cost of KV cache becomes dominant. This significantly inflates the overall cost of Full-BoN. In contrast, ST-BoN performs early truncation and retains only a single path up to ~10k tokens, keeping its cost only marginally higher than that of greedy decoding.

• Second, ST-BoN still exhibits a markedly better cost–performance trade-off. For instance, on Qwen3-8B-Thinking, to reach 73.3 accuracy, ST-BoN requires only 13.8% (3.2/23.2) of the cost of Full-BoN w/ RM. Notably, even when ST-BoN uses $N=160$, Full-BoN at $N=10$ remains more expensive while underperforming.

These results provide strong evidence for the scalability and efficiency of ST-BoN in the context of long-CoT reasoning models.

Further Insight: To explain this phenomenon, we find support from a recent study [1], which shows that the accuracy of the first reasoning step often predicts final success, even when later decoding involves rollback or reflection. This further supports the effectiveness behind ST-BoN’s early truncation design in any LLM paradigm.
[1] Lost at the Beginning of Reasoning. https://arxiv.org/abs/2506.22058

We hope these results satisfactorily address your concerns and underscore the broader significance of our contributions. We look forward to your positive feedback, thanks again for your constructive comments.

Thank you for clarifying my questions and providing additional experimental results, especially those for comparing with Full-BoN on long reasoning tasks.. It is exciting to see such a great accuracy-efficiency trade-off for your proposed method. I will keep my accept rating as this is a solid work.

Thank you very much for recognizing our work. We especially appreciate your acknowledgment of its well-motivated and theoretically grounded design, extensive experiments/ablations, and strong generalization. Regarding the weaknesses and questions you raised, we address them one by one as follows.

W: The sensitivity of ST-BoN with respect to the target generation length is understudied.

Thank you for pointing this out. To sensitivity of ST-BoN with respect to the target generation length, we choose some long-CoT reasoning models to demonstrate this issue. These models exhibit a "slow-thinking" characteristic, often producing outputs that are significantly longer than those of "fast-thinking" non-reasoning models. Specifically, we report results on Qwen3-8B-Thinking and Deepseek-R1-Distill-Qwen-7B, using the AIME24+25 dataset.
(Due to computational resource constraints, Full-BoN was only evaluated up to $N \leq 40$.)

Qwen3-8B-Thinking (Accuracy / Cost):

Deepseek-R1-Distill-Qwen-7B (Accuracy / Cost):

Key insights:

• First, due to the much longer generation lengths of reasoning models (often >10k tokens) compared to non-reasoning tasks (often <1k), the memory cost of KV cache becomes dominant. This significantly inflates the overall cost of Full-BoN. In contrast, ST-BoN performs early truncation and retains only a single path up to ~10k tokens, keeping its cost only marginally higher than that of greedy decoding.

• Second, ST-BoN still exhibits a markedly better cost–performance trade-off. For instance, on Qwen3-8B-Thinking, to reach 73.3 accuracy, ST-BoN requires only 13.8% (3.2/23.2) of the cost of Full-BoN w/ RM. Notably, even when ST-BoN uses $N=160$, Full-BoN at $N=10$ remains more expensive while underperforming.

These results provide strong evidence that ST-BoN consistently achieves a favorable cost–performance trade-off across both short and long output scenarios, reinforcing the robustness with respect to the target generation length of our ST-BoN.

Q1-Q3

• Code: Of course, we are preparing a clean version of the code, which will be released once the publication.

• Writing: Thanks for your suggestions on improving our writing. We will give them full consideration and incorporate the revisions in the next version.

We hope these results satisfactorily address your concerns and underscore the broader significance of our contributions. We look forward to your positive feedback, thanks again for your constructive comments.

Thank you for your constructive feedback. Below, we address the weaknesses (W1–W2) and questions (Q1–Q5) you raised one by one:

W1: Baseline Concerns

Thank you for pointing out these potential baselines. We present a Summary Table here and provide detailed explanations in responses to Q1 and Q2.

Summary Table

Model: Qwen2.5-7B-Instruct

• MATH (Accuracy / Cost):

• GPQA (Accuracy / Cost):

W2: Limited Novelty: Metric is heavily based on a prior work (chain of embedding).

<Please refer to Reviewer QemV’s "W3(1)" for a detailed response.>

Overall, the core idea of ST-BoN lies in the self-truncating BoN framework. The chain-of-embedding consistency metric is more about an empirical contribution. The two parts are complementary and equally essential.

Notably, both Reviewer QemV and Reviewer GGmq explicitly praised the novelty of ST-BoN.

Q1: Beam Search[1], MCTS[2], or Adaptive Inference-Time Compute[5], were not compared.

Our research aims to optimize the specific best-of-N paradigm. From the perspective of the search paradigm, [1], [2], and our approach are orthogonal --- as illustrated in Figure 2 of [1], the three types of methods are fundamentally independent:

• ST-BoN (ours) represents a randomized global sampling search,
• Beam Search[1] performs a local heuristic search,
• MCTS[2] performs a global tree-based heuristic search driven by simulation and value backpropagation.

Only Adaptive[5] directly targets best-of-N optimization.

Despite this, we agree that comparing against these methods can provide valuable insights and strengthen connections with parallel lines of research. Since [1] and [2] do not involve random sampling, they do not have the specific hyperparameter $N$. For fair comparison, we follow [1] by setting beam width and MCTS expansion to $[\sqrt{N}]$. Results are shown in the summary table in W1:

• Cost:

  • Beam Search must expand $k$ paths at each decoding step.
  • MCTS performs expansion and PRM-based scoring throughout.
  • Adaptive uses serial sampling with adaptive stopping, which inhibits GPU parallelism, and it still requires reward models; All three incur significantly higher computational cost compared to ST-BoN, which benefits from GPU-friendly parallel sampling and early truncation.

• Cost-performance Trade-off: ST-BoN demonstrates a clear advantage:

  • By progressively increasing $N$, ST-BoN can achieve the same performance level as the other three baselines while maintaining substantially lower computational cost
  • Under the same cost, ST-BoN still performs better.

These results further reinforce the "cost" and "cost-performance trade-off" contribution of our ST-BoN.

Q2: Generative reward models [3,4] which reduce the flop usage are not compared against.

The comparison with GRM can be naturally included under the “Full-BoN w/ RM” baseline, treating it as an alternative reward model. Specifically, we selected Gemma-7B-GRM as proposed in [3]. Results are shown in the summary table in W1:

• Cost: GRM outperforms RM in terms of accuracy, but its outputs are in natural language, which are significantly longer --- resulting in higher computational cost compared to RM.

• Cost-performance Trade-off: ST-BoN continues to demonstrate a strong advantage.

  • For example, on GPQA, when reaching an accuracy of ~36.5%, ST-BoN incurs only 3.7/14.4 = 25.7% of the computational cost of Full-BoN w/ GRM.
  • Under the same cost budget, ST-BoN clearly outperforms Full-BoN w/ GRM.

These results further reinforce the "cost" and "cost-performance trade-off" contribution of our ST-BoN.

Q3: Compare to the rule-based (oracle) reward model to evaluate the bias of the PRM.

Thanks for bringing up the idea of rule-based reward models. Since no specific reference was provided, we investigated some work and found that [7] uses predefined features and trains probes to model rewards in the safety domain, and [8] introduces an agent system that filters verifiable signals like instruction-following, fluency, and factuality. These work may align with rule-based reward models you mentioned.

However, such approaches have not been extended to the reasoning domain, where correctness is essentially the only feature that matters. If one fits a reward function based on correctness, it may essentially reduce the method back to the PRM paradigm, thereby blurring the distinction between the two. Moreover, our task is not based on a formal language, so automatic evaluation tools similar to code execution engines cannot be applied.

As an alternative, we used a more approximate oracle: LLM-as-a-Judge, since larger LLMs are typically more capable. We chose the latest ChatGPT-4o to evaluate the correctness of all sampled outputs, selecting the most frequently judged correct answer as the final output. The cost of using the Judge LLM includes both inference latency and API token usage.

• GPQA in Qwen2.5-7B-Instruct (Accuracy / Cost):

• Cost: Introducing an Oracle as the reward model does lead to improved performance, but it also significantly increases computational cost --- including inference latency and API usage --- which runs counter to our motivation of "optimizing the best-of-N paradigm".

• Cost-performance Trade-off: When achieving an accuracy of around 36.2, and excluding API overhead, the computational cost of ST-BoN is only 3.7 / 20.1 = 18.4% that of Full-BoN w/ Oracle.

These results further reinforce the "cost" and "cost-performance trade-off" contribution of our ST-BoN.

Q4: Evaluate how an RM trained on the summarization task would compare.

First, we included the summarization task mainly to show ST-BoN’s generalization ability. Training reward models across domains is challenging due to the lack of quality preference data—for example, there’s no standard reward model for summarization. ST-BoN avoids relying on such priors, making it more scalable and adaptable across tasks.

Despite this, we are happy to add a summarization-specific RM for clearer comparisons. Since no reward model and preference data exist for this task, we fine-tuned one based on ArmoRM-Llama-3-8B[6]. Using ChatGPT-4o, we annotated 6,000 samples from the XSum training set with 5-point ratings to train this summarization-specific RM.

We then conducted a human evaluation comparing three settings: Full-BoN w/ RM (our original baseline), Full-BoN w/ RM-Sum (the fine-tuned summarization reward model), and ST-BoN. The results are shown below:

• CNNDM in Qwen2.5-7B-Instruct (Accuracy / Cost):

The reward model fine-tuned on summarization data does yield improvements over its pre-trained counterpart. However, ST-BoN continues to show a clear advantage in cost-performance trade-off. To achieve a human rating of 4.43, ST-BoN requires only 4.3/11.8=36.8% of the cost compared to Full-BoN w/ RM-Sum. Furthermore, at a comparable computational budget (~3×), ST-BoN significantly outperforms Full-BoN w/ RM-Sum in output quality.

These results further reinforce the "cost" and "cost-performance trade-off" contribution of our ST-BoN.

Q5: Evaluation with Long-CoT models.

<Please refer to Reviewer GGmq’s Q2 for a detailed response.>

Overall, these results provide strong evidence for the scalability and efficiency of ST-BoN in the context of long-CoT reasoning models.

We hope that these additional results adequately address your concerns. We look forward to your positive feedback.

[6] Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts

[7] Rule Based Rewards for Language Model Safety

[8] Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems

Thanks for your recognition of our work, particularly the well-motivated method, sound theoretical justification, robust experimentation, and comprehensive coverage of related work. Below, we address the weaknesses (W1 - W2) you raised one by one:

W1: The experimentation of Section 5 does not conclusively demonstrate the universality of the internal consistency measure.

Thank you for pointing this out. We agree that the experiments in Section 5 provide a comprehensive cost–performance evaluation but do not directly isolate the contribution of individual components, such as the role of internal consistency measure.

To address this, we conducted a fine-grained ablation study in Section 6.1 (Figures 6, 14, and 15) to assess the universality of the internal consistency measure. Specifically, we replaced the "internal consistency measure" with two "output-based consistency measures" --- semantic similarity and string overlap --- and evaluated the correlation between early self-estimation results and final correctness. We conducted experiments across three models and found that only our internal consistency measure consistently achieved correlations significantly above random, demonstrating its universality.

W2: The paper's societal impact: how does the use of the generating trajectory sampling method they propose going to affect models trustworthiness, safety, and inherent biases?

We appreciate this important discussion. Societal implications are indeed crucial for applied research, and we summarize our perspective below:

• Trustworthiness: ST-BoN relies on internal consistency signals to guide decoding, moving the selection process into the model's latent space. Prior work [1] has shown a strong correlation between consistency and correctness. Building on this, our theoretical analysis (Section 3.1) and empirical findings (Section 6.1) further support the reliability of latent-space decision-making. Together, these two parts provide a solid foundation for the model trustworthiness under our method.

• Safety: ST-BoN introduces no additional safety risks. The sampling follows the model’s native distribution without adversarial perturbation, and the consistency-based selection mechanism operates without external reward models, thus avoiding interference from out-of-distribution signals.

• Biases: ST-BoN inherits the base model’s biases, as it does not perform explicit debiasing. However, by leveraging multiple parallel samplings, it can identify and truncate transient biases that arise sporadically during generation. These biased trajectories may be excluded as outliers through the consistency-based strategy, thereby potentially mitigating their impact.

We will incorporate these points into an additional “Societal Impact” section in the revised version of the paper.

We hope these clarifications address your concerns and provide a clearer understanding of our empirical conclusions. We sincerely appreciate your constructive comments and look forward to positive feedback.

[1] Self-consistency improves chain of thought reasoning in language models.

Thank you for your recognition of our work, especially the novelty of the proposed method and the strength of both the theoretical analysis and empirical evaluation. Regarding the weaknesses you raised (W1 - W3), we address them one by one as follows.

W1: The experimental results primarily focus on relatively smaller models (7B parameter scale)

Thank you for pointing this out. We kindly clarify that our paper already includes experiments using models with 70B+ parameters (Qwen2.5-72B-Instruct), as noted by Reviewer ciSh in Strength-4 "A study of robustness to increasing model size has been performed and reported on in Appendix D.1."

Due to space constraints in the main text, these results are presented in Figure 10 of Appendix D.1. The experimental results demonstrate that our method remains effective even on larger-scale models.

W2: Technical details such as specific implementation considerations or hyperparameter selection (e.g., setting $\tau$)

Thank you for pointing this out. $\tau$ is the only hyperparameter in ST-BoN, and we have already provided its intuitive explanation (Section 3.2) and selection rationale (Section 6.2) in the paper. Here, we present the key information:

• Intuitive Explanations I: Why set buffer window $\tau$ ?

The intuitive explanation here can be referred to in the original text from lines 140-142: Performing self-estimation at a single moment can introduce randomness, since pairwise sequence differences only begin to emerge at time $c$, and these differences may not be substantial. This is verified in Section 6.1.

• Intuitive Explanations II: Why define $\tau$ as $mc$ ?

The intuitive explanation here can be referred to in the original text from lines 143–147: "We set $\tau \propto c$, i.e., $\tau = mc$, where $m$ is a proportional constant. The intuition is that a smaller $c$ indicates an early divergence between samplings, which reflects greater randomness, so a smaller window may suffice to capture later differences. In contrast, a larger $c$ suggests lower randomness, which requires a larger window to capture sufficient later difference."

• Hyperparameter Selection: Why select $\tau$ as $c$ ?

The hyperparameter selection rationle here is demonstrated by the ablation study in Section 6.2. Figure 7 presents an ablation study on the cost–performance trade-off across four datasets by varying the value of $\tau$. As concluded in lines 313–315: Figure 7 shows that when $\tau < c$, the performance gain relative to cost increases faster. However, beyond $c$, this gain slows significantly, making $c$ the optimal choice for balancing performance and cost in our experiments."

W3: Although novel, the core idea builds incrementally upon previous work using consistency strategies and latent embeddings; I know some works about early termination or convergence analysis in search algorithms, which is not included in related works or baselines

(1) the core idea builds incrementally upon previous work

Thank you for recognizing the novelty of our method. Regarding the incremental construction of the core idea, we believe that the core idea of ST-BoN lies in "the design of a complete self-truncation framework that is free of full generation and reward models", but not only the consistency metric itself:

• Prior to Section 3.2, our focus was entirely on how to eliminate the use of full generation and the reward model in BoN, without mentioning any details about specific consistency metrics such as the “chain-of-embedding” (includes Figure 1, which only illustrates the framework). All of our motivation and theoretical modeling are aimed at proposing this framework, rather than any particular metric. This constitutes the core of ST-BoN and serves as a fundamental premise.

• Under this premise, the early consistency algorithm introduced in Section 3.2 should be viewed more as an empirical contribution. It makes the theoretically sound self-truncating framework proposed in Section 3.1 and Figure 1 practically implementable. From this perspective, whether the metric design is entirely novel or inspired by prior work is not essential; refining an already validated metric could even lend greater credibility to the overall approach.

From another perspective, the core contribution of ST-BoN --- efficiency improvement --- does not stem from any particular consistency metric, but rather from the self-truncating framework itself. As noted by Reviewer GGmq: “The overhead from BoN is a critical issue, and the proposed method does not rely on any additional model or training to successfully reduce the overhead, which is significant and insightful and can motivate more work exploring consistency during sampling.” --- it is this framework design that holds the most promise for inspiring further work. Therefore, the overall framework of ST-BoN and the consistency metric are equally important and mutually reinforcing in their contributions.

(2) I know some works about early termination or convergence analysis in search algorithms, which is not included in related works or baselines. (This work can be considered as one-step search algorithm)

We thank you for pointing out related work on early termination or convergence analysis in search algorithms. For example, [1] has explored pruning strategies in beam search.

However, we would like to clarify that our goal is to optimize the best-of-N paradigm itself, which is an independent search paradigm (randomized sampling search). Other search paradigms, such as beam search and MCTS, are representative of heuristic-guided search. These paradigms differ substantially in both algorithmic structure and optimization dynamics, and we consider them orthogonal research directions.

In our current version, we have already provided an in-depth discussion of early termination techniques within the Best-of-N search framework. That said, we fully acknowledge the value of cross-paradigm insights. We will carefully consider your suggestion and plan to include a broader discussion of early termination strategies in other search paradigms in the next revision.

We hope these clarifications address your concerns and provide a clearer understanding of our arguments, contributions, and empirical conclusions. We sincerely appreciate your constructive comments and look forward to positive feedback.

[1] Beam search pruning in speech recognition using a posterior probability-based confidence measure.