We sincerely appreciate your thoughtful and detailed feedback. Below, we provide thorough responses to your questions and concerns.

1. Question

1.1 OOD Robustness

Soft Thinking does face OOD issues. However, the OOD problem here does not refer to tasks or problems. Instead, it specifically refers to the fact that the weighted sum embeddings of concept tokens were not seen during training. We believe lightweight fine-tuning could mitigate this issue. However, this work focuses on introducing the concept and algorithm itself, leaving robustness optimization as future work.

1.2 Sensitivity Analysis

We conducted hyperparameter sensitivity analysis on QwQ-32B and DeepSeek-Distill-Qwen-32B across AIME2024 and LiveCodeBench datasets.

1.2.1 QwQ32b (aime2024)

Length = 256

Entropy = 0.1

1.2.2 QwQ-32B (livecodebench)

Length = 512

Entropy = 0.05

1.2.3 DeepSeek-R1-Distill-Qwen-32B (aime2024)

Length=256

Entropy = 0.1

1.2.4 DeepSeek-R1-Distill-Qwen-32B (livecodebench)

Length=512

Entropy = 0.05

1.2.5 Analysis

The results demonstrate that our method is robust to hyperparameter variations across a large spectrum:

• Entropy Sensitivity:
  For QwQ-32B on AIME2024, entropy threshold ranging from 0.075 to 0.3 consistently achieve optimal results, exceeding the baseline. Similarly, for LiveCodeBench, entropy threshold values between 0.01 and 0.2 outperform the baseline. For DeepSeek-32B, the trends are similar, showing robustness across varying entropy thresholds.
• Length Sensitivity:
  On AIME2024, lengths threshold from 64 to 512 work effectively, while on LiveCodeBench, lengths threshold from 256 to 1024 exceed baseline performance.

Optimal hyperparameter ranges vary slightly across tasks but share a large overlap and are independent of the models used. Coding tasks require lower entropy and longer lengths threthod to generate longer code blocks.

For general settings, entropy = 0.1 and length = 512 perform well across all datasets and models, consistently improving upon the baseline. In our paper, for optimal results, we selected entropy = 0.1 and length = 256 for math tasks, and entropy = 0.05 and length = 512 for code tasks.

Therefore, our method has good robustness to parameters and does not require heavy parameter tuning.

We will add these important sensitive analysis to camera ready version.

1.3. Smaller Models

Additional experiments on Deepseek-distill-qwen series models:

Our experiments reveal that Soft Thinking does not significantly outperform the baseline on smaller models. However, as model size increases, Soft Thinking gradually surpasses the baseline. This is due to the following reasons:

1. In smaller models, even embeddings of semantically distant tokens may not form angles close to 90°, resulting in a higher proportion of irrelevant tokens in the probability distribution and increased noise.
2. Concept tokens are OOD, requiring the model to possess strong generalization abilities to understand embeddings derived from linear interpolation.

Moreover, our work addresses the issue faced by methods like COCONUT, which rely heavily on weight tying and therefore fail to work effectively on larger models.

2. Weakness

2.1 Problem 1: Semantic Collapse Risk

Thank you for your insightful observation. To clarify, Soft Thinking does not aggregate embeddings of tokens with vastly different semantics. The weighted probabilities are derived from the hidden state multiplied by the LM head and processed through a softmax function. This ensures that tokens with higher weights are semantically closer to the hidden state, while tokens with distant or opposite semantics have near-zero probabilities in the distribution.

For instance, combinations like "I'll start" and "can use" at positions 13 and 14 (as shown in Figure 4) represent semantically consistent reasoning paths. In contrast, unrelated concepts such as "integral" and "loop" would not appear simultaneously in the same token. We hope this explanation alleviates concerns about semantic collapse risk. We will include this clarification in the revised paper. Thank you for pointing it out.

2.2 Problem 2: Similarity to Other Methods

We want to clarify that our approach is fundamentally different from these works. Soft Thinking innovatively extends LLM reasoning into continuous space without requiring additional modules or training. Below are specific distinctions:

• COCONUT: Our method overcomes several limitations of COCONUT, including reliance on weight tying, the need for training, and incompatibility with large models. These differences were discussed in the related work section (line 92-100).

• DoLA: The statement Weighted different layer representations generate concept vectors is not very accurate. DoLA does not use weighted representations. Instead, it contrasts the probability distributions of premature layers and mature layers. In addition, DoLA amplifies factual knowledge encoded in higher layers by contrasting probabilities but does not generate abstract "concept vectors and still use discrete tokens.

Nevertheless, thank your for pointing out these methods. We will discuss them in camera ready version for clarification.

2.3 Cold Stop

Cold Stop is a straightforward and effective approach to reducing token counts and addressing OOD issues. We primarily validate the effectiveness of Cold Stop through experiments detailed in the ablation section. Regarding the theoretical aspect, due to space constraints, we will provide a more comprehensive theoretical derivation in the camera-ready version.

2.4 Comparison to Speculative Decoding and ACT

Thank you for sharing these methods. Speculative Decoding focuses on accelerating decoding speed by using smaller models, not skipping several tokens. ACT determines the number of computational steps or layers needed based on input complexity. While ACT operates vertically (layer-wise), Soft Thinking operates horizontally (reasoning steps). These are orthogonal optimization directions, and Soft Thinking could be combined with these methods, which we consider a promising future direction. These possible research directions don't impact the innovation of Soft Thinking.

2.5 Limited Analysis of Failure Cases

Our experiments reveal that tasks requiring lengthy code generation demand higher thresholds, as lower ones may cause premature termination. We will include examples illustrating this issue in the revised paper.

2.6 Benchmarks Focus on STEM

The seven benchmarks (four math, three coding) we selected are widely recognized and commonly used, providing a comprehensive comparison of different methods. However, we will expand our experiments to include additional benchmarks in the revised paper to further validate generalization.

2.7 Related Works Comparison

Methods like Tree of Thoughts (ToT) and Graph of Thoughts (GoT) require extensive sampling (e.g., 16-64 samples for self-consistency), making their token usage significantly higher (up to 64x CoT). We will add experiments in the revised paper to highlight the trade-offs between effectiveness and efficiency, due to the constraints of the rebuttal period and computational resources.

Regarding Soft Prompt, while both methods contain "soft" in their names, they are fundamentally different. Soft Prompt focuses on parameter-efficient fine-tuning, requiring additional training and fixed soft prompts during inference. In contrast, Soft Thinking expands reasoning tokens into continuous space, addressing a different problem altogether.

Acknowledgment

We sincerely respect and appreciate the hard work and professionalism demonstrated in your review. We look forward to your feedback on our revisions and thank you again for your valuable time and effort.

Hi, when you have a moment, could you please take a look at the authors' response to see whether it addresses your concerns, or if there are still points that remain unclear? Thank you.

Thank you for acknowledging that our responses have addressed your previous concerns. We appreciate your recognition of our work.

While our response has partially addressed them, we would like to kindly ask for clarification on which specific aspects remain unresolved. We are more than happy to discuss further and provide additional explanations if needed.

Best,

Authors

We sincerely appreciate your thoughtful and detailed feedback. Below, we provide thorough responses to your questions and concerns.

1. The text in the image is too small.

Thank you for your suggestion. We will adjust the proportions between the blank space and the text in the images, enlarge the text, and enhance the contrast between the text and the background to improve readability.

2. Extremely Long OOD Inputs

Soft Thinking is capable of handling extremely long OOD inputs effectively. For instance, on the AIME2024 dataset, the average reasoning length exceeds 10k tokens, while on LiveCodeBench, it surpasses 7k tokens, with the maximum length reaching over 30k tokens. These results demonstrate that the model performs well on extremely long reasoning tasks.

3. Hyperparameter Sensitivity Analysis

We conducted hyperparameter sensitivity analysis on QwQ-32B and DeepSeek-Distill-Qwen-32B across AIME2024 and LiveCodeBench datasets.

3.1 QwQ32b (aime2024)

Length = 256, Varying Entropy

Entropy = 0.1, Varying Length

3.2 QwQ-32B (livecodebench)

Length = 512, Varying Entropy

Entropy = 0.05, Varying Length

3.3 DeepSeek-R1-Distill-Qwen-32B (aime2024)

Length=256, Varying Entropy

Entropy = 0.1, Varying Length

3.4 DeepSeek-R1-Distill-Qwen-32B (livecodebench)

Length=512, Varying Entropy

Entropy = 0.05, Varying Length

3.5 Analysis

The results demonstrate that our method is robust to hyperparameter variations across a large spectrum:

• Entropy Sensitivity:
  For QwQ-32B on AIME2024, entropy threshold values ranging from 0.075 to 0.3 consistently achieve optimal results, exceeding the baseline. Similarly, for LiveCodeBench, entropy threshold values between 0.01 and 0.2 outperform the baseline. For DeepSeek-32B, the trends are similar, showing robustness across varying entropy thresholds.

• Length Sensitivity:
  On AIME2024, lengths threshold from 64 to 512 work effectively, while on LiveCodeBench, lengths threshold from 256 to 1024 exceed baseline performance.

Optimal hyperparameter ranges vary slightly across tasks but share a large overlap and are independent of the models used. Coding tasks require lower entropy and longer lengths threthod to generate longer code blocks, as overly aggressive stopping can truncate meaningful outputs.

For general settings, entropy = 0.1 and length = 512 perform well across all datasets and models, consistently improving upon the baseline. In our paper, for optimal results, we selected entropy = 0.1 and length = 256 for math tasks, and entropy = 0.05 and length = 512 for code tasks.

Therefore, our method has good robustness to parameters and does not require heavy parameter tuning.

We will add these important sensitive analysis to camera ready version.

4. Cold Stop and generation length on LiveCodeBench:

The increase in average generation length for correct solutions with Cold Stop is a result of the model solving more problems, many of which require longer reasoning chains. Specifically, without Cold Stop, the model achieves an accuracy of 56.98%, with an average generation length of 6,877 tokens for correct solutions. When Cold Stop is applied, the accuracy improves significantly to 62.72%, allowing the model to correctly solve more problems, including many that demand reasoning chains exceeding 10k tokens.

As a result, the inclusion of these more challenging problems naturally increases the average generation length for correct solutions to 7,535 tokens. This does not indicate that Cold Stop's stopping conditions are suboptimal for complex code tasks. Instead, it highlights Cold Stop helps to handle more difficult problems, which require longer reasoning chains.

5. Comparison with Tree of Thoughts and self-consistency:

Your suggestion regarding the lack of comparison with these methods is very valid. The primary reason is that our approach targets both effectiveness and efficiency, while methods like Tree of Thoughts or self-consistency require extensive sampling or searches. For example, self-consistency often requires sampling 16 or 32 times, leading to token usage that is 16 or 32 times higher than the baseline CoT. Moreover, integrating ToT and self-consistency into SGLang demands significant development time. Given the constraints of the rebuttal period and computational resources, we will conduct experiments comparing these methods in the camera-ready version to emphasize the trade-offs between effectiveness and efficiency.

Acknowledgment

We genuinely appreciate the diligent and professional nature of your review. Your insightful comments have been invaluable in refining our paper, and we trust that the clarifications above adequately resolve your concerns. We eagerly await your response to our revisions and extend our gratitude once more for your valuable time and effort.

Hi, when you have a moment, could you please take a look at the authors' response to see whether it addresses your concerns, or if there are still points that remain unclear? Thank you.

We truly value the your comprehensive and insightful feedback. Below, we offer detailed replies to your questions.

Explanation to Parallel Thinking

1. From a qualitative perspective

Soft Thinking retains the original probability distribution, preserving more information compared to traditional sampling methods. In standard sampling, the probability distribution collapses into a one-hot vector (where the position corresponding to the selected token is set to 1). This increases certainty, as the entropy of a one-hot distribution is zero, leaving no room for uncertainty.

2. From a theoretical perspective

Soft Thinking also provides a theoretical explanation for how it enables parallel reasoning.

In Section 3.3, Equation (10) demonstrates:

Here, we prove that the concept token is a linear approximation of all possible discrete tokens at this step. The full outer summation over the first token is replaced by a single evaluation at the concept token $ct_1$ via linear approximation.

Subsequently, in Equations (11) and (12), we apply the same linearization recursively:

Given $ct_1$, the conditional probability expands as:

where $ct_2 = \sum_{t_2} p(t_2 \mid x, ct_1) t_2$.

Repeating this process for all $m$ steps yields the continuous expansion:

We demonstrate that the multi-step concept tokens ($ct_1, \ldots, ct_m$) are linear approximations of all possible path combinations of discrete tokens ($t_1, \ldots, t_m$) at each step, weighted by their respective path probabilities. As a result, a single reasoning path composed of concept tokens can approximate all possible discrete token reasoning paths, enabling the exploration of multiple reasoning paths in parallel within a single reasoning trajectory.

3. Specific Examples

Additionally, as illustrated in Figure 4 of the paper, we observe examples of parallel reasoning in practice. For instance, in steps 13–14, combinations such as "I’ll start," "I can use," and "I’ll break" emerge; similarly, in steps 18–20, combinations like "break down the number into" and "break down the multiplication to" appear. These practical examples further demonstrate the ability of Soft Thinking to conduct parallel reasoning.

Thus, the paper already provides qualitative evidence, theoretical proof, and case studies to explain how continuous space reasoning enables token diversity and facilitates the exploration of multiple reasoning paths in parallel.

Nevertheless, we acknowledge the need for further clarity in this explanation. We will rewrite and improve this section in the revision to address the concerns raised by the reviewer.

Acknowledgment

We genuinely appreciate the diligent and professional nature of your review. Your insightful comments have been invaluable in refining our paper, and we trust that the clarifications offered above adequately address your concerns. We eagerly await your response to our updated submission and thank you once more for your considerable time and effort.

Hi, when you have a moment, could you please take a look at the authors' response to see whether it addresses your concerns, or if there are still points that remain unclear? Thank you.

We sincerely appreciate the reviewer’s thorough and constructive feedback on our paper. Your valuable insights have helped us identify areas for improvement and refine our work further. Below, we provide detailed responses to all the questions and concerns raised.

1. Hyperparameter Sensitivity Analysis

We conducted hyperparameter sensitivity analysis on QwQ-32B and DeepSeek-Distill-Qwen-32B across AIME2024 and LiveCodeBench datasets.

1.1 QwQ32b (aime2024)

Length = 256, Varying Entropy

Entropy = 0.1, Varying Length

1.2 QwQ-32B (livecodebench)

Length = 512, Varying Entropy

Entropy = 0.05, Varying Length

1.3 DeepSeek-R1-Distill-Qwen-32B (aime2024)

Length=256, Varying Entropy

Entropy = 0.1, Varying Length

1.4 DeepSeek-R1-Distill-Qwen-32B (livecodebench)

Length=512, Varying Entropy

Entropy = 0.05, Varying Length

1.5 Analysis

The results demonstrate that our method is robust to hyperparameter variations across a large spectrum:

• Entropy Sensitivity:
  For QwQ-32B on AIME2024, entropy threshold values ranging from 0.075 to 0.3 consistently achieve optimal results, exceeding the baseline. Similarly, for LiveCodeBench, entropy threshold values between 0.01 and 0.2 outperform the baseline. For DeepSeek-32B, the trends are similar, showing robustness across varying entropy thresholds.

• Length Sensitivity:
  On AIME2024, lengths threshold from 64 to 512 work effectively, while on LiveCodeBench, lengths threshold from 256 to 1024 exceed baseline performance.

Optimal hyperparameter ranges vary slightly across tasks but share a large overlap and are independent of the models used. Coding tasks require lower entropy and longer lengths threthod to generate longer code blocks, as overly aggressive stopping can truncate meaningful outputs.

For general settings, entropy = 0.1 and length = 512 perform well across all datasets and models, consistently improving upon the baseline. In our paper, for optimal results, we selected entropy = 0.1 and length = 256 for math tasks, and entropy = 0.05 and length = 512 for code tasks.

Therefore, our method has good robustness to parameters and does not require heavy parameter tuning.

We will add these important sensitive analysis to camera ready version.

2. Explanation of Cold Stop Mechanism

2.1 How Cold Stop Works

Cold Stop does not fundamentally alter the model’s reasoning ability or enable it to produce correct reasoning paths where it previously could not. Instead, its purposes are:

1. Make the answer easy to parse: By stopping reasoning once the model reaches high confidence, Cold Stop ensures that correct answers are output cleanly and easily to parse by our evaluation scripts automatically.
2. Reducing Token Usage: It halts unnecessary reasoning steps once the model has already reached the correct conclusion, thereby saving tokens.

Therefore, for those samples where the judgment changed from incorrect to correct after using Cold Stop, the correct answer has already been output before stopping.

In addition, since generation rarely happens in baseline (CoT), Cold Stop will not improve the accuracy.

Below is a simple example to illustrate how Cold Stop works. We only show the top 1 token of Soft Thinking for visualization:

Question:
1+1=?

Without cold stop:
<think>Let me solve this question. … (some thinking) …. Therefore the correct answer is 2. Emmm, let me verify my answer in another way. Let’s Let’s Let’s Let’s Let’s …  (repetition occurs until the maximum number of tokens is reached)

With cold stop:
<think>Let me solve this question. … (some thinking) …. Therefore the correct answer is 2. Emmm, let me verify my answer in another way. Let’s Let’s Let’s (cold stop here)</think> The final answer is 2.

For the example without cold stop, the correct answer appears in the thinking process but not in the output, and can not be parsed for evaluation. With Cold Stop, we stop the generation collapse properly and let the model conclude so that we can parse the answer.

2.2 Improvement from Cold Stop

Accuracy improvements stem from Soft Thinking itself, which enables the model to explore abstract and parallel reasoning paths to find correct solutions. Cold Stop primarily serves as a utility to enhance token efficiency and prevent collapse during inference. Because it cannot make the model generate more content, it will instead reduce the generated content.

Regarding token efficiency, we analyzed the token reductions contributed by Soft Thinking and Cold Stop. The following table summarizes the results:

Both Soft Thinking (with or without Cold Stop) reduce token usage in terms of total and correct solution length. Cold Stop further mitigates generation collapse, ensuring correct answers are cleanly output while reducing tokens. As a result, accuracy improvement is from Soft Thinking and efficiency improvement is from both Soft Thinking and Cold Stop.

For baseline (CoT), generation collapse (repetition) is rare. Since Cold Stop is a method to let model generate output faster, it will not improve the accuracy.

3. Missing Related Work

We appreciate the reviewer pointing out missing references. In the camera-ready version, we will update the "Related Work" section to include relevant papers such as:

• Implicit Chain of Thought Reasoning via Knowledge Distillation
• From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
• Follow-up works building on these ideas

Thank you for highlighting this oversight, and we will ensure these contributions are properly acknowledged.

4. Paper Formatting

Regarding the term "Visualization of Shorted Example," "Shortened" is a more precise term to describe the concise reasoning chains generated by Soft Thinking. We will update this phrasing in the final version.

Acknowledgment

We sincerely respect and appreciate the hard work and professionalism demonstrated in your review. Your thoughtful feedback has been instrumental in improving our paper, and we hope the explanations provided above address all concerns. We look forward to your feedback on our revisions and thank you again for your valuable time and effort.

Hi, when you have a moment, could you please take a look at the authors' response to see whether it addresses your concerns, or if there are still points that remain unclear? Thank you.