Thanks for your valuable comments. Our responses to your concerns are as follows:

Q1&W4: Performance of L1-Max

We would like to clarify that both L1-Exact and L1-Max are trained on DeepScaleR-1.5B-Preview, which is a stronger model obtained by further tuning DeepSeek-R1-Distill-Qwen-1.5B using GRPO. As such, the performance of L1-Max benefits from additional training and a more capable backbone.

In contrast, our ACPO model are trained based on DeepSeek-R1-Distill-Qwen-1.5B, without the additional GRPO training step. Therefore, the comparison is not directly fair in terms of model capacity.

To reflect this, we report L1 baselines separately in Table 1 as reference points, rather than treating them as primary baselines.

Q2: Reward Function effects on fast and slow thinking behavior

In the reward computation phase of ACPO, for queries whose estimated difficulty exceeds a predefined threshold, our system pattern reward includes a positive term proportional to the ratio of <slow_think> tokens in the response. This design explicitly encourages the model to allocate more slow thinking to harder questions during reinforcement learning. Conversely, for easier queries (below the threshold), the reward encourages a higher proportion of fast thinking.

As shown in Figure 3, we visualize the distribution of fast vs. slow thinking tokens across different difficulty levels in the MATH dataset. The results show a clear increase in the proportion of slow thinking as task difficulty increases, validating the intended behavior of our reward function.

Q3&W1: Ablation studies of reward

To assess the individual contributions of each reward component, we conduct ablation studies by incrementally adding each reward term during training. The results on MATH 500 and AIME 2024 are summarized below.

From the results we can see that:

1. The accuracy reward improves task performance while moderately reducing token usage. 2) Adding the online TLB reward significantly compresses reasoning length, though it may slightly affect accuracy due to more aggressive compression. 3) The system pattern reward helps recover some accuracy while maintaining efficient token usage.

We will include reward loss curves and additional analysis in the final version to provide a more comprehensive ablation study.

W2: The optimization objective of RL phase

While the SFT stage introduces token-level thinking modes, the RL phase reinforces the intended structural relationship through the system pattern reward. Specifically, this reward is positively correlated with the proportion of <slow_think> or <fast_think> tokens, depending on the estimated difficulty of the question.

For harder questions (above a predefined difficulty threshold), the model receives higher reward for allocating more <slow_think> tokens, promoting more deliberate and thorough reasoning. Conversely, for easier questions, the reward encourages a higher proportion of <fast_think> tokens to favor brevity and efficiency. This dynamic reward design guides the model to allocate cognitive effort adaptively based on task complexity.

As shown in Figure 3, the model learns to produce longer slow thinking segments for more difficult problems, and shorter reasoning traces for easier ones.

W3: Generalization to other domains

We understand the reviewer’s concern regarding generalization. While our initial experiments focused on math reasoning, we additionally evaluated our method on the GPQA Diamond dataset. As shown in the table below, ACPO significantly reduces reasoning token usage while maintaining comparable accuracy, demonstrating its potential to generalize beyond math.

Our method is task-agnostic and can be extended to other domains involving complex reasoning. Our current training data focuses primarily on math problems. Tasks like code generation and agent pose additional challenges due to the task constraints. Nevertheless, we plan to synthesize fast/slow thinking data in these domains to enable supervised fine-tuning and ACPO training, further enhancing the method’s applicability to a broader range of complex reasoning tasks.

Thanks for your valuable comments. Our responses to your concerns are as follows:

W1: Generalization of ACPO

We understand the reviewer's concern regarding generalization. While our original experiments focused on math reasoning tasks, we agree that broader evaluation is important. To this end, we conducted additional experiments on GPQA Diamond. As shown in the table below, ACPO reduces reasoning token usage while maintaining comparable accuracy, demonstrating its generalization potential beyond math.

Our method is task-agnostic and can be extended to other domains involving complex reasoning. In future work, we plan to synthesize more fast/slow thinking data in general domains for supervised fine-tuning and ACPO training, to further enhance its applicability across a wider range of tasks.

Q1: Quantify the computational saving

We thank the reviewer for highlighting this important aspect. One of the key motivations of our method is indeed to reduce overthinking and improve reasoning efficiency during inference.

In Table 1 and Table 2, we report Accuracy, Token Count, and ACU to demonstrate the effectiveness of our approach. Additionally, we compute the token compression ratio, defined as: Compression Ratio = 1 − tokens(ACPO) / tokens(Baseline), which quantifies how much token usage is reduced compared to baseline models.

As shown in the table below, ACPO consistently reduces the number of reasoning tokens across all datasets and backbones. Notably, the maximum compression of ACPO-1.5B reaches 70.58% on the MATH dataset. We will include additional quantitative analyses of token savings in the final version to better illustrate the computational efficiency gains enabled by ACPO.

Thanks for your valuable comments. Our responses to your concerns are as follows:

Q1: Generalization to other domains

We understand the reviewer’s concern regarding generalization. While our initial experiments focused on math reasoning, we additionally evaluated our method on the GPQA Diamond dataset. As shown in the table below, ACPO significantly reduces reasoning token usage while maintaining comparable accuracy, demonstrating its potential to generalize beyond math.

Our method is task-agnostic and can be extended to other domains involving complex reasoning. Our current training data focuses primarily on math problems. Tasks like code generation pose additional challenges due to some formatting constraints. Nevertheless, we plan to synthesize fast/slow thinking data in these domains to enable supervised fine-tuning and ACPO training, further enhancing the method’s applicability to a broader range of complex reasoning tasks.

Q2: Evaluation on more challenging benchmarks

To strengthen our evaluation on challenging benchmarks, we tested our method on AIME 2025 and AMC 2023. As shown below, ACPO achieves effective token compression on both datasets. Notably, it achieves higher compression on the simpler AMC 2023, and on the more difficult AIME 2025, it improves accuracy while reducing reasoning length, further demonstrating the effectiveness of our method in adapting reasoning length to task complexity.

Q3：Effect of SFT warmup

We thank the reviewer for highlighting the cold-start phase. Its main purpose is not to boost accuracy, but to teach the model to emit <fast_think> and <slow_think> tokens in a consistent and semantically meaningful way. This structured initialization lays the foundation for the reinforcement learning phase.

To assess the impact of this phase, we report the model's accuracy before and after SFT. As shown below, the accuracy remains largely stable. We will clarify this point more explicitly in the future version.

Q4: Intuition behind reward design

For tasks like math problem solving, answer correctness is the primary objective, and any reasoning trace—no matter how efficient or well-structured—is ultimately invalid if it leads to the wrong answer.

To enforce this principle, we assign a mild negative reward to incorrect answers, ensuring that no positive reward is given unless the final prediction is correct. Using zero reward for incorrect answers can unintentionally result in a positive total reward if the reasoning trace scores well on length or structure, which contradicts the task goal.

This design encourages the model to first guarantee correctness, and then optimize for reasoning efficiency and cognitive control. We found this constraint important to prevent reward hacking and guide the model toward meaningful improvements.

Q5：GRPO baseline results

We thank the reviewer for pointing this out. We will supplement Table 2 with the corresponding GRPO results and include GRPO as a baseline across all evaluated datasets in the updated version.

Thanks for your valuable comments. Our responses to your concerns are as follows:

Q1: Clarification on how fast and slow thinking modes are interleaved

The Dual Process Reasoning in Figure 1 indeed corresponds to step-level thinking-mode annotations, not per-instance labels. Each response includes interleaved <fast_think> and <slow_think> tags to mark different reasoning steps, enabling fine-grained cognitive switching within a single trace. This interleaving behavior is learned through a two-stage process: supervised fine-tuning with explicitly annotated data and reinforcement learning with system-aware rewards. A concrete token-level example is shown in Figure 4. We will revise Figure 1 to clarify this more explicitly.

Q2：Annotation data quality

We acknowledge that annotation quality is critical. In our pipeline, we use the same model with different prompts to generate short and long responses, which encourages differences in reasoning granularity while largely preserving the core solution structure.

We rely on GPT-4 to perform the annotation automatically and filter out any samples where GPT-4 fails to annotate due to structural mismatch or unclear reasoning trace. We spot-checked a subset of samples to verify that the generated tags aligned with our intended labeling protocol.

We appreciate the reviewer’s suggestion and will further review all annotated data to ensure consistency and quality.

Q3: Differentiation between fast and slow reasoning

In our framework, the distinction between fast and slow reasoning is established during data construction by prompting the same model with different instructions. The short responses are encouraged to be concise and direct, while the long responses are prompted to be more thorough and elaborative.

To obtain step-level annotations, we compare the short and long response pairs using GPT-4. Reasoning steps shared by both responses are labeled as slow thinking, while steps present only in the long response are labeled as fast thinking. This approach does not rely on a fixed token-length threshold, but instead uses semantic comparison to determine reasoning granularity.

Below, we report the average token lengths of short, long, and annotated responses to provide a reference for their length distribution.

Q4: Comparison with LIMO performance

We would like to clarify that the LIMO paper uses Qwen2.5-32B-Instruct as its backbone for supervised fine-tuning, while our experiments are conducted on smaller models (DeepSeek-R1-Distill-Qwen-1.5B, 7B, and DeepSeek-R1-Distill-LLaMA-8B). Given the significant gap in model capacity, the accuracy numbers are not directly comparable.

Additionally, our SFT phase does not use the original LIMO solutions, but the newly generated reasoning paths annotated with <fast_think> and <slow_think> tokens.

To isolate the impact of backbone, we fine-tuned DeepSeek-R1-Distill-Qwen-1.5B on the original LIMO dataset to serve as a fair comparison. As shown in the table below, the performance of the trained model is even lower than our ACPO results.

Finally, we would like to emphasize that our method is not solely aimed at maximizing accuracy, but at achieving comparable performance with more efficient (i.e., shorter) reasoning traces.

Q5: Controversies of Dual-Mode Thinking

We are pleasantly surprised and grateful that the reviewer recognized our method as an alternative to the hard binary of fast and slow thinking. In fact, our step-level thinking-mode switching is designed to reflect the dynamic nature of the reasoning process, where fast and slow thinking are interleaved rather than strictly separated.

Rigidly dividing cognition into two distinct modes, or assigning a fixed mode to each problem, is a coarse approach. What inspires us from the dual process theory is not the binary classification itself, but the notion of dynamic cognitive adjustment during reasoning.

We will further clarify this point in the final version to better explain our motivation.

Q6: Impact of token budget on performance and failure cases

While ACPO achieves substantial token savings, we acknowledge that aggressive compression may occasionally lead to performance degradation.

We conducted case studies where the baseline model answered correctly but the ACPO-trained model failed. On these cases, baseline responses averaged 7691 tokens, while ACPO responses averaged 3999 tokens. The baseline typically explored more intermediate steps and reasoning paths, while ACPO missed certain details due to insufficient exploration. These failures mostly occurred in more challenging cases where longer reasoning is more beneficial. However, our dynamic budget mechanism helps reduce some of these failures by allocating more tokens to harder problems.

Due to space constraints, we will show the detailed case study in the final version to further clarify this point.

Q7: Analysis of ACPO on distribution shift

To assess ACPO’s robustness under distribution shift, we evaluated both the baseline and ACPO-trained models on GSM-Plus.

As shown in the table below, ACPO maintains comparable accuracy while effectively reducing the reasoning length, demonstrating its generalization ability under distribution shift.

W1：Some detailed explanations

Some experiments were not conducted on GSM8K because it is relatively simple, and the compared baselines did not consider it as a target test set. Additionally, due to the lack of released models or code, or missing reported results, we were unable to include them for comparison. We will include GSM8K results once the corresponding baseline models become available.

W2: Generalization to other domains

While our original experiments focused on math reasoning tasks, we agree that broader evaluation is important. To this end, we conducted additional experiments on GPQA Diamond. As shown in the table below, ACPO reduces reasoning token usage while maintaining comparable accuracy, demonstrating its generalization potential beyond math.

Our method is task-agnostic and can be extended to other domains involving complex reasoning. In future work, we plan to synthesize more fast/slow thinking data in general domains for supervised fine-tuning and ACPO training, to further enhance its applicability across a wider range of tasks.