HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows

1. Clarifying slow vs. fast thinking: Thank you for the insightful comment. We agree that the concepts of slow and fast thinking (as introduced by Kahneman [1]) are complex. In our work, we use these terms more narrowly to distinguish between quick, single-step reasoning (what we term "fast thinking") and more deliberate, multi-step problem-solving that breaks down and analyzes sub-tasks (our "slow thinking"). We can clarify this terminology in the revision to avoid confusion with Kahneman's broader definitions. Our experiments demonstrate that strategically combining these modes of reasoning, even in this narrower context, can significantly enhance complex problem-solving in LLMs. We believe this is a valuable insight while acknowledging the full scope of human cognition is not captured.

2. Additional related work: We appreciate the pointer to several additional relevant works. We couldn't include all related previous work due to the page limit. We can expand our literature review to include these insightful papers [2] [3] [4] [5]. Thank you for strengthening our background discussion.

3. Additional baselines: Thank you for the suggestion to expand our baselines. In the revision, we can add experiments comparing our approach to more existing baselines.

4. Workflow illustrations: We have included two figures that visually illustrate our hybrid thinking approach (Figure 1) and our multi-stage dynamic workflow (Figure 2). Figure 2 includes: 1) Problem decomposition 2) Specialized expert design and 3) Workflow execution. We can clarify further if there are anything not clear enough.

Thank you the insightful suggestions!

1. Regarding the GPT-4-Turbo + CoT filtering concern: We want to clarify that using GPT-4-Turbo + CoT for validation serves primarily as a quality check to ensure problems are well-formed and unambiguous, not as a difficulty filter. Many problems that pass this basic validation still require more sophisticated reasoning approaches to solve efficiently or accurately. This is evidenced by our experimental results where GPT-4-Turbo with CoT alone achieves only 50.8% average accuracy across benchmarks (Table 1), while slow thinking and hybrid approaches show significant improvements (+22.4% and +21.6% respectively). Additionally, our synthesis process intentionally generates problems of varying complexity through multiple sources - using both GPT-4 and Claude-3-Opus to encourage diversity, and incorporating different genres of puzzles. The validation step simply ensures these problems are well-defined and have deterministic solutions.

2. Regarding the contamination test: We appreciate this important point about data contamination. We have conducted additional analysis to ensure the integrity of our evaluation. We performed a detailed comparison between our synthesized training data and the evaluation benchmarks (BBH, MATH, DeepMind Math, and GameOf24) using n-gram overlap analysis. For BBH, while we used the task formats as inspiration, our synthesis process generated entirely new problems with different contexts and solutions. For mathematical problems, we ensured our generated problems use different problem scenarios compared to MATH and DeepMind Math test sets. The GameOf24 benchmark is also different as GameOf24 is a specific mathematical puzzle format not directly targeted in our training data. To further strengthen this point, we could include detailed contamination analysis results in the appendix of our final version.

3. Regarding the accuracy claim: We agree that our current phrasing about hybrid thinking achieving 'the highest overall accuracy' needs revision for better precision. We will modify this statement to more accurately reflect our results: 'Hybrid thinking achieves the highest accuracy on three of the four datasets (BBH, MATH, and GameOf24) while providing an effective balance between computational efficiency and performance.' This better represents our findings while maintaining scientific accuracy. As shown in Table 1, while slow thinking has a slightly higher average accuracy (73.2% vs 72.4%), hybrid thinking provides this strong performance while using significantly fewer inference tokens (3105.5 vs 4432.0 tokens on average, as shown in Table 2).

4. Regarding minor comments:

• Thank you for pointing out the discrepancy between the table captions and contents for Tables 1 and 3. We will update the captions to remove the mention of the Fast/Slow ratio, as this information is not directly shown in the tables. The Fast/Slow ratio was moved to Figure 5.

• We appreciate the suggestion to further discuss the interesting finding mentioned in Section 6.3. In the revised paper, we will expand on this point to provide additional insights and analysis regarding how the hybrid thinking tuning process not only improves the model's slow thinking capabilities but also enhances its fast thinking (CoT) performance. This deeper discussion can help highlight the benefits and trade-offs of our proposed approach.

• We recognize that there is a small typo so we will correct the sentence to "The GPT-4-Turbo model demonstrates a higher reliance on fast thinking for BBH, MATH, DeepMind MATH tasks compared with Llama-3-8B-Instruct model."

Thank you for your valuable feedback on our paper. We would like to address each of your points:

1. Regarding the missing citation and discussion of the concurrent work "System-1.x: Learning to Balance Fast and Slow Planning with Language Models", we appreciate you bringing to our attention. As it was a concurrent work published several weeks before our work, we were not able to include it during the preparation and submission of our paper. We will certainly add a discussion about this work in the related work section of our revised paper, comparing and contrasting their approach with ours.

2. While there may appear to be similarities between our work and System-1.x at a high level, such as using chain-of-thought (CoT) for fast thinking and more deliberate planning for slow thinking, We believe our work has several novel contributions:

• Our Dynamic Workflow approach goes beyond just CoT and agentic planning. It dynamically decomposes complex problems into sub-tasks and assembles specialized LLM experts and symbolic reasoning tools in an adaptive workflow to solve each sub-task. This allows integrating diverse reasoning capabilities in a more flexible and extensible manner.

• Our Hybrid Thinking framework dynamically combines fast and slow thinking based on the problem complexity, rather than using a fixed hybridization factor. This allows our approach to adapt more effectively to problems of varying difficulty without manual tuning.

• We propose a novel method for automatically synthesizing a large-scale dataset of challenging reasoning problems and use it to finetune open-source language models with hybrid thinking abilities. This enables enhancing the reasoning capabilities of smaller models.

3. We agree that including System-1.x as an additional baseline would strengthen our experimental evaluation. In the revised paper, we can include experiments comparing our approach with System-1.x. We believe that the Dynamic Workflow, adaptive Hybrid Thinking, and model finetuning contributions of our work, along with the strong empirical results, demonstrate the novelty and significance of our approach for complex reasoning with language models.

Thank you for your thoughtful feedback on our paper. We would like to address your comments as follows:

1. Regarding the technical novelty of our framework: While our HDFlow framework does incorporate engineering design elements, we believe it makes several novel technical contributions:

• The dynamic workflow mechanism we proposed can enable LLMs to automatically decompose complex problems into sub-tasks and dynamically integrate specialized LLMs or symbolic reasoning tools to solve each sub-task. This goes beyond prior works that rely on manual workflow design of previous work.
• Our hybrid thinking approach introduces a new way to strategically combine fast thinking (e.g. CoT) and slow thinking (dynamic workflow) based on problem complexity.
• We propose a novel data synthesis pipeline to automatically generate a large-scale dataset of challenging reasoning problems. This enables efficient training - hybrid thinking tuning - of smaller LLMs for complex reasoning.

We can revise the paper to better highlight these technical contributions and clarify how they advance the state-of-the-art.

2. Comparison to prior fast/slow thinking approaches: Thank you for pointing out relevant prior work on combining fast and slow thinking, such as SWIFTSAGE. While some prior works have explored fast/slow thinking, our approach is different in several ways:

• HDFlow uses an adaptive mechanism to strategically switch between fast CoT and slow dynamic workflows based on solution verifiers and problem complexity. However, Prior works like SWIFTSAGE is a prompting method, while our HDFlow framework introduces hybrid thinking tuning that can enable small LLMs to do hybrid thinking beyond prompting.
• Our dynamic workflows enable more flexible integration of specialized LLMs and symbolic tools compared to fixed agent designs.
• We demonstrate benefits on more challenging reasoning benchmarks requiring multi-step thinking and tool use.

In the revision, we can include additional experiments comparing HDFlow to more baselines. We will clarify these differences and include empirical comparisons in the revision.

3. Definition of fast thinking: You raise a good point that the CoT prompting used in our fast thinking is more elaborate than the rapid intuitive responses typically associated with "fast thinking" in dual process theory. To clarify, we use "fast thinking" to refer to the model's initial attempt to solve the problem using its core capabilities, without employing the additional decomposition and tool use of our slow dynamic workflows. While CoT does involve some deliberation, it is still faster and more direct than our multi-stage slow thinking approach. We agree CoT exists on a spectrum between fully intuitive and fully analytical thinking. We will revise the terminology in the paper to avoid confusion with the typical definitions of "fast" and "slow" thinking.