Improving LLMs‘ Generalized Reasoning Abilities by Graph Problems

Q3: Magnitude and Significance of Model Improvements After Continue-Pretraining on GraphPile

That’s a great observation about our experiments, and we’re happy to elaborate！To provide a clearer picture, we present a detailed breakdown of the improvements on both graph-specific tasks and general reasoning tasks (including math, commonsense, logical reasoning, and coding).

For reference,

• Graph tasks include: GraphWiz and GraphInstruct.
• Math tasks include: GSM8K, MATH, GSM8K-Hard, SVAMP, ASDIV, MAWPS, MINERVA_MATH, MMLU_STEM, TABMWP, MATHQA, and SAT_Math.
• Logical reasoning tasks include: Zebra Puzzle, Ruletaker, and ProofWriter.
• Commonsense tasks include: StrategyQA and Hellaswag.
• Coding tasks include: Livecodebench and CLRS.

The table below summarizes the performance before and after continue-pretraining (CPT) on GraphPile:

On graph tasks, all models see substantial improvements after CPT on GraphPile:

• Llama3: +40.5 points (19.9 → 60.4)
• Llama3.1: +45.7 points (17.9 → 63.6)
• Gemma2: +33.1 points (25.7 → 58.8)

This demonstrates a significant boost in in-domain (graph reasoning) capability.

On general reasoning tasks (out-of-domain), the improvements are smaller but still consistent and meaningful:

• Llama3: +9.1 points (39.8 → 48.9)
• Llama3.1: +6.8 points (43.2 → 50.0)
• Gemma2: +3.8 points (30.4 → 34.2)

While the gains on general tasks are less dramatic than on graph tasks, it is important to note that GraphPile is the first CPT dataset to demonstrate general reasoning improvements across diverse domains. In comparison, previous CPT works have mainly focused on enhancing performance within a single domain (e.g., code, math, or biomedicine) [6,7,8].

In summary, our approach not only delivers notable improvements in graph reasoning, but also pioneers the transfer of CPT benefits to broader general reasoning—highlighting a key contribution and distinction of GraphPile relative to prior work.

[6] Lu et al. MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code. ArXiv preprint, 2024.
[7] Shao et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv preprint, 2024.
[8] Chen et al. Meditron-70b: Scaling medical pretraining for large language models. ArXiv preprint, 2023.

---

Q4: Problems After Continue-Pretraining

You raise an excellent question regarding the potential issues associated with continue-pretraining! While our models demonstrate clear improvements on reasoning tasks, our experimental results demonstrates a decline in performance on some relatively simple tasks such as translation and summarization. However, this phenomenon is not unique to our models. Similar trends have also been reported in other reasoning-oriented LLMs, including Qwen-math [9] and DeepSeek-math [10].

[9] Yang et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. ArXiv preprint, 2024.

[10] Shao et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv preprint, 2024.

Q2: GraphPile’s Distinct Advantages Over Existing Graph Reasoning Datasets

Thank you for your suggestion to clarify the distinction between GraphPile and previous datasets. We summarize the key differences between GraphPile and representative datasets such as GraphWiz, GraphInstruct, InstructGraph, and GraphArena in the table below (focusing on graph problem reasoning tasks):

CoT: Chain-of-Thought, PoT: Program-of-Thought, ToE: Trace-of-Execution, CPT: Continue-Pretraining.
Simple Answer refers to providing only the answer to a given problem without any intermediate reasoning or explanation.

Key advantages of GraphPile:

1. Graph Diversity:
   GraphPile uniquely combines both synthetic and real-world graphs, whereas prior datasets include only one type. This diversity exposes models to a wider range of scenarios and enhances generalization.

2. Problem-Solving Paradigms:
   GraphPile is the first dataset to systematically include multiple reasoning paradigms—Chain-of-Thought (CoT), Program-of-Thought (PoT), and Trace-of-Execution (ToE)—enabling more robust and systematic problem-solving. In contrast, existing datasets are limited to simple answers or CoT.

3. Scale:
   GraphPile is orders of magnitude larger, containing 2.68 million samples and over 10.9 billion tokens, compared to tens of thousands in previous datasets.

4. Task Coverage:
   With 24 types of graph problems, GraphPile covers a much broader spectrum of reasoning tasks, including logical, numerical, enumerative, and topological reasoning, while most prior datasets include fewer than 10 task types.

5. CPT Compatibility:
   Thanks to its scale and diversity, GraphPile is the first dataset specifically designed for continue-pretraining. This enables LLMs to acquire deep and transferable graph reasoning abilities. Previous datasets, due to their limited size and diversity, are not suitable for effective CPT and thus cannot support broad generalization.

In summary, GraphPile stands out by offering greater diversity, richer reasoning paradigms, larger scale, and CPT-oriented design. These advantages make it uniquely suited for developing more robust and generalizable reasoning models. Thank you again for your helpful suggestion, which allowed us to highlight the contributions of GraphPile more clearly.

---

Empirical Evidence

Our experiments (see Table 2) demonstrate that pretraining on graph reasoning tasks leads to significant improvements not only on graph-specific benchmarks, but also on general reasoning tasks (mathematics, logic, code, and commonsense). This empirically validates the theoretical breadth and depth argument above.

Summary:

• Graph reasoning uniquely combines all major reasoning paradigms and introduces new ones.
• Its intrinsic difficulty compels models to develop transferable, high-level reasoning skills.
• Our results show that improvements in graph reasoning translate to broad gains in general reasoning.

Thank you for prompting us to clarify this central contribution of our work.

[1] Chen et al. Take the bull by the horns: Hard sample-reweighted continual training improves llm generalization. ArXiv preprint 2024.

---

Q2: Distinguishing GraphPile from Existing Graph Reasoning Datasets

You raise a crucial point—comparing our dataset with other graph reasoning datasets would provide a more comprehensive validation. We summarize the key differences between GraphPile and representative datasets such as GraphWiz, GraphInstruct, InstructGraph, and GraphArena in the table below (focusing on graph problem reasoning tasks):

CoT: Chain-of-Thought, PoT: Program-of-Thought, ToE: Trace-of-Execution, CPT: Continue-Pretraining.
Simple Answer refers to providing only the answer to a given problem without any intermediate reasoning or explanation.

Key advantages of GraphPile:

1. Graph Diversity:
   GraphPile uniquely combines both synthetic and real-world graphs, whereas prior datasets include only one type. This diversity exposes models to a wider range of scenarios and enhances generalization.

2. Problem-Solving Paradigms:
   GraphPile is the first dataset to systematically include multiple reasoning paradigms—Chain-of-Thought (CoT), Program-of-Thought (PoT), and Trace-of-Execution (ToE)—enabling more robust and systematic problem-solving. In contrast, existing datasets are limited to simple answers or CoT.

3. Scale:
   GraphPile is orders of magnitude larger, containing 2.68 million samples and over 10.9 billion tokens, compared to tens of thousands in previous datasets.

4. Task Coverage:
   With 24 types of graph problems, GraphPile covers a much broader spectrum of reasoning tasks, including logical, numerical, enumerative, and topological reasoning, while most prior datasets include fewer than 10 task types.

5. CPT Compatibility:
   Thanks to its scale and diversity, GraphPile is the first dataset specifically designed for continue-pretraining. This enables LLMs to acquire deep and transferable graph reasoning abilities. Previous datasets, due to their limited size and diversity, are not suitable for effective CPT and thus cannot support broad generalization.

In summary, GraphPile stands out by offering greater diversity, richer reasoning paradigms, larger scale, and CPT-oriented design. These advantages make it uniquely suited for developing more robust and generalizable reasoning models. Thank you again for your helpful suggestion, which allows us to highlight the contributions of GraphPile more clearly.

---

Q3: Including Base Model Performance in Table 4

Thank you for your valuable suggestion. We agree that including the performance of the base model in Table 4 will help clarify the contribution of each component in GraphPile. We will add these results in the revised version to provide a more comprehensive and transparent comparison.

Q3: Statistical Significance Tests for Table 2

Thank you for your suggestion regarding statistical significance tests for the improvements in Table 2. We appreciate your attention to the rigor of our results.

In our main experiments, the results are deterministic: we set the temperature to 0, so the model produces identical outputs for the same inputs across multiple runs. Consequently, there is no variance, and statistical significance testing is not applicable. This determinism stems from the role of the temperature parameter, which directly controls output diversity—higher temperatures yield more varied and creative responses, while lower temperatures result in more conservative and consistent outputs. When the temperature is set to 0, the LLM generates the same output for identical inputs every time.

To further evaluate the robustness and statistical significance of the improvements, we conduct additional experiments by varying the temperature (0, 0.3, 0.6, 0.9). For each reasoning domain, we select one representative dataset: GSM8K (math), CLRS (code), RuleTaker (logical reasoning), and GraphInstruct (graph reasoning). The results for Llama3-8B and its continued-pretrained version (GraphMind-Llama3) are shown below:

Full Results Across Temperatures

Statistical Significance Analysis

We further compute the mean, mean difference, and p-value (using an independent-sample t-test across the four temperature settings) for each dataset:

As shown, GraphMind-Llama3 consistently outperforms the base Llama3 model on all datasets. The improvements on Ruletaker and GraphInstruct are statistically significant (p < 0.05), confirming the reliability of the observed gains. On GSM8K and CLRS, although GraphMind-Llama3 achieves higher mean performance, the difference is not statistically significant, mainly due to the large variance across different temperature settings, which results in higher p-values.

Overall, these results demonstrate that GraphMind’s improvements are robust across both deterministic and stochastic decoding conditions, and in several cases are statistically significant according to standard significance tests.

Thanks a lot for conducting the additional experiments.

However, I suspect the correctness of the results. According to https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, the 1.5B model can achieve 80% on MATH-500, which is a much more difficult dataset than GSM. The reported 56.1 on GSM8K is indeed concerning to me. Also, MMLU-STEM of 31.8 and SAT of 40.8 also seems like a severe under-reporting to me. I believe that you didn't force the <thinking> token in the prompt template.

I would strongly recommend re-doing the evaluation of the baseline models.

Re-evaluation on the Baseline Model

We appreciate your careful review and for bringing these issues to our attention. In our initial evaluation, we overlooked the unique characteristics of reasoning models and maintained the same settings as those used for non-reasoning models. We have re-evaluated the base versoin of DeepSeek-R1-Distill-Qwen-1.5B on GSM8K, MMLU-STEM, and SAT using OpenCompass—a well-known and widely used platform for evaluating LLMs—with the official DeepSeek-R1 settings.

• We adopt a sampling strategy to mitigate repetitions that can occur with greedy decoding, setting temperature=0.6, top_p=0.95, and max_new_tokens=32,768.
• For each dataset, we conduct 8 evaluation runs.
• For answer validation, we utilize LLM-based verification to reduce misjudgments from rule-based evaluation.

The updated results are as follows:

Compared to our previous evaluation, the performance of base model is indeed improved under this official setting. We thank you again for your valuable feedback, which has helped us enhance the rigor and accuracy of our evaluation.

We sincerely appreciate the reviewer's thoughtful engagement with our work. We would like to clarify several critical aspects regarding the experimental results and model selection rationale that directly address the concerns about "mixed outcomes":

1.The goal of Graphpile

First and foremost, our dataset is specifically designed for continued pretraining (CPT). In standard CPT settings, the goal is to further improve base models—that is, models that have only undergone general pretraining, without any domain-specific supervised fine-tuning. This is precisely the setup where our dataset shows its greatest strength.

2. Model-Specific Performance Clarification

While acknowledging the DeepSeek-r1-Distill-1.5B results may appear suboptimal, we emphasize that our methodology was primarily validated on base models without prior specialized training (Llama3/3.1 & Qwen-2.5-Coder series). The Qwen-2.5-Coder experiments demonstrate statistically significant improvements:

• Graph Reasoning : +20.0 absolute gain across both 1.5B & 7B variants

• General Reasoning : +16.0 (1.5B) and +6.3 (7B) improvements These results strongly validate our dataset's effectiveness for base model pretraining, particularly in knowledge acquisition rather than overfitting to specific benchmarks.

3. DeepSeek-Distill Analysis

As for the relatively weaker performance on the ds-distill models, we believe this is primarily due to the fact that these models are not standard base models. Instead, they have undergone extensive supervised fine-tuning (SFT) on large-scale distillation datasets during their post-training phase—many of which are heavily focused on math and code, and possibly even overlap significantly with specific benchmarks (raising concerns of potential overfitting). Applying CPT on such models is fundamentally different: it often results in catastrophic forgetting [1][2][3] of previously memorized task-specific patterns, especially when the CPT data (like ours) is not specifically aligned with the model's prior posttraining domain.

Therefore, the underwhelming performance on the distill model does not contradict the effectiveness of our dataset—it simply reflects the fact that CPT is not well-suited for models that have already been heavily fine-tuned on narrow, domain-specific objectives. Our dataset was not designed for that use case, and we do not claim it excels there.

We hope this clarifies the intended use case and the demonstrated effectiveness of our dataset under the standard CPT setting on base models.

References:

[1] Jiang et al. Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning, in ICLR 2025.

[2] Dou et al. Loramoe: Revolutionizing mixture of experts for maintaining world knowledge in language model alignment. Arxiv preprint 2023.

[3] Chen et al. How is chatgpt’s behavior changing over time? Arxiv preprint 2023.