SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning

Dear Reviewer,

We sincerely thank you for your thorough review and valuable feedback. Your insightful comments will help us significantly improve the quality of our paper.

1. For Missing Competitive Baselines in Figure 2 (In response to Q1)

• Our Response: We sincerely thank you for this valuable suggestion. In Figure 2, we mainly highlight the advantages of SQL-R1 over the NL2SQL method using different scale-based models, so all the compared models are not fully presented. In Table 1, we highlight the above baselines (including Alpha-SQL and SQL-o1) and their related information, and compare them. We really appreciate your valuable comments and will modify and supplement them in the revised version.

2. For Selective Dataset Analysis (In response to Q2)

• Our Response: We appreciate your question about this place. We must clarify that the models of all scales of SQL-R1 use the SynSQL-Complex-5K dataset for the RL phase. Therefore, all the results are dependent on this dataset. Also, we strongly agree with your issues regarding RL training on the default dataset. However, the original dataset contains 2.5M pieces of training data, which would be a very expensive and time-consuming challenge for RL training (more than 2 months). We regret not being able to complete the above training and provide corresponding results now.

• Additionally, we note that the proposed work aims to use a small amount of synthetic data to stimulate the model's NL2SQL deep reasoning ability. The data cost is also one requirement for the work's efficient operation. We will fully consider your comments and continue improving this area's work thereafter.

3. For the Performance Analysis by SQL Complexity (In response to Q3)

• Our Response: We appreciate your question about this place. We conducted a statistical analysis of the accuracy rates for various difficulty levels based on the BIRD development dataset, as presented in the table below:

• The results displayed show that SQL-R1 maintains a notable accuracy advantage over the current baseline across the three subsets: Simple, Moderate, and Challenging. This observation indicates that SQL-R1 exhibits a universally beneficial capability in reasoning and generation across different levels of SQL generation tasks.

• Moreover, when we expanded SQL-R1 from a parameter size of 7B to 14B, the EX rates for the Simple and Moderate difficulty levels experienced a marginal change of only 0.3% and a decrease of 1.1%, respectively. However, it is essential to highlight the significant improvement in the Challenging difficulty category, where the accuracy surged from 51.0% to 56.5%, with an increase of 5.5 percentage points. This finding suggests that augmenting model capacity primarily alleviates the generalization bottleneck associated with high-difficulty queries.

• This phenomenon corroborates previous research findings, which assert that larger-scale models are more adept at capturing long-range dependencies, particularly when queries involve multiple table joins, nested subqueries, or complex aggregations. The above insights underscore the critical role of model size of SQL-R1 in enhancing performance on intricate SQL generation tasks.

4. For the Relationship to Recent RL Fine-tuning Effectiveness Debates (In response to Q4)

• Our Response: We thank the reviewers for pointing out the insightful connection between SQL-R1 and the recent study on complexity-dependent RL effectiveness. Below we provide some discussions:

• Complexity-Dependent Pattern Analysis: We appreciate this insightful connection to recent research on RL fine-tuning effectiveness. While SQL-R1 shows some similarities in complexity-dependent patterns, our analysis reveals certain unique characteristics that differ from previous findings:

• Computational Budget Impact: Based on our experimental results, we observe that increasing model capacity from 7B to 14B yields the highest relative gains (5.5% improvement) on challenging-complexity queries, while showing minimal impact on simple and moderate queries. This suggests that expanding computational resources primarily benefits the model's ability to handle more complex SQL generation tasks involving intricate joins, nested queries and aggregations.

• In summary, SQL-R1 exhibits performance that is partially consistent with the above studies. For NL2SQL tasks, RL training can effectively improve the model's ability to solve higher complexity problems and trigger deeper thinking and reasoning. However, overstrengthening the ability to think about complex problems also leads to the risk of overthinking the model on simple problems. Balancing the above preferences during training will help the model to make more accurate judgments and draw inferences.

5. For the issue of Insufficient Baseline Analysis (In response to W2)

• Our Response: We sincerely appreciate your insightful question regarding baseline analysis. We are grateful for the opportunity to provide a comprehensive response to this important point.

• Comparsion with workflow-based NL2SQL methods: In contrast to various multi-module workflow paradigms such as DIN-SQL, DAIL-SQL, MAC-SQL, SuperSQL, MCTS-SQL, OpenSearch-SQL and CHASE-SQL, SQL-R1 introduces a streamlined end-to-end decoding strategy centered on self-consistency, which mitigates the necessity for explicit task decomposition, complex schema linking, the generation of intermediate representations, debug processes, and Monte-Carlo tree searches. Traditional pipeline-oriented systems predominantly depend on proprietary LLMs like GPT-4/4o and Gemini-1.5-Pro, which typically necessitate multiple iterations for refining SQL skeletons and often entail using external tools for schema retrieval and execution feedback. This reliance results in substantial computational overhead and heightened engineering complexity. Conversely, by leveraging a single open-source backbone, SQL-R1 achieves competitive accuracy metrics of 87.6% and 88.7% on the Spider development and test sets, respectively, and 66.6% on the BIRD development set. This approach simplifies deployment and diminishes token costs while upholding a robust performance profile.

• Comparsion with SFT-based NL2SQL methods: SQL-R1 marks a significant advancement beyond traditional reliance on SFT methods, such as CodeS, DTS-SQL, CHESS, and OmniSQL. While these SFT-based approaches demonstrate strong memorization of common SQL patterns—reflected in their Spider test scores ranging from 79.4% to 88.9%—they often struggle with generalization on more challenging benchmarks like BIRD, where scores range from 54.6% to 67.1%. To overcome this limitation, SQL-R1 integrates self-consistency reranking during decoding, effectively bridging the performance gap between in-domain and cross-domain tasks. We will add these detailed explanations to the revised manuscript.

We are deeply grateful to the reviewer for the constructive feedback. We have carefully addressed each concern with detailed responses and additional analyses. We hope these clarifications adequately address your questions and will incorporate all suggestions in our revised manuscript. We greatly appreciate your help in improving this work.

Sincerely,

The Authors

Dear Reviewer YmQu,

We sincerely appreciate you for the raised score and the helpful review that enabled us to enhance the discussion of our work and strengthen our experimental analysis.

Sincerely,

The Authors

Dear Reviewer,

We sincerely thank you for your detailed review and constructive feedback. We will carefully address each of your questions and concerns with detailed clarifications and responses.

1. Distinction from Reasoning-SQL and Originality of SQL-R1 (In response to W1)

• Our Response: Thank you for your questions and suggestions for novelty。 We need to unequivocally clarify that SQL-R1 and Reasoning-SQL are concurrent works, and we publicly released our research in early April. It is crucial to acknowledge that Reasoning-SQL should not diminish our novelty or impose a requirement for us to compare ourselves with Reasoning-SQL.

• Additionally, our work is also significantly different from Reasoning-SQL: first, our Reward function design is not consistent, and we achieve better execution accuracy without the complications of Schema filtering. Furthermore, we take our work seriously by training SQL-R1 entirely on synthetic data, which sets it apart from Reasoning-SQL that relies on the training set associated with BIRD and Spider. Our method demonstrates much stronger generalization capabilities. These fundamental differences undeniably confirm that SQL-R1 stands as an independent contribution, not a mere re-implementation.

• In summary, SQL-R1 represents a novel contribution with its unique reward design and synthetic data training approach. While developed concurrently with Reasoning-SQL, our open-source implementation and distinct methodology demonstrate SQL-R1's independent value and advancement of NL2SQL capabilities.

2. Issue of the Experimental Results (In response to Q2 and W3)

• Our Response: Thank you for your question. For Q2, we expound relevant results in accordance with the article report of Reasoning-SQL. We take this matter very seriously and feel it is crucial to address it transparently. Therefore, we open source the code of SQL-R1, which is also the originality of our work at the experimental level. In contrast, Reasoning-SQL does not open source the relevant code, which significantly complicates our ability to directly verify the experimental results of its report. As a result, we adhere strictly to the original description and results provided in the article.

• For W3, we are preparing to submit the results, but the relevant tests need to wait for the official conduct. According to the existing results, our confidence interval of the reported results is in [70,73]. Relevant results are added to the revised version when they are ready.

3. Latency of SQL-R1 with Self-Consistency (In response to W2)

• Our Response: This is an excellent suggestion for the cost. We evaluated the latency of SQL-R1 on the BIRD development set with the same inference setting mentioned in the Section 3.1:

• Compared with the greedy search method (1 candidate and temperature=0), the self-consistency method with 8 candidates adds only an average of 0.7 seconds on each query while improving execution accuracy by 2.9%, which represents a favorable trade-off. Additionally, the self-consistency method employed in the proposed SQL-R1 is consistent with that of end-to-end NL2SQL models.
• In comparison to CHESS (61.5%), which employs a multi-component workflow, the end-to-end SQL-R1 (66.6%) demonstrates not only superior execution accuracy but also significant advantages in latency and token usage efficiency. We will add these crucial comparative results to the revised manuscript.

4. Issue of Figure 9 (In response to Q1)

• Our Response: We appreciate your question. The results in Figure 9 are based on the 7B model tested without SFT. In the experiment section, we mentioned that SQL-R1-7B performs relatively well after SFT + RL, so its highest execution accuracy reaches 66.6. However, in the ablation experiment and the rest of the control, we used the model without SFT cold start for the convenience of comparison, so t. So, the highest accuracy of SQL-R1-7B here was 63.1, which was directly trained by RL. We will add these crucial comparative results to the revised manuscript.

5. Discussion of the reward function design (In response to Q3)

• Our Response: We appreciate your question. The weight of the reward function is designed by integrating its importance to NL2SQL and the perspective of preventing reward hacking. For the mentioned w/o $S_f$ ablation experiment, we modified the corresponding logic so that the model directly evaluated the executability, which easily led to execution errors when the model could not correctly extract SQL, thus obtaining invalid negative feedback and reducing the performance due to the impact of the sr function problem. We explain that this reward is sparse, and when facing more difficult samples in training, the model has less expectation to obtain this reward successfully. Hence, its influence on the model is relatively small. This is also an area for improvement, but it has undeniably contributed to the growth of the model's capabilities.

6. Discussion of length reward (In response to Q4)

• Our Response: We appreciate your question. The length reward is a smoothness reward designed to encourage the model to think deeper and try different ways of thinking. Figures 3 and 4 are good examples of qualitative comparison. The length reward helps the model navigate longer thought paths, resulting in a top-down thought process. Without the length reward, the model's ability to generate difficult samples will be weakened.

We sincerely appreciate all the reviewers' constructive feedback and questions. We have provided detailed responses to address the concerns about SQL-R1's originality, experimental results, latency analysis, and reward function design. We have demonstrated that SQL-R1 is a concurrent and independent work with unique contributions, including novel reward design and a synthetic data training approach. The comprehensive experimental results and analyses validate the effectiveness and efficiency of our method. We are committed to maintaining transparency by open-sourcing our code and will continue improving our work based on the valuable suggestions received. These clarifications and additional analyses strengthen our paper's contribution to the NL2SQL research community.

Sincerely,

The Authors

Dear Reviewer Bp1S,

We hope this message finds you well. As the deadline for discussions is fast approaching, we want to take a moment to express our sincere gratitude for your valuable review on our submission. Your insights are immensely appreciated, and we strive to ensure that our response has effectively addressed all your comments and major concerns.

If you could confirm whether our rebuttal have met your expectations or if there are further aspects you would like us to clarify, it would be tremendously helpful.

Thank you once again for your time and expertise. We look forward to hearing from you soon.

Sincerely,

The Authors

Dear Reviewer,

We appreciate your detailed review and constructive feedback. Your positive assessment and valuable suggestions will help enhance our paper's clarity.

1. Evaluation on Spider2.0 and generalisation evidence (In response to Question 1 and Question 2)

• Our Response: Thanks for the crucial questions. To address these concerns, we firstly evaluate SQL-R1 on Spider 2.0-SQLite subset . All models use the same self-consistency setup (n=8).

• The findings presented above demonstrate that SQL-R1 surpasses existing methodologies in addressing Spider2.0 complex tasks related to NL2SQL. This advantage can be ascribed to the model's enhanced inferential capabilities, which contribute to its overall efficacy in conducting sophisticated scenario analyses. Regarding variations in dialects such as PostgreSQL and MySQL, additional time is required to perform thorough testing on Spider2.0. We assure that supplementary results will be provided once they are finalized.

• For generalisation of the SQL-R1, Table 4 shows execution accuracy on different NL2SQL tasks, including Spider-Realistic, Spider-DK and Spider-Syn.

• The findings presented above demonstrate that SQL-R1 surpasses existing methodologies in addressing Spider2.0 complex tasks related to NL2SQL. This advantage can be ascribed to the model's enhanced inferential capabilities, which contribute to its overall efficacy in conducting sophisticated scenario analyses. We will add these detailed explanations to the revised manuscript.

2. Algorithmic Novelty (In response to Weakness 1)

• Our Response: We appreciate your question. We now provide a more detailed explanation of the algorithmic novelty versus reuse. We acknowledge that our work draws inspiration from DeepSeek-R1, which is also mentioned in the Related Work section. However, we clarify that some task-specific redesigns presented here are novel and essential for advancing the framework.

  1. We have adapted the GRPO algorithm for the NL2SQL task, introducing a unique combination of reward functions specifically designed for this application, including execution and result rewards. This innovative approach enhances the model’s ability to generate accurate SQL queries from natural language input. We have customized and modified some task-specific principles (e.g, schema linking, database content retrieval) to better align with the specific requirements and challenges of the NL2SQL task.

  2. In this work, we utilized limited synthesized complex samples as training inputs for reinforcement learning (RL) to encourage the model to autonomously explore more robust reasoning pathways and inferential capabilities in NL2SQL translation. This approach contrasts with the conventional methodologies that typically employ training sets derived from specific domain databases, such as the Spider and BIRD datasets. Our findings indicate that synthesized data from diverse sources can significantly enhance the model's inferential strength and generalization ability, thereby contributing to the advancement of NL2SQL systems.

• We hope that these explanations will help to address your concerns. And

3. Error Analysis by Complexity and Reward Sensitivity (In response to Weakness 2)

• Our Response: We sincerely appreciate your insightful question regarding the performance analysis across different complexity levels.

• Complexity Analysis: We conducted a statistical analysis of the accuracy rates for various difficulty levels based on the BIRD development dataset, as presented in the table below:

• The results displayed show that SQL-R1 maintains a notable accuracy advantage over the current baseline across the three subsets: Simple, Moderate, and Challenging. This observation indicates that SQL-R1 exhibits a universally beneficial capability in reasoning and generation across different levels of SQL generation tasks.

• Moreover, when we expanded SQL-R1 from a parameter size of 7B to 14B, the EX rates for the Simple and Moderate difficulty levels experienced a marginal change of only 0.3% and a decrease of 1.1%, respectively. However, it is essential to highlight the significant improvement in the Challenging difficulty category, where the accuracy surged from 51.0% to 56.5%, with an increase of 5.5 percentage points. This finding suggests that augmenting model capacity primarily alleviates the generalization bottleneck associated with high-difficulty queries.

• This phenomenon corroborates previous research findings, which assert that larger-scale models are more adept at capturing long-range dependencies, particularly when queries involve multiple table joins, nested subqueries, or complex aggregations. The above insights underscore the critical role of model size of SQL-R1 in enhancing performance on intricate SQL generation tasks.

• Reward Sensitivity Analysis: We conducted experiments on the BIRD development set by adjusting the weights of different reward components to evaluate their impact on model performance, using execution accuracy as the evaluation metric.

• The sensitivity analysis results reveal several key insights about the reward function design:

  1. Setting Execution and Result rewards to minimal weights leads to significant performance degradation, indicating potential reward hacking behavior where the model learns to exploit simpler patterns rather than developing robust reasoning capabilities.

  2. Increasing reward weights by large margins destabilizes training, particularly during early policy exploration phases. The model may receive extreme rewards too frequently before establishing stable reasoning patterns.

  3. The lower sensitivity to Result reward adjustments suggests that while the model successfully explores executable SQL queries, semantic gaps between questions persist. This makes it challenging for the model to achieve high Result rewards, even with increased weights consistently.

  4. A fixed Length reward proves suboptimal for encouraging deep reasoning. The smooth sensitivity design we adopted maintains meaningful differentiation while avoiding training oscillations. However, excessive Length rewards can also trigger reward hacking, where the model generates unnecessarily verbose solutions.

• These findings validate our balanced reward function design choices that promote stable learning while preventing various forms of reward exploitation. The results demonstrate the importance of careful reward calibration in reinforcement learning for complex NL2SQL tasks.

We sincerely thank the reviewer for the constructive feedback and suggestions. We have provided detailed responses to address your concerns about algorithmic novelty, error analysis, and experimental results. We will incorporate all these clarifications and additional analyses in the revised manuscript.

Sincerely,

The Authors

Dear Reviewer,

We appreciate your detailed review and constructive feedback. Your positive assessment and valuable suggestions will help enhance our paper's clarity.

1. Latency of SQL-R1 with Self-Consistency (In response to Question and Weakness 2)

• Our Response: This is an excellent suggestion for the cost. We evaluated the latency of SQL-R1 on the BIRD development set, with the same inference setting and evaluation metric (Execution Accuracy, EX) mentioned in Section 3.1:

• Compared with the greedy search method (1 candidate and temperature=0), the self-consistency method with 8 candidates adds only an average of 0.7 seconds on each query while improving execution accuracy by 2.9%, which represents a favorable trade-off. Additionally, the self-consistency method employed in the proposed SQL-R1 is consistent with that of end-to-end NL2SQL models.
• In comparison to CHESS (61.5%), which employs a multi-component workflow, the end-to-end SQL-R1 (66.6%) demonstrates not only superior execution accuracy but also significant advantages in latency and token usage efficiency. We will add these crucial comparative results to the revised manuscript.

2. Robustness on Complex NL2SQL Tasks and Cross-dialect Generalization (In response to Weakness 1)

• Our Response: Thank you for suggesting the evaluation of SQL-R1 on the complex task related to NL2SQL. We now evaluated SQL-R1 on Spider2.0-SQLite subset, with the same inference setting and evaluation metric (Execution Accuracy, EX) mentioned in Section 3.1:

• The results presented above demonstrate that SQL-R1 outperforms existing methodologies in handling complex NL2SQL tasks. This advantage can be attributed to the model's superior inference capabilities, which enhance its overall effectiveness in intricate scenario analyses.
• As for different dialects like PostgreSQL and MySQL, we currently need more time to conduct comprehensive testing on Spider2.0. We assure that we will add more results in the revision version once they are ready.

[1] Li, Haoyang, et al. "Omnisql: Synthesizing high-quality text-to-sql data at scale." arXiv preprint arXiv:2503.02240 (2025).

3. Robustness to Real-world Noise and Multi-table Generalization (In response to Weakness 3)

• Our Response: Thanks for your concern on the real-world noise. Table 4 presents evaluations conducted on the Spider-DK, Spider-Syn, and Spider-Realistic benchmarks, which include synthetic and real-world noise. We present a snapshot of the results in Table 4 below:

• This results demonstrate that the methodology proposed in this paper consistently outperforms baseline methods while maintaining robust performance in challenging environments. Notably, SQL-R1 is trained entirely on synthetic data, yet the results in Table 1 and the aforementioned findings indicate its impressive resilience and effectiveness without the necessity for additional fine-tuning on other domain databases.

• Additionally, we acknowledge that there remains room for improvement. For instance, exposing the model to more accurate filtered schemas could facilitate independent exploration and enhancement of its reasoning capabilities, particularly in complex table environments. Such approaches could lead to significant advancements in model performance when tackling diverse database contexts. Therefore, while our results are promising, ongoing research and experimentation in these areas could further elevate the robustness and applicability of our findings in real-world scenarios.

4. Novelty of SQL-R1 (In response to Weakness 4)

• Our Response: Thank you for your suggestion on the novelty of SQL-R1. We now provide a more detailed explanation of the novelty of SQL-R1.

  • We have adapted the GRPO algorithm for the NL2SQL task, introducing a unique combination of reward functions specifically designed for this application, including execution and result rewards. This innovative approach enhances the model’s ability to generate accurate SQL queries from natural language input. We have customized and modified some task-specific principles (e.g, schema linking, database content retrieval) to better align with the specific requirements and challenges of the NL2SQL task.

  • In this work, we utilized limited synthesized complex samples as training inputs for reinforcement learning (RL) to encourage the model to autonomously explore more robust reasoning pathways and inferential capabilities in NL2SQL translation. This approach contrasts with the conventional methodologies that typically employ training sets derived from specific domain databases, such as the Spider and BIRD datasets. Our findings indicate that synthesized data from diverse sources can significantly enhance the model's inferential strength and generalization ability, thereby contributing to the advancement of NL2SQL systems.

We believe that these additions and discussions will greatly enhance the evaluation of the paper by providing essential comparative data and context regarding costs. We appreciate the reviewer’s constructive feedback, which aids us in improving the rigor of our study. We hope that the revised manuscript will meet the standards for acceptance.

Sincerely,

The Authors