Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL

We would like to thank the reviewer for their constructive feedback. In response to your comments, we conducted the following experiments, which we hope adequately address your concerns regarding the paper:

1. We appreciate the reviewer’s thoughtful observation regarding the instability of RL training. To clarify, Table 1 in our paper reports results for five different runs with each of our partial rewards, each initialized from the same base policy. Each of our proposed partial rewards consistently led to performance improvements over the SFT baseline, indicating that the observed gains are robust and reproducible. Furthermore, to better demonstrate the effectiveness of our RL approach, we conducted an additional experiment where we applied our full reward design on top of the SFT policy model. This led to substantially higher performance, reinforcing the value of our composite reward strategy and its ability to amplify model performance beyond both the SFT and RL-only setups:

2. To better demonstrate the performance gains achieved by combining rule-based rewards with the LLM-as-a-judge reward, we conducted an experiment comparing the Majority@k performance of a model trained with our full RL reward composition to that of a model trained using only the LLM-as-a-judge reward. The results show consistent improvements across all values of k with almost 2% gain, highlighting the importance and effectiveness of the composite reward design in enhancing overall model performance:

https://anonymous.4open.science/r/REASONING-SQL-REBUTTAL-A664/majority@k.png

Regarding your question about the database schema, we would like to clarify that the database schema is indeed an integral part of the prompt and is explicitly included within it. As mentioned with the following format in the prompts in appendix section:

[Table creation statements]
 {DATABASE_SCHEMA}

We’re writing to kindly follow up on our responses to your valuable comments. If you haven’t had a chance yet, we would greatly appreciate it if you could take a moment to review our responses and, if appropriate, update your scores accordingly.

Thank you again for your time and valuable feedback throughout this process. Please don’t hesitate to let us know if any further clarification is needed from our side.

We would like to thank the reviewer for their constructive feedback.

1. We agree with the reviewer that access to gold SQL queries is required for computing the rewards. However, alternative approaches such as test case generation from natural language inputs—as proposed in [1]—or synthetic SQL generation, as demonstrated in Omni-SQL [2], can serve as auxiliary methods for SQL verification. These techniques offer promising directions for reducing dependency on gold queries during evaluation or training.

To address your question regarding the effectiveness of using the SFT baseline as an initialization point, we conducted an additional experiment using Qwen 2.5 7B STaR SFT as the policy model to initialize training. The results, presented in the table below, show a clear performance gain on the BIRD dev set, demonstrating the effectiveness of our approach in boosting performance beyond both the SFT baseline and RL-only training:

[1]: Using LLM to select the right SQL Query from candidates [2]: Omnisql: Synthesizing high-quality text-to-sql data at scale

We would like to thank the reviewer for their constructive feedback. In response to your comments, we conducted the following experiments, which we hope adequately address your concerns regarding the paper:

1. To ensure a fair comparison, all performance results reported in Table 2 are obtained in the zero-shot chain-of-thought (CoT) setting, using user-defined instructions adapted from the prompt proposed by CHASE-SQL, one of the current state-of-the-art methods in text-to-SQL. While it is true that the base model demonstrates strong performance in its 14B version, the 7B version initially performs at 58.73, which is lower than both o3-mini and Gemini 1.5 Pro. However, after applying our proposed reinforcement learning (RL) training, the performance of the 7B model improves significantly to 64.01, outperforming all closed-source models except Gemini 2.0 Flash. This result supports our claim that the proposed reward design and training method effectively boost performance beyond models like o3-mini and Gemini 1.5 Pro. Additionally, for the results with the filtered schema setting, the base 14B model performs worse than o3-mini but after RL-training the model achieves 4% performance gain and outperformed all of the strong base models.

2. We agree with the reviewer that schema linking is one of the most challenging sub-tasks in the text-to-SQL pipeline. However, we would like to clarify that the purpose of including perfect schema linking in our study was to demonstrate the theoretical upper bound achievable by our method under ideal conditions. As shown in Table 1 (EX column), our results using an imperfect schema linking method already exhibit improved performance over the baselines, reinforcing the effectiveness of our overall reward design.

3. We would like to thank the reviewer for highlighting this important experimental consideration. To address your question, we conducted an experiment comparing CHASE + Qwen-7B base, the SFT baseline, and our proposed Reasoning-SQL method. The results show that Reasoning-SQL consistently outperforms the other approaches, primarily due to its higher average@k performance, as detailed in Appendix A.4.

5. As noted in the DeepSeek R1 [5] paper and other recent works [4], training with GRPO does not improve the model’s pass@k performance beyond its pre-trained baseline. However, it can enhance average@k or majority@k performance by improving pass@1 and the overall quality of generated responses. While this may slightly reduce the model's theoretical upper bound (i.e., maximum possible pass@k), in practice, achieving that upper bound is highly challenging. In contrast, improving the average quality of responses is often more valuable—especially for recent text-to-SQL methods like XiYan-SQL and CHASE-SQL, which rely on query selection from a candidate pool. In these cases, the quality of that candidate pool (average@k) is crucial. This benefit is evident when comparing the performance of Reasoning-SQL 7B and the SFT baseline within the CHASE-SQL pipeline, as demonstrated below. Further supporting evidence can also be found in Appendix A.4, where the RL-trained model shows improved average@k performance.

[1]: Din-sql: Decomposed in-context learning of text-to-sql with self-correction [2]: Codes: Towards building open-source language models for text-to-sql [3]: A Study of In-Context-Learning-Based Text-to-SQL Errors [4]: Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? [5]: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

We would like to thank the reviewer for their constructive feedback. In response to your comments, we conducted the following experiments, which we hope adequately address your concerns regarding the paper:

1. The reward design proposed in our work builds upon valuable insights from prior studies in the text-to-SQL domain. As highlighted in previous works such as DIN-SQL [1], codeS [2], and especially MapleRepair [3], errors in text-to-SQL generation are typically categorized into syntax errors, schema errors, semantic errors, logic errors, convention errors, and gold SQL type errors. Our reward components are specifically designed to target each of these error categories, assigning partial rewards based on the model’s correctness in each area. More specifically: The syntax check reward is designed to reduce syntax errors. The schema linking reward encourages accurate schema references, which account for a significant proportion of text-to-SQL errors. For semantic and logical errors, which require reasoning, we employ the LLM-as-a-judge reward. To address convention errors, our n-gram-based reward helps the model learn and adhere to standard SQL conventions. Although Table 1 of the paper includes an ablation study of the reward components, we have now added two additional experiments. In these, we trained the Qwen-coder 7B model using only the n-gram reward and only the schema linking reward, respectively. These experiments, presented below, further demonstrate the effectiveness of our proposed reward combination in improving overall model performance.

2. We agree with the reviewer that simpler rewards are generally harder to hack. However, the simplest reward design in the text-to-SQL domain—execution accuracy—has a significant limitation, as discussed in the paper. Its binary nature fails to provide informative feedback when a query is only partially correct. This limitation is particularly critical for policy optimization methods like GRPO, which depend on group-based reward signals. In cases where all queries receive either zero or one as a reward, the model receives little to no gradient signal for learning and improvement. Furthermore, we would like to highlight that Table 1 in our paper presents a direct comparison between execution accuracy as the simplest reward and our proposed composite reward design. Our approach achieves an execution accuracy of 64.01%, compared to 62.32% achieved by using execution accuracy alone, demonstrating the added value of our more nuanced reward strategy.

3. We would like to thank the reviewer for highlighting recent works in the field. We note, however, that a direct comparison with XiYan-SQL is already included in Table 3 of our paper. As for the Omni-SQL paper, it was released concurrently with our submission, and ThinkSQL was published after both our submission and the COLM conference deadline, making it impossible to include it in the original paper. Nevertheless, as per your suggestion, we included a direct comparison with both Omni-SQL and ThinkSQL on the BIRD development set, as shown in the following table, where our approach significantly outperform both of them: | Method | Execution acc | |---------------------|---------------| | Think2SQL 7B | 56.1 | | Omni-SQL 32B | 69.23 | | Reasoning-SQL 14B | 72.29 |

4. The primary goal of our approach in this paper is to propose a reward design strategy aimed at improving the model’s reasoning ability and execution accuracy, rather than to compare different policy optimization methods. We chose to use GRPO primarily because it has a lower memory footprint compared to PPO and has demonstrated strong performance on reasoning-intensive tasks such as MATH and code generation.

We’re writing to kindly follow up on our responses to your valuable comments. If you haven’t had a chance yet, we would greatly appreciate it if you could take a moment to review our responses and, if appropriate, update your scores accordingly.

Thank you again for your time and valuable feedback throughout this process. Please don’t hesitate to let us know if any further clarification is needed from our side.