Cypher-RI: Reinforcement Learning for Integrating Schema Selection into Cypher Generation

We sincerely thank the reviewer for their thorough evaluation and constructive feedback. The suggestions have been invaluable in helping us strengthen our work. In the following sections, we address each of the reviewer's points in detail, providing clarifications and new experimental results to support our claims and demonstrate the effectiveness of our method.

• Q1: Reward Function Design and Ablation.

  Our reward function is central to the reinforcement learning process, providing essential supervision signals where explicit intermediate annotations are unavailable. The function is designed to guide the model through the key stages of the Text-to-Cypher task by decomposing the total reward into three distinct components.

  The total reward is the sum of three parts: $r_{format}$, $r_{selection}$ and $r_{execution}$.

  Format Reward: This reward encourages the model to generate its reasoning (thought process) before providing the final Cypher query. This "think first, answer later" structure improves the model's logical flow and interpretability.

  $$r_{format} = \begin{cases} a, & \text{if the format is correct} \newline -a, & \text{if the format is incorrect} \end{cases}$$

  Schema Selection Reward: This reward is crucial for the core task of identifying the question related schema. It guides the model to select the right nodes, relationships, and properties needed to construct the query.

  $$r_{selection} = \begin{cases} 2.0, & \text{if the selected schema matches the gold answer} \newline b, & \text{if the selected schema does not match the gold answer} \newline -2.0, & \text{if the selected schema is unparseable} \end{cases}$$

  Execution Reward: This is the ultimate measure of success. The reward is based on whether the final generated Cypher query executes and produces the same result as the ground truth query.

  $$r_{execution} = \begin{cases} 2.0, & \text{if the generated Cypher executes consistently with the gold answer} \newline b, & \text{if the generated Cypher executes inconsistently with the gold answer} \newline -2.0, & \text{if the generated Cypher fails to execute} \end{cases}$$

  To determine the optimal values for the hyperparameters $a$ and $b$, we conducted the following ablation studies.

  1.Determining the Value of a

  Our hypothesis was that the format reward ($a$) should be substantial enough to enforce the desired output structure but smaller than the primary task rewards for selection and execution (which have a magnitude of 2.0). A value that is too high might cause the model to prioritize formatting over correctness. We tested values of $a\in\{0.5,1.0,2.0\}$.

  Methods	EX(%)	PSJS (%)	Exec. (%)
  Cypher_RI( $a=0.5$)	69.04	74.62	99.04
  Cypher_RI( $a=2.0$)	68.36	74.35	98.74
  Cypher_RI( $a=1.0$)	69.59	75.21	99.28

  The results confirm our hypothesis. Setting $a=1.0$ achieves the best performance across all metrics, striking the right balance between enforcing structure and prioritizing query correctness.

  1.Determining the Value of b

  Next, we investigated the appropriate penalty ($b$) for outputs that are syntactically valid but semantically incorrect (i.e., a parseable schema that is wrong, or an executable query that returns the wrong result). We theorized that a moderate penalty would be more effective than a penalty that is too lenient or too harsh. A moderate penalty discourages incorrect solutions while still recognizing that a syntactically valid attempt is better than a complete failure. We tested values of $b\in\{−1.0,−1.5,−2.0\}$.

  Methods	EX(%)	PSJS (%)	Exec. (%)
  Cypher_RI( $b=-1.0$)	68.02	74.10	98.43
  Cypher_RI( $b=-2.0$)	65.72	72.83	98.05
  Cypher_RI( $b=-1.5$)	69.59	75.21	99.28

  The experiment demonstrates that setting $b=−1.5$ yields the best performance. A lenient penalty of −1.0 is insufficient, while a harsh penalty of −2.0 (the same as for a complete failure) is detrimental. This result supports our strategy of providing a nuanced, moderate penalty for partially correct attempts.

• Q2: Discussion Sample Sizes of Training.

  To ensure a fair comparison in our main experiments, the sample size is identical for all methods: SFT, RL and our Cypher-RI. We used a dataset of 8,295 samples.

  This dataset was curated through a careful process to enhance training efficiency for RL: For each question in the source dataset, we used Qwen-2.5-Coder-7B to generate 8 candidate Cypher queries with schema selection. We then filtered these generations, keeping only those prompts where the 8 attempts were either all correct (mean accuracy = 1.0) or all incorrect (mean accuracy = 0.0).

  Inspired by the reviewer's question, we further investigated how performance scales with data size. We trained all three models on subsets of 2k, 4k, and our full ~8k dataset.

  Methods	EX(%)	PSJS (%)	Exec. (%)
  SFT-2k	36.43	39.27	87.22
  SFT-4k	40.84	44.53	91.78
  SFT-8k	44.34	49.80	94.34
  RL(w/o schema selection)-2k	49.19	53.92	89.82
  RL(w/o schema selection)-4k	55.20	60.03	93.38
  RL(w/o schema selection)-8k	58.35	64.55	96.12
  Cypher-RI-2k	58.73	65.21	93.78
  Cypher-RI-4k	65.20	70.29	97.74
  Cypher-RI-8k	69.59	75.21	99.28

  The results show that all methods improve as the number of training samples increases, though the gains are not substantial. We think this phenomenon might be related to the limited diversity and complexity of the training data, and improving these aspects could be a valuable direction for future work.

• W1: Integrating schema selection into SQL Generation.

  To prove that our core idea—explicitly integrating schema selection within an RL framework—is an important component for many query languages, we applied it to the Text-to-SQL domain. We adapted the existing SQL-R1 [1] framework by incorporating a schema selection step and a corresponding reward component ($S_c$), while keeping all other experimental conditions identical.

  The reward function in SQL-R1 is:

  $$S=S_f+S_e+S_r+S_l$$

  where the components represent rewards for format $S_f$, execution $S_e$, result correctness $S_r$, and query length $S_l$.

  The modified reward function is:

  $$S=S_f+S_e+S_r+S_l+S_c$$

  where the new schema selection reward $S_c$ is:

  $$S_c = \begin{cases} 3, & \text{if schema selection result is correct} \newline 0, & \text{if json format of selected schema is incorrect} \newline -3, & \text{if schema selection result is incorrect} \end{cases}$$

  We evaluated this enhanced framework on the Spider and BIRD benchmarks.

  Methods	Base Model	Spider(Dev)	Spider(Test)	BIRD(Dev)
  OmniSQL	Qwen2.5-Coder-7B	85.5	88.9	66.1
  DAIL-SQL	GPT-4	83.6	86.2	54.8
  MAC-SQL	GPT-4	86.8	82.8	59.4
  SQL-R1	Qwen2.5-Coder-7B	84.5	86.1	63.1
  Ours	Qwen2.5-Coder-7B	87.1	86.8	65.4

  Our approach yields consistent improvements, with a 2.6-point gain on Spider (Dev) and a 2.3-point gain on BIRD (Dev). These results confirm that explicitly rewarding schema selection within an RL loop is a robust and portable methodology that enhances performance in other major Text-to-Query systems.

  In summary, we introduce the first RL-based framework for the important Text-to-Cypher task, achieving SOTA performance with a practical, efficient model. Furthermore, our new experiments demonstrate that the core principle of our work is not a narrow, domain-specific trick, but a generalizable methodology that improves performance on other query languages and database types.

• W2: Ablation study on Neo4J-Text2Cypher dataset.

  For this new experiment, we trained SFT, RL(w/o schema selection), and Cypher-RI on a 4k-sample subset of the Neo4J-Text2Cypher training set.

  Methods	EX(%)	Google-BLEU (%)
  SFT	26.32	49.58
  RL(w/o schema selection)	30.81	63.29
  Cypher-RI	38.58	65.73

  The results clearly confirm the robustness of our method. Cypher-RI achieves an EX score of 38.58%, which is a significant improvement of ~7.8 points over RL(w/o schema selection) and ~12.3 points over SFT. This demonstrates that the framework of Cypher-RI provides a generalization advantage.

References:

[1] SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning

We sincerely thank the reviewer for their insightful feedback and positive evaluation of our work. Below, we address the questions and weaknesses raised.

• Q1: Experiments on Reasoning Models

  Our primary experiments were completed prior to the official release of the Gemini-2.5 series. We have now benchmarked our model against a suite of more powerful reasoning models on CypherBench.

  Methods	EX(%)	PSJS (%)	Exec. (%)
  Qwen3-8B	48.72	66.49	81.26
  Qwen3-32B	58.56	81.23	80.75
  DeepSeek-R1	60.01	73.52	96.68
  o3-mini	62.39	74.73	98.00
  Gemini 2.5-pro	74.14	81.70	99.45
  Cypher-RI	69.59	75.21	99.28

  Our 7B Cypher-RI model significantly outperforms much larger models, including the Qwen3-32B and DeepSeek-R1.

• Q2: Accuracy of Schema Selection.

  The accuracy of the initial schema selection step is a critical component of our model's success. The following table reports this accuracy on both benchmarks.

  Methods	CypherBench	Neo4j-Text2Cypher
  Qwen2.5-Coder-7B	76.45	73.98
  Cypher-RI	95.87	93.87

  Our Cypher-RI framework boosts schema selection accuracy by a remarkable ~20 percentage points over the base model. This confirms that the model effectively learns to identify the correct schema elements.

• Q3: Comparison of inference time.

  To assess practical usability, we benchmarked the inference latency of Cypher-RI against the GPT-4o API. Cypher-RI was deployed on a single NVIDIA A800 GPU using the vLLM inference server. The results below show the average time to process one question.

  Methods	Inference Time
  GPT-4o	2.79
  Cypher-RI	0.61

  Our locally-deployed 7B model is approximately 4.5 times faster than the GPT-4o API, offering state-of-the-art accuracy with significantly lower latency and computational cost, making it a highly practical solution.

• W1: Discussions about Reward design

  Our choice of a simple, rule-based reward function was a deliberate one, aimed at ensuring stable learning and preventing reward hacking. In complex generation tasks, learned or overly intricate reward functions can sometimes be exploited by the model to maximize the reward signal without actually improving performance on the intended task. A clear, rule-based function directly incentivizes the desired behaviors—correct format, valid schema selection, and accurate query execution—which is crucial for effective and targeted learning in an RL framework.

  Furthermore, our framework is not limited by this initial design; it is inherently flexible and can be generalized to incorporate other objectives. We have conducted a new experiment introducing a length-based penalty into our reward function to explore the trade-off between performance and inference cost. The modified reward is:

  $$r=r_{format}+(r_{selection}+r_{execution})*r_{length}$$ $$r_{\text{length}} = \begin{cases} 1, & r_{selection}+r_{execution}<=0 \\ \text{clip}(\alpha*(n_{max}-n_{y}),0,1), & r_{selection}+r_{execution}> 0 \end{cases}$$

  Here, $n_y$ is the number of tokens generated, and $n_{max}$ is a hyperparameter setting the desired token limit. This allows us to explicitly guide the model to produce reasoning chains of varying lengths.

  The results on CypherBench for models trained with different $n_{max}$ values are below:

  Methods	Inference Tokens	EX(%)	PSJS (%)	Exec. (%)
  Qwen2.5-Coder-7B	53	13.12	22.65	62.95
  Cypher_RI($n_{max}=256$)	223	45.78	49.65	91.93
  Cypher_RI($n_{max}=512$)	436	60.78	64.29	96.93
  Cypher_RI($n_{max}=1024$)	901	66.99	71.73	98.93
  Cypher_RI($n_{max}=1280$)	1045	67.46	74.39	99.11
  Cypher_RI(w/o $n_{max}$)	1482	69.59	75.21	99.28

  By incorporating the length-related reward term $r_{length}$ and adjusting the parameter $n_{max}$, we can effectively control the model’s output length. This demonstrates that, although simple, our reward design is not rigid; rather, it is modular and extensible, enabling the integration of additional objectives such as computational efficiency or domain-specific constraints.

• W2: Relies on Existing RL Technique

  We agree with the reviewer that our work employs an established RL algorithm (GRPO) as its optimization engine. We believe this is a strength, as it grounds our method in a proven, stable technique.

  Our core contribution is not a new RL algorithm, but rather a novel framework for the Text-to-Cypher task. The originality lies in formulating schema selection as an explicit, learnable step within the model's reasoning process. We designed a custom training template, rollout mechanism, and reward function specifically to enable the model to learn this complex, multi-stage task without direct supervision on reasoning traces.

  The effectiveness of our approach is demonstrated by strong empirical performance, including outperformance of significantly larger models. This validates the originality and impact of our formulation. Additionally, to highlight the generality of our methodology beyond graph databases, we successfully apply our framework to the Text-to-SQL task, as elaborated in our response to Reviewer W3.

• W3: Expand scope of our method

  To demonstrate the broader applicability of our work, we have conducted two new sets of experiments.

  1. Generalization to a New Graph Query Language (Gremlin)

  We evaluated our Cypher-trained model on a public Gremlin dataset [1] to test its zero-shot, cross-language generalization. The model was not fine-tuned on Gremlin.

  Methods	EX(%)
  Qwen2.5-Coder-7B	60.73
  Text2Cypher-Gemma-3-27B	70.09
  Qwen3-32B	73.41
  GPT-4o	80.06
  Gemini 2.5-pro	88.82
  Cypher-RI	77.79

  Remarkably, our model, despite being trained only on Cypher data, improves performance on Gremlin by 17% over its base model and 7.7% over a 27B model fine-tuned for Text-to-Cypher. This indicates that our training method helps the model learn underlying graph logic that partially transfers to other query languages.

  2.Expand Schema Selection to SQL Generation

  To validate the generalizability of our proposed methodology, we extended its application from graph databases to the Text-to-SQL domain. This involved integrating our core concept—an explicit schema selection stage within a RL framework—into an existing Text-to-SQL pipeline.

  Our experiment builds upon the SQL-R1 [2] framework. In its original form, SQL-R1 employs an RL agent optimized with the following composite reward function:

  $$S=S_f+S_e+S_r+S_l$$ $$S_r = \begin{cases} 3, & \text{if query result is correct} \newline 0, & \text{if format is incorrect or SQL candidate is not executable} \newline -3, & \text{if query result is incorrect} \end{cases}$$

  where the components represent rewards for format $S_f$, execution $S_e$, result correctness $S_r$, and query length $S_l$.

  To integrate schema selection into the training process, we modified the prompt to require the model to perform schema selection before generating the SQL query, similar to our Cypher-RI approach. Consequently, we introduced a new reward function:

  $$S=S_f+S_e+S_r+S_l+S_c$$ $$S_c = \begin{cases} 3, & \text{if schema selection result is correct} \newline 0, & \text{if json format of selected schema is incorrect} \newline -3, & \text{if schema selection result is incorrect} \end{cases}$$

  where $S_c$ is the new schema selection reward.

  For a controlled and fair comparison, we omitting the SFT step to isolate the performance gains attributable to our RL-based approach and strictly adhered to the experimental conditions of SQL-R1.

  We evaluated our enhanced framework on the Spider and BIRD benchmarks. The evaluation results are detailed in the table below.

  Methods	Base Model	Spider(Dev)	Spider(Test)	BIRD(Dev)
  OmniSQL	Qwen2.5-Coder-7B	85.5	88.9	66.1
  DAIL-SQL	GPT-4	83.6	86.2	54.8
  MAC-SQL	GPT-4	86.8	82.8	59.4
  SQL-R1	Qwen2.5-Coder-7B	84.5	86.1	63.1
  Ours	Qwen2.5-Coder-7B	87.1	86.8	65.4

  Our approach improves performance across the board, achieving a 2.6-point gain on Spider (Dev) and a 2.3-point gain on BIRD (Dev).

  These experiments confirm that our central idea—explicitly integrating schema selection within an RL framework—is a portable principle that enhances performance beyond just Cypher. It is a robust methodology applicable to a wider range of Text-to-Query systems.

References:

[1] numb3r33/know_gremlin dataset in huggingface

[2] SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning

Dear Reviewer,

We hope this message finds you well. We wanted to follow up and kindly ask whether our response has addressed your concerns. Your feedback means a great deal to us, and we truly appreciate your time and insights.

Dear Reviewer,

Thank you again for your thoughtful feedback on our submission.

As the discussion period is concluding soon, and the updated ratings are not visible this year, we were hoping to confirm if our response successfully addressed your concerns.

Any further feedback you might have would be greatly appreciated. We are ready to provide any additional clarifications needed.

We sincerely thank the reviewer for their valuable feedback and for acknowledging the strengths of our work. We wish to address the main concern regarding the technical contribution and originality, and provide new experimental results that further underscore the significance and generality of our method.

The reviewer suggests our work is "more or less like a direct application of GRPO to the domain specific problem." We respectfully argue that this perspective undervalues two key aspects of our contribution: (1) the significance of the problem itself and our novelty in addressing it, and (2) the generalizability of our proposed framework, which we demonstrate with new experiments.

1.The Importance and Novelty of the Text-to-Cypher Challenge

While the RL algorithm (GRPO) is not our invention, our primary contribution lies in designing a novel framework, Cypher-RI, that is the first to successfully leverage reinforcement learning to holistically solve the Text-to-Cypher task by unifying reasoning and schema selection.

Graph databases are increasingly central to modern data infrastructure, yet their potential is capped by the complexity of query languages like Cypher. Automating the translation from natural language to Cypher is a critical bottleneck that, if solved effectively, unlocks vast capabilities for a broader, non-technical audience. The value and difficulty of this domain should not be understated.

Prior to our work, the prevailing methods relied on supervised fine-tuning or few-shot prompting. These approaches often struggle with complex queries and lack robustness.

Cypher-RI is the first framework to demonstrate that reinforcement learning can effectively benefit Cypher generation without requiring supervised reasoning traces. Moreover, it integrates the crucial component of schema selection as an explicit reasoning step within the RL process. This is a significant methodological shift for the Text-to-Cypher domain. The "application" itself—designing the specific reasoning trace, reward function, and training pipeline that makes this possible is the core technical innovation.

2.New Experiments to Demonstrate Practicality and Broader Applicability

Practical Usability and Inference Efficiency

We benchmarked Cypher-RI's inference latency against the GPT-4o API.

Our locally-deployed 7B model is approximately 4.5 times faster than the GPT-4o API. This result highlights that Cypher-RI offers a state-of-the-art solution that is highly practical for real-world deployment due to its lower latency and computational cost.

Zero-Shot Generalization to a New Graph Query Language

We tested our Cypher-trained model on a public Gremlin dataset to assess its ability to generalize to an unseen graph query language without any fine-tuning.

Remarkably, Cypher-RI improves performance on Gremlin by 17% over its base model and 7.7% over a 27B model fine-tuned for Text-to-Cypher. This indicates that our training method helps the model learn underlying graph logic that partially transfers across languages, proving our approach teaches more than just syntactic memorization.

Expanding the Methodology to Text-to-SQL Generation

To prove that our core idea—explicitly integrating schema selection within an RL framework—is a portable principle, we applied it to the Text-to-SQL domain. We adapted the existing SQL-R1 framework by incorporating a schema selection step and a corresponding reward component ($S_c$), while keeping all other experimental conditions identical.

The reward function in SQL-R1 is:

$$S=S_f+S_e+S_r+S_l$$

where the components represent rewards for format $S_f$, execution $S_e$, result correctness $S_r$, and query length $S_l$.

The modified reward function is:

$$S=S_f+S_e+S_r+S_l+S_c$$

where the new schema selection reward $S_c$ is:

$$S_c = \begin{cases} 3, & \text{if schema selection result is correct} \newline 0, & \text{if json format of selected schema is incorrect} \newline -3, & \text{if schema selection result is incorrect} \end{cases}$$

We evaluated this enhanced framework on the Spider and BIRD benchmarks.

Our approach yields consistent improvements, with a 2.6-point gain on Spider (Dev) and a 2.3-point gain on BIRD (Dev). These results confirm that explicitly rewarding schema selection within an RL loop is a robust and portable methodology that enhances performance in other major Text-to-Query systems.

In summary, we introduce the first RL-based framework for the important Text-to-Cypher task, achieving SOTA performance with a practical, efficient model. Furthermore, our new experiments demonstrate that the core principle of our work is not a narrow, domain-specific trick, but a generalizable methodology that improves performance on other query languages and database types.

We sincerely thank the reviewer for their constructive feedback, which has helped us improve our work. We address the specific questions and weaknesses below.

• Q1: Accuracy of JSON Schema Generation

  We have evaluated the accuracy of generating valid JSON format. The results on both benchmark datasets are presented below.

  	CypherBench(%)	Neo4j-Text2Cypher(%)
  Qwen2.5-Coder-7B	78.32	62.65
  Cypher-RI	100.00	99.62

  As shown in the table below, our method achieves near-perfect format accuracy on both datasets: 100.00% on CypherBench and 99.62% on Neo4j-Text2Cypher.

  In addition to evaluating accuracy in JSON format, we also assessed the accuracy of schema selection on both benchmarks.

  Methods	CypherBench(%)	Neo4j-Text2Cypher(%)
  Qwen2.5-Coder-7B	76.45	73.98
  Cypher-RI	95.87	93.87

  As shown, our model improves the accuracy of schema selection by a significant margin of 20 percentage points over the base model on both datasets. This demonstrates that the RL process effectively teaches the model to select needed schema, which is a critical first step in the generation pipeline.

• W1: Trade-off Between Inference Cost and Accuracy.

  We agree with the reviewer that the trade-off between inference cost and performance is a critical aspect of reasoning models. While longer reasoning chains can improve accuracy, the lack of control over their length makes it difficult to manage test-time compute budgets.

  To investigate this trade-off, we designed an additional experiment, inspired by prior work [1,2,3], aimed at controlling the length of the generated reasoning tokens. We introduce a length-based penalty, $r_{length}$, into our reward function. The modified reward is:

  $$r=r_{format}+(r_{selection}+r_{execution})*r_{length}$$ $$r_{\text{length}} = \begin{cases} 1, & r_{selection}+r_{execution}<=0 \newline \text{clip}(\alpha*(n_{max}-n_{y}),0,1), & r_{selection}+r_{execution}> 0 \end{cases}$$

  where, $n_y$ is the number of tokens generated during rollout, and $n_{max}$ is a hyperparameter that sets the desired maximum token limit. By adjusting $n_{max}$ during training, we can explicitly guide the model to produce reasoning chains of varying lengths.

  The results on CypherBench for models trained with different $n_{max}$ values are below:

  Methods	Inference Tokens	EX(%)	PSJS (%)	Exec. (%)
  Qwen2.5-Coder-7B	53	13.12	22.65	62.95
  Cypher_RI($n_{max}=256$)	223	45.78	49.65	91.93
  Cypher_RI($n_{max}=512$)	436	60.78	64.29	96.93
  Cypher_RI($n_{max}=1024$)	901	66.99	71.73	98.93
  Cypher_RI($n_{max}=1280$)	1045	67.46	74.39	99.11
  Cypher_RI(w/o $n_{max}$)	1482	69.59	75.21	99.28

  As we increase the allowed inference tokens, the model's performance improves across all metrics. However, the gains exhibit diminishing returns. For example, increasing the token count from 223 to 436 yields a 15% absolute gain in Execution Accuracy (EX), while increasing from 1045 to 1482 tokens yields only a 2.13% gain. This analysis shows that our model can be adapted to control inference costs. By incorporating a length-based reward, we can choose a balance between performance and computational budget.

• W2: Comparison with Additional Reasoning Models.

  Per the reviewer's suggestion, we have benchmarked our model against a wider array of reasoning language models. The results on CypherBench are as follows:

  Methods	EX(%)	PSJS (%)	Exec. (%)
  Qwen3-8B	48.72	66.49	81.26
  Qwen3-32B	58.56	81.23	80.75
  DeepSeek-R1	60.01	73.52	96.68
  o3-mini	62.39	74.73	98.00
  Gemini 2.5-pro	74.14	81.70	99.45
  Cypher-RI	69.59	75.21	99.28

  These experiments further strengthen our paper's claims. Our model significantly outperforms much larger models, including the Qwen3-32B and DeepSeek-R1.

  References:

  [1] L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning

  [2] AdaptThink: Reasoning Models Can Learn When to Think

  [3] Learn to Reason Efficiently with Adaptive Length-based Reward Shaping