SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

Dear Reviewer 2vNV, we deeply appreciate your insightful feedback and suggestions for our work. In our responses below, we address each primary question (denoted as Q) and comment (denoted as C). Should there be any misunderstandings of the questions, please kindly let us know; we are eager to communicate with you throughout the discussion period.

Q1: …difflib-based reward encourages shallow formatting mimicry rather than semantic…

We want to kindly highlight that the main novelty of SWE-RL lies in the insight that real-world software bugs often follow contextualized patterns, aligning with the plastic surgery hypothesis [1] in program repair. The difflib-based reward can efficiently capture partial correctness of code changes and makes large-scale training possible; execution-based rewards require expensive environment setup and heavy data curation [2], limiting scalability.

As a result, SWE-RL requires non-trivial reasoning to localize errors and generate patches correctly to solve the software issues. During RL, such reasoning patterns emerge, involving planning and backtracking. We show one example below. Please also kindly refer to Figure 3 in the paper for more examples.

We need to identify where the issue of not preserving…
But wait, in the ToDoItem constructor, there's this line: _type = description…
but that's not the issue…
But then I saw it: in the second constructor…
The actual issue is likely…
And then it hits me…
I think I've got it…

Meanwhile, test-based reward is never perfect. Sometimes, it can be worse than similarity metrics where incorrect patches pass all the tests but miss the true intention. For example, [3] shows that the insufficient tests in SWE-bench impacts 40.9% of SWE-Bench Lite and 24.4% of SWE-Bench Verified leaderboard entries.

In the table below, we also did an additional experiment showing that combining SWE-RL reward with execution is superior to applying execution-only reward alone. Please kindly refer to Q1 from Reviewer n4Dr for more details.

Q2: What empirical evidence links SWE-RL training to improved general reasoning…

Great question. In the training, the model needs to generate a correct patch in one step conditioned on the relevant code context that is typically long. To maximize the reward, it performs extensive and accurate reasoning to find the correct edit location and to produce a correct patch. This process helps the model bootstrap its general reasoning capabilities that are transferable to other tasks. In the table below, we can observe that the maximum thinking length (measured in character count) continuously increases during training:

Similarly, in the following table, we measured the average output length of the original, SFT, and SWE-RL checkpoints on HumanEval+ and MATH. After RL, the reasoning is longer, supporting our assumption that the model acquires general reasoning ability:

Also, we can observe qualitative examples like the one shared in Q1 and those in Figure 3, indicating that SWE-RL incentivizes the model to develop new reasoning strategies.

We use the most appropriate prompts during evaluation and did an additional experiments to align the prompts. As shown below, RL performs the best regardless of the prompt change. Please kindly refer to Q2 from Reviewer ZGKJ for more details.

Q3: Why didn’t the authors include an LLaMA-3 SFT baseline…

Please kindly note that we already included the SFT baseline in Table 2 for repair performance evaluation. The SFT checkpoint uses the same PR dataset as in RL. We also did an additional end to end evaluation below, showing that RL is superior to SFT.

Q4: Can the authors report confidence intervals…

Absolutely. For SWE-bench Verified, the typical paired standard error std(A-B) is 2% for models A and B. Thus two models typically need to differ by 1.96 * 2% (~4%) to be significantly different at the 0.05 level. Coincidentally, std(A) is also around 2% for SWE-bench Verified.

Q5: What was the observed variance in the reward signal…

Good point. As shown in Figure 5, the training reward steadily increased during training. It’s overall stable and the variance is low.

C1: …the KL Divergence Target Is Not Fully Defined…

During RL, $\pi_{\rm{ref}}$ is always set to be the original Llama 3 and kept frozen. Our objective formula and KL definition follows the standard of existing work [4]. We will clarify this in the revision.

C2: Group Size… under-specified by the authors.

Please kindly refer to Section 3.1 of our paper. We described that we sampled "16 rollouts from each of the 32 problems in every batch", so the group size is 16. We will make it more clear in the revision.

C3: …include an ablation study on group size, clipping, and KL penalty…

Thanks for the suggestion. Our hyperparameter choices largely follow the existing best practice in the literature [4]. Unfortunately, we do not have the compute budget to ablate all combinations of the hyperparameters, but this is a valuable suggestion and we will include more ablations in the future when we have sufficient computing resources.

C4: The size or quality of the PR task is not considered… Why 500…

We did rigorous filtering to ensure the quality of the PRs used in the RL training. Please kindly refer to Appendix A for more details regarding the data curation process. Regarding the 500 patches, we demonstrated in Figure 4 in the paper that 500 patches enabled continuous test time scaling of the issue solve rate on SWE-bench Verified.

C5: …claim to OOD domains is not adequately supported…

1-2% performance gains on HumanEval or CRUXEval are not likely to be significant by themselves (at the 0.05 level). That is why we reported on multiple evaluations where all results are consistently in favor of RL, thus giving increased significance (for example, via Fisher’s combined probability test). To support this, we conducted additional statistical testing (link omitted due to rebuttal policy), showing that >0.8% on MMLU (14k examples), 3% on CRUXEval, and >3% on the full MATH are already significant. So the combination of our results will reach significance at the 0.05 level. We will include this statistical analysis in the revision of our paper.

C6: …Variance still exists…

Indeed greedy decoding does not remove variance, and we make no such claims. Most papers and leaderboards do not report the confidence intervals, likely to avoid cluttered tables that do not convey sufficiently useful information. Fortunately, the confidence intervals are mostly a property of the evaluation task. For example, on SWE-bench verified, we measured that a 4% difference is significant for all 90+ results on the leaderboard.

C7: …outperforming DeepSeekCoder-33B and WizardCoder-15B …without GPT-4o’s SWE-bench Verified number…

Please kindly note that we did not compare DeepSeekCoder or WizardCoder in the paper because they are not widely used for SWE-bench related tasks. Meanwhile, we did include GPT-4o's score in Table 1 for both SWE-agent (23.2%) and Agentless (38.8%), and SWE-RL is superior (41.0%).

C8: …no analysis of reward variance…

Please kindly refer to Figure 5 in the paper. We show that the training reward steadily increases during training. It’s stable and the variance is low.

C9: The authors should pay special attention to… words and the choice of color

We appreciate the suggestions. We will improve the clarity of the paper in the revision.

C10: What does the Python difflib do?

As we explained in the paper, difflib computes sequence alignments to measure textual similarity; we use it (e.g., via SequenceMatcher on unified diffs) to score overlap between predicted and target patches, yielding a dense reward for localized edits.

C11: …lack of deeper reward design and validation…

We acknowledge the reward is intentionally simple, but training was stable with low variance (C8) and we observe emergent, multi-step reasoning behaviors in traces (Q1/Q2). Combining SWE-RL reward is also better than using execution-only reward alone (Q1).

C12: …general reasoning improvement… remains unsubstantiated…

We compare an SFT baseline trained on the same PR data (Q3) and also present behavioral diagnostics (Q2), both indicating that RL drives the out-of-domain gains while SFT leads to memorization. We will clarify this point in the revision.

C13: SWE-RL appears novel… without critical innovation.

As we explained in Q1, the main novelty of SWE-RL lies in the insight that real-world software bugs, unlike open-ended code generation, often follow constrained, localized patterns, aligning with the plastic surgery hypothesis [1] in program repair. The insight itself does not seem novel as it has been known for a decade, but SWE-RL is the first to show that the simple difflib-based reward signal can already enable scalable RL on massive real-world software data. This finding will impact lots of future work in this critical application domain.

C14: …reference SWE papers from among these top-tier venues…

Great suggestion. We’ll expand Related Work to cite representative papers from top-tier software engineering venues in the revision.

[1] Barr et al. The Plastic Surgery Hypothesis

[2] Pan et al. Training Software Engineering Agents and Verifiers with SWE-Gym

[3] Yu et al. UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench

[4] DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Thank you for taking the time to read our response and for your additional feedback. We truly appreciate it! We address your additional concerns as follows:

• A patch with high textual overlap may still break invariants, violate specifications, or introduce subtle bugs.
• difflib does not seem to reason about control/data flow, side effects, or test coverage.
• I am not quite convinced that the learned model would not over-optimize on formatting or superficial token reuse (aligning with the plastic surgery hypothesis), while missing deeper, bug-fixing behavior.

We agree that the similarity reward is not perfect, but as we demonstrated in Q1, it can efficiently capture partial correctness of code changes and make large-scale training possible. Over the course of training, the model learns to produce semantically correct patches with this reward signal. In contrast, execution-based rewards require expensive environment setup and heavy data curation that limit scalability.

To support this point, we provided a concrete example in Q1 showing that SWE-RL can trigger emergent reasoning patterns rather than superficial token reuse:

We need to identify where the issue of not preserving…
But wait, in the ToDoItem constructor, there's this line: _type = description…
but that's not the issue…
But then I saw it: in the second constructor…
The actual issue is likely…
And then it hits me…
I think I've got it…

Meanwhile, we conducted an additional experiment showing that combining the similarity reward with execution is superior to using execution-based reward alone:

These results indicated that the difflib-based similarity reward, despite its simplistic design, can essentially bootstrap the model's general reasoning capability and enhance its bug-fixing ability through reinforcement learning.

• In software engineering, reproducibility and configuration stability are essential. RL methods are notoriously unstable, and without robustness checks, practitioners adopting this framework may fail to reproduce improvements.

We fully agree that reproducibility and stability are essential. To ensure reproducibility, we have described our training hyperparameters clearly in the paper. We’ll also open source the pipeline, model, data, and also all evaluation results after careful privacy reviews.

As we indicated in C3, exhaustive ablations over all hyperparameter and component combinations are infeasible under our current compute budget; however, we will incorporate this valuable suggestion and add more ablations when we have sufficient computing resources. Despite these resource constraints, we conducted as many additional experiments as possible to show that our method is genuinely effective rather than noise. For example, the table below shows that SWE-RL consistently outperforms the SFT baseline across different prompt setups.

Regarding stability, as we showed in Figure 5 of the paper, the reward steadily increases during the training with a low variance. In the table below, we also provide concrete reward scores in different training steps, showing the increasing trend:

Thank you again for your feedback. We hope our response could address your additional concerns. Should you have any new questions, please don't hesitate to let us know.

Dear Reviewer QFgA, we deeply appreciate your insightful feedback and suggestions for our work. In our responses below, we address each primary question (denoted as Q) and comment (denoted as C). Additionally, we will revise our paper to incorporate editorial suggestions. Should there be any misunderstandings of the questions, please kindly let us know; we are eager to communicate with you throughout the discussion period.

Q1: It is unexpected to see degradation of SFT model on general tasks in Table 3 with respect to the base model, given that coding and general SFT data are also included in training. Do you have any insights?

Great question. We believe that the underlying reason is that RL teaches the model generalized reasoning while SFT only lets the model memorize the data it is trained on, as supported by a recent paper [1]. SWE-RL, while being a single RL task, requires the model to do complex reasoning on the long context input comprising hundreds to thousands of lines of code to correctly localize the bug and generate the patch. This process incentivizes the model to learn generalized reasoning.

C1: Did you employ any test-time scaling during RL training...

Thank you for the thoughtful question. No, we did not use test‑time scaling during RL training: each rollout produces a single patch and is rewarded via patch similarity, and we do not train any separate selector or reranker. We still report test‑time–scaled evaluation to follow existing work like Agentless [2], also validating the generalizability of SWE-RL where the reasoning learned through SWE-RL transfers to other subtasks. As a result, in training, we only need to guide the model to generate the best patch for each rollout, keeping the RL scalable and execution‑free. For a matched setting, our pass@1 (no scaling) results are reported in Table 2 and show consistent gains over SFT.

C2: The comparison between the RL and the SFT model on the original SWE-Bench verified task (instead of the repair task) is critical but not included.

Great point. The main reason why we evaluate the SFT and RL model on the repair task is to focus on comparing their bug fixing and code editing capabilities, isolating the effect of localization used in the Agentless framework. To have a better understanding of their respective performance on the full SWE-bench Verified task, we conducted an additional experiment below to show the SFT pass@1 with the end to end pipeline. From the table, we can see that SFT is worse than RL by 4.8 points. According to our statistical analysis, a 4-point difference is statistically significant enough to show that RL is better than SFT on SWE-bench Verified.

C3: Have you tried SFT with synthetic code editing data only?...

Good question. We mixed different SFT data to make sure the model can follow different kinds of instructions and can successfully complete all the steps in Agentless-Mini. In the table below, we also conducted an SFT experiment with only code editing data. It has a similar score to the mixed SFT model in the repair only setting. This checkpoint cannot be evaluated in the other Agentless steps because it cannot follow instructions well and suffers from many format errors in the other subtasks such as localization and test generation.

[1] Chu et al. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training.

[2] Xia et al. Agentless: Demystifying LLM-Based Software Engineering Agents

Dear Reviewer ZGKJ, we deeply appreciate your insightful feedback and suggestions for our work. In our responses below, we address each primary question (denoted as Q) and comment (denoted as C). Should there be any misunderstandings of the questions, please kindly let us know; we are eager to communicate with you throughout the discussion period.

Q1: …what specific mechanism or training dynamics enable skill transfer from patch generation to tasks like MATH or HumanEval+?

Great question. In SWE-RL training, the model needs to generate a correct patch in one step conditioned on the relevant code context that is typically long. To maximize the reward, it needs to perform extensive and accurate reasoning to find the correct edit location and to produce a correct patch. This process involves significant reasoning and helps the model bootstrap its general reasoning capabilities that are transferable to other tasks. In the table below, we can observe that the maximum thinking length (measured in character count) continuously increases during training:

Similarly, in the following table, we measured the average output length of the original, SFT, and SWE-RL checkpoints on HumanEval+ and MATH. After RL, the reasoning is longer, supporting our assumption that the model acquires general reasoning ability through improved software issue solving.

Q2: …did you evaluate the SFT baseline using the exact same prompt as used during RL training and evaluation?…

We use the most appropriate prompts tailored to each model checkpoint during evaluation. For SWE-bench, both the SFT and RL checkpoints are evaluated using identical system and user prompts. For out-of-domain benchmarks like HumanEval+ and MATH, we keep the user prompts the same across checkpoints, but apply the system prompt (shown in Figure 2) only to the RL-trained model. This is because the SFT checkpoint's training data includes system prompts for code editing tasks, but not for general coding tasks. As a result, applying a system prompt in those general tasks would be out-of-distribution for the SFT model. In contrast, the RL checkpoint is expected to generalize its reasoning under the system prompt, making its use appropriate.

To confirm that this difference in prompting doesn't unfairly benefit the RL model, we ran an additional experiment applying the system prompt to the SFT checkpoint on HumanEval+ and MATH as well. This allowed us to test whether the system prompt itself influences performance on general tasks for SFT.

From the result, we can see that the SFT checkpoint is still worse than the RL one despite the prompt alignment. It’s also worse than the prompt we used for evaluation in the paper. It matches our claim that RL leads to generalized reasoning while SFT is more about memorizing the data patterns.

Q3: The difflib-based reward function operates on surface-level textual similarity and is agnostic to semantic correctness…

We want to kindly highlight that the main novelty of SWE-RL lies in the insight that real-world software bugs, unlike open-ended code generation, often follow constrained, localized patterns, aligning with the plastic surgery hypothesis [1] in program repair. The difflib-based reward can efficiently capture partial correctness of code changes. While we do observe some high-reward patches that fail the tests, these episodes can still improve reasoning because the difflib signal implicitly incentivizes accurate bug localization and minimal, targeted edits, which are necessary intermediate steps toward a correct fix. With more training steps, the model can learn to refine these patches further and produce semantically correct solutions. Crucially, SWE-RL reward is what makes large-scale training possible; execution-based rewards require expensive environment setup and heavy data curation, limiting scalability. For example, SWE-Gym [2] requires 200 human annotation hours and 10,000 CPU core hours to produce 2438 trainable instances, while we collect 273k instances for training fully automatically.

In the meantime, test-based reward is never perfect. In certain cases, it can be worse than similarity metrics where incorrect patches pass all the tests but miss the true intention of the problem. This claim is well supported by the literature. For example, [3] shows that the insufficient tests in SWE-bench impacts 40.9% of SWE-Bench Lite and 24.4% of SWE-Bench Verified leaderboard entries. Below is an example illustrated in the paper, where the original issue requires a function that computes polynomial fits for data, to handle missing data in the inputs x and y. However, the original test case only considers scenarios where both x and y have missing data. The generated patch, while passing the test, fails to handle additional cases where only one input has missing data.

# Insufficient test
def test_missing_data(self, df):
  groupby = GroupBy(["group"])
  df.iloc[5:10] = np.nan
  res1 = PolyFit()(df[["x", "y"]], groupby, "x", {})
  res2 = PolyFit()(df[["x", "y" ]].dropna(), groupby , "x", {})
  assert_frame_equal(res1, res2)

# Incorrect patch that passes the test
def _fit_predict(self, data):
  y = data["y"].dropna()
  x = data["x"].dropna()
  if x.shape[0] != y.shape[0]:
    raise ValueError ("x and y must have the same number of non - missing values ")
  if x.nunique() <= self.order:
    # TODO warn ?
    xx = yy = []

To further support our point, we did an additional experiment showing that combining SWE-RL reward with execution is superior to applying execution reward alone. The results are in the table below. Please kindly refer to Q1 from Reviewer n4Dr for more details:

Q4: …How does this setting support the emergence of non-trivial reasoning strategies, as opposed to shallow pattern recognition?

As we explain in Q3, SWE-RL requires non-trivial reasoning to localize errors and generate patches correctly to solve the software issues. During RL, such reasoning patterns emerge, involving planning and backtracking. We show one example below. Please also kindly refer to Figure 3 in the paper for more examples.

We need to identify where the issue of not preserving single quotes in TODO comments occurs… In the ToDoItem class, the description is being extracted…
But wait, in the ToDoItem constructor, there's this line: _type = description.Substring(0, idxDelimiter);. However, the idxDelimiter is being found …but that's not the issue here.
But then I saw it: in the second constructor of ToDoItem…
The actual issue is likely due to… And then it hits me… I think I've got it now.

Q5: …please provide concrete behavioral evidence that the model acquired new reasoning capabilities…

Great point. As we showed in Q1 and Q2, the reasoning length of the model increases after SWE-RL in both training and evaluation. According to Figure 3 in the paper and the example we showed in Q4, SWE-RL triggers the model’s emergent reasoning skills such as planning, self-reflection, and divide-and-conquer. Below, we also show a concrete example on MATH where the model doesn’t do much reasoning before SWE-RL, but reasons deeply afterward:

A math problem whose ground-truth answer is $\frac{\sqrt{21}}{5}$:

Before SWE-RL

We have that $\sin R = \frac{2}{5} = \frac{RS}{RT}$, so $RS = 2$ and $RT = 5$.  By the Pythagorean theorem… Then $\sin T = \frac{RS}{RT} = \frac{2}{5}$.

Answer: $\frac{2}{5}$

After SWE-RL:

To find $\sin{T}$, we first need to understand the given information. We're dealing with a right-angled triangle $\triangle{RST}$, and it's given that $\sin{R}=\frac{2}{5}$…
From the diagram, we see that the length of the hypotenuse…
Next, to find $\sin{T}$, we need to determine the length of the side opposite…
However…
Given $RS = 2$ and $RT = 5$…
But, we actually need to use $ST$ and $RT$ …
Thus, $\sin{T} = \frac{ST}{RT} = \frac{\sqrt{21}}{5}$.
</think>
<solution>
Answer: $\frac{\sqrt{21}}{5}$ 
</solution>

C1: The paper makes limited methodological contributions…

As we explained in Q3, the main novelty of SWE-RL lies in the insight that real-world software bugs, unlike open-ended code generation, often follow constrained, localized patterns, aligning with the plastic surgery hypothesis [1] in program repair. Furthermore, SWE-RL is the first to show that the simple difflib-based reward signal can already enable scalable and effective RL on massive real-world software data. This finding will impact lots of future work in this critical application domain.

C2: The reported cross-task improvements are small and not convincingly explained…

Indeed, 1-2% performance gains on HumanEval or CRUXEval are not likely to be significant by themselves (at the 0.05 level). That is why we reported on multiple evaluations where all results are consistently in favor of RL, thus giving increased significance (for example, via Fisher’s combined probability test). To support this, we conducted additional statistical testing (link omitted due to rebuttal policy), showing that >0.8% on MMLU (14k examples), 3% on CRUXEval, and >3% on the full MATH are already significant. So the combination of our results will reach significance at the 0.05 level. We will include this statistical analysis in the revision of our paper.

C3/C4/C5 are covered in the previous responses.

[1] Barr et al. The Plastic Surgery Hypothesis.

[2] Pan et al. Training Software Engineering Agents and Verifiers with SWE-Gym.

[3] Yu et al. UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench.

Dear Reviewer n4Dr, we deeply appreciate your insightful feedback and suggestions for our work. In our responses below, we address each primary question (denoted as Q) and comment (denoted as C). Additionally, we will revise our paper to incorporate editorial suggestions. Should there be any misunderstandings of the questions, please kindly let us know; we are eager to communicate with you throughout the discussion period.

Q1: Have the authors considered semantic diffing or static analysis tools (e.g., AST comparison or test outcome delta) for more robust rewards?

Great point. We applied basic patch normalization (e.g., header and whitespace removal) before comparison, but not at the AST level. Regarding test outcomes, we conducted an additional experiment incorporating unit-test results into reward calculation. As detailed in Q2, we also enabled the model to interact with the environment using tools such as Bash and an editor. Due to time constraints, this experiment was performed with Llama-3.1-8B-Instruct. The table below shows pass@1 on SWE-bench Verified comparing three settings: (1) the baseline Llama-3.1-8B-Instruct, (2) RL with a binary execution-only reward (fail = 0, pass = 1), and (3) RL with execution reward combined with the SWE-RL reward. All models were trained for 400 RL steps on the SWE-Gym [1] dataset.

Both reward signals significantly improve over the baseline, and the SWE-RL reward complements execution-based rewards, aligning with our insight that software issues often require contextualized fixes, consistent with the Plastic Surgery Hypothesis [2] in program repair. While execution-based methods require substantial data collection in executable environments, the SWE-RL reward scales to large, open software corpora.

Q2: Could test failures or runtime logs be incorporated as feedback in future RL training?

Absolutely! In the Q1 experiments, we allowed the model to issue Bash commands freely when solving issues. Although not strictly enforced, the model could execute tests and use them as feedback to improve its solutions. In the table below, we divide trajectories generated during RL by whether they observed and fixed test errors, and we report their pass@1.

A concrete example of the LLM leveraging runtime logs for better issue solving:

...
[assistant]
Now that we've updated the import statement, we should be able to run the test again.

Let's try running the test again to see if the import issue is resolved.

<tool: bash>
python -m unittest /testbed/tests/test_cognitoidp_models.py
</tool>
</bash>

[user]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/miniconda3/envs/testbed/lib/python3.12/unittest/__main__.py", line 18, in <module>
    main(module=None)
  File "/opt/miniconda3/envs/testbed/lib/python3.12/unittest/main.py", line 104, in __init__
    self.parseArgs(argv)
  File "/opt/miniconda3/envs/testbed/lib/python3.12/unittest/main.py", line 153, in parseArgs
    self.createTests()
  File "/opt/miniconda3/envs/testbed/lib/python3.12/unittest/main.py", line 164, in createTests
    self.test = self.testLoader.loadTestsFromNames(self.testNames,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/testbed/lib/python3.12/unittest/loader.py", line 207, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/testbed/lib/python3.12/unittest/loader.py", line 137, in loadTestsFromName
    module = __import__(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/testbed/tests/test_cognitoidp_models.py", line 5, in <module>
    class TestCognitoResourceServer(unittest.TestCase):
  File "/testbed/tests/test_cognitoidp_models.py", line 6, in TestCognitoResourceServer
    @mock_cognito_idp
     ^^^^^^^^^^^^^^^^
NameError: name 'mock_cognito_idp' is not defined. Did you mean: 'mock_cognitoidp'?

[assistant]
The error message indicates that `mock_cognito_idp` is not defined, even though we've already updated the import statement to `mock_cognitoidp`.

It looks like there's another issue with the usage of `mock_cognito_idp` in the `TestCognitoResourceServer` class. We should actually use `@mock_cognitoidp` instead of `@mock_cognito_idp`.

Let's update the test file to use `@mock_cognitoidp` instead.

<tool: edit>
/testbed/tests/test_cognitoidp_models.py
<<<<<<< SEARCH
    @mock_cognito_idp
=======
    @mock_cognitoidp
>>>>>>> REPLACE
</tool>

[user]
File /testbed/tests/test_cognitoidp_models.py has been successfully updated at line 6 and now contains 22 lines.<|eot_id|>

[assistant]
Now that we've updated the decorator to use `mock_cognitoidp` correctly, we should be able to run the test again.

Let's try running the test again to see if the usage issue is resolved.

<tool: bash>
python -m unittest /testbed/tests/test_cognitoidp_models.py
</tool>
</bash>

[user]
.
----------------------------------------------------------------------
Ran 1 test in 0.038s

OK

...

These results indicate that incorporating tests improves issue solving, and we will include this valuable idea in future iterations.

C1: Reward function limitation: ...the reward is based on string-level patch similarity, not semantic equivalence. This penalizes functionally correct but syntactically different fixes, potentially limiting solution diversity

We want to kindly highlight that the main novelty of SWE-RL lies in the insight that real-world software bugs, unlike open-ended code generation, often follows constrained, localized patterns, aligning with the plastic surgery hypothesis [2] in program repair. The difflib-based reward can efficiently capture partial correctness of code changes. Crucially, this reward is what makes large-scale training possible; execution-based rewards require expensive environment setup and heavy data curation [1], limiting scalability. SWE-RL unlocks scalable, generalizable RL on real-world software data for the first time.

In the meantime, test-based reward is never perfect. In certain cases, it can be worse than similarity metrics where incorrect patches pass all the tests but miss the true intention of the problem. This claim is well supported by the literature. For example, EvalPlus [4] shows that the insufficient tests in the original HumanEval and MBPP benchmarks lead to many erroneous code solutions being accepted. Furthermore, [5] shows that the insufficient tests in SWE-bench impact 40.9% of SWE-Bench Lite and 24.4% of SWE-Bench Verified submissions. Below is an example illustrated in the paper, where the original issue requires a function that computes polynomial fits for data, to handle missing data in the inputs x and y. However, the original test case only considers scenarios where both x and y have missing data. The generated patch, while passing the test, fails to handle additional cases where only one input has missing data.

# Insufficient test
def test_missing_data(self, df):
  groupby = GroupBy(["group"])
  df.iloc[5:10] = np.nan
  res1 = PolyFit()(df[["x", "y"]], groupby, "x", {})
  res2 = PolyFit()(df[["x", "y" ]].dropna(), groupby , "x", {})
  assert_frame_equal(res1, res2)

# Incorrect patch that passes the test
def _fit_predict(self, data):
  y = data["y"].dropna()
  x = data["x"].dropna()
  if x.shape[0] != y.shape[0]:
    raise ValueError ("x and y must have the same number of non - missing values ")
  if x.nunique() <= self.order:
    # TODO warn ?
    xx = yy = []

Our insights are also backed by the results we showed in Q1 and Q2, where combining SWE-RL reward can surpass execution-only signal in a multi-turn RL setup.

C2: Limited interactivity: The pipeline is not interactive or agentic—it lacks tool use, exploration, or test-driven feedback during training...

Thank you for the thoughtful point. Our design choice was to prioritize scalability: SWE‑RL trains on large, real‑world repos without execution environments. The reward is orthogonal to interactivity; as Q1–Q2 show, adding it to agentic, tool-using RL improves over execution‑only signals. We view the practical recipe as hybrid, first training at scale with SWE‑RL, then doing agentic RL on executable subsets with test‑driven feedback. We’ll clarify this trade‑off and our roadmap in the paper.

C3: Single-task RL training: ...While generalization is observed, multi-task RL training might further enhance its capabilities.

Great suggestion! We plan to explore multi-task RL training in future work to further understand the capability transfer and generalization.

C4: The training cost is too high, which requires 512 H100 GPUs for ~ 32 wall-time hours, which may limit accessibility and reproducibility despite using open-source models.

We acknowledge the relatively high training cost. However, this cost is expected for doing RL at scale on software engineering tasks and remains affordable for industry labs (e.g., agentic systems like Kimi K2 [3] operate with over 10k parallel containers during training, substantially larger GPU and CPU cost). To ensure reproducibility, we have described our training hyperparameters clearly in the paper. We’ll also open source the pipeline, model, data, and also all evaluation results after careful privacy reviews.

[1] Pan et al. Training Software Engineering Agents and Verifiers with SWE-Gym.

[2] Barr et al. The Plastic Surgery Hypothesis.

[3] Kimi Team. Kimi K2: Open Agentic Intelligence.

[4] Liu et al. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation.

[5] Yu et al. UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench.