Teaching Large Language Models to Self-Debug

Thanks for raising the concern. We appreciate that you consider our work as innovative and well-written. We would like to clarify our experimental setup as follows.

Self-debugging does not have different test input/output pairs to decide the code correctness compared to baselines

We want to emphasize that self-debugging does not utilize extra input-output pairs compared to the baselines. Specifically, for text-to-SQL generation, there is no unit test for both self-debugging and all other baselines. Therefore, the LLM needs to infer the code correctness completely by itself. As described in Section 4.3, on MBPP, we follow the baseline approaches to include 1 unit test in the problem description, and other hidden unit tests are only used for the final evaluation on the code correctness. Therefore, the LLM also needs to infer the code correctness, since passing the given unit test does not necessarily mean that the predicted code is fully accurate and can pass the hidden unit tests. Note that during the self-debugging process, the hidden unit tests are never used, and the generated code is only executed on the 1 given unit test that already shows up in the problem description for generating the initial code. For code translation, we consider the setting where all unit tests are available, since the ground truth execution results can already be obtained by executing the code in the source programming language, which also follows the evaluation setup in prior work.

We compared to baselines that utilize code execution

As described in Section 2 and Section 5 (page 5), we compared self-debugging to baselines that generate multiple programs, and use code execution to select the final response.

On the Spider benchmark for text-to-SQL generation, since the unit tests are not available, the baseline takes the majority vote of execution results as the final prediction. As shown in Figure 4 (a), self-debugging from greedy decoding matches the performance of the execution-based baseline that uses 16 samples, and self-debugging consistently improves the sample efficiency. For Transcoder, our baseline utilizes all unit tests to determine the code correctness. Note that the performance of this baseline is equivalent to the reviewer’s second suggested solution, since we consider the prediction from this baseline to be accurate if any one of the generated samples is correct. Again, we show that self-debugging from greedy decoding matches the baseline that generates 10 samples, and we presented the full analysis in Appendix E.1.

The usage of richer execution information is a strength of our self-debugging design

First, we would like to note that the reranking techniques LEVER and MBR-Exec also utilize code execution to select the response. The main difference between our work and these execution-based reranking techniques lies in the way we utilize the execution results. Specifically, the baseline approaches generate multiple samples all at once, thus the model can generate the same wrong code multiple times despite observing the execution errors. On the other hand, self-debugging can leverage past predictions and code execution to reduce the redundant cost of resampling wrong predictions, which is an advantage over existing reranking techniques.

Furthermore, the key motivation of our self-debugging design is to provide complementary performance gain on top of all kinds of existing code generation methods. As shown in Figure 1 and described in Section 3, the first step of each self-debugging turn is code generation, which can be implemented as any existing baseline such as greedy decoding and execution-based reranking. Therefore, the subsequent debugging step leverages richer execution information than the first step of code generation by design. We demonstrate in Table 1, Figure 4 (a) and Appendix E.1 that self-debugging does not only improve over greedy decoding, but also consistently improves the performance when the baseline utilizes multiple samples.

We hope the above discussion addresses your concerns. Please do not hesitate to contact us for any further questions or clarifications.

We thank the reviewer for the encouraging comments.

Concrete example of self-debugging

Thanks for the feedback. We have added Figure 2 and 3 in the main paper to show the self-debugging workflow, which are abbreviated versions of Figures 6 and 7 in Appendix F. The 2 figures demonstrate concrete running examples of how self-debugging works with different prompt formats.

Sampling budget comparison

Compared to the sampling cost for obtaining the initial code for debugging, self-debugging requires the LLM to generate more tokens by design. For example, in Table 2, self-debugging generates more samples than the greedy decoding baseline. This extra cost is also needed for other related work, e.g., self-refine and Reflexion.

On the other hand, as discussed in Section 5.2.1, we demonstrate that to improve the accuracy over greedy decoding, self-debugging achieves better performance with a lower inference cost. To compare the cost in terms of the number of tokens, we consider two settings: (1) when unit tests are not available; e.g., on Spider for text-to-SQL generation; and (2) when unit tests are available; i.e., on Transcoder and MBPP for Python code generation.

On the Spider benchmark, with explanation feedback, on average a full debugging turn generates 5.5 times more tokens than simply generating the SQL query. By comparing the performance of self-debugging starting from greedy decoding (80.8% accuracy) to the baseline that samples 16 programs without self-debugging (80.7% accuracy), we can see that self-debugging also reduces the number of generated tokens compared to the baseline.

On Transcoder and MBPP, due to the availability of unit tests, the improvement with explanation feedback is generally less significant than on Spider. In this case, the unit test feedback is a better option to reduce the sampling budget. On average, a full debugging turn with the unit test feedback generates 2.5 times more tokens than simply generating the initial code, thus self-debugging is again more sample-efficient in terms of the token cost.

Furthermore, despite the fact that generating explanation and trace feedback increases the number of generated tokens, we note that one benefit of such feedback is to provide more interpretable rationales on how LLMs infer the code correctness, and how the fixes are conducted. Such natural language feedback helps us to investigate the models’ weaknesses, and potentially can be used to finetune the model and improve its overall debugging and coding ability.

Comparison to Reflexion

The Reflexion paper evaluates their approach on AlfWorld and HotPotQA, which are decision-making tasks and question-answering tasks. For both benchmarks, they assume that the oracle reward is available, which tells the LLM whether the current state is successful or the current answer is correct.

On the contrary, our work focuses on code generation tasks, where the LLM does not always have access to the oracle reward about the code correctness. For example, on text-to-SQL generation where the unit tests are not available, the LLM needs to determine completely by itself when the debugging process should terminate. Furthermore, we proposed different prompt formats to enable self-debugging for diverse tasks and LLMs, and we demonstrated that the self-debugging ability can be taught with few-shot prompting, even if the model is not specially tuned for zero-shot debugging. For example, with code-davinci-002 that is not instruction tuned, we showed that self-debugging with few-shot prompting achieves similar or better performance gain than GPT-4.

Questions on fixing logic errors

For all tasks, our evaluation metric is the execution accuracy, which considers the predicted code to be correct when it passes the given unit tests or its outputs match those of the ground truth code. Therefore, when we only use unit tests to determine whether to continue the self-debugging process, e.g., for Transcoder experiments where all unit tests are available, programs that pass the unit tests but have other logic errors would not be detected. On the other hand, when we request the LLM to determine the correctness of the generated code, e.g., on Spider and MBPP, the LLM can correct such logical errors to some extent. In our experiments, we manually investigated tens of correct predictions from each benchmark, and we did not notice such false positives.

We thank the reviewer for the constructive feedback.

Significance of the Spider improvement

We would like to note that 2-3% improvement on Spider is non-trivial. As shown in Figure 4 (a), without self-debugging, a 3% improvement from greedy decoding requires 16 samples. In particular, Figure 4 (b) shows that self-debugging improves the accuracy on the hardest SQL tasks by 9%, which is a significant boost.

Significance of the code explanation on Transcoder and MBPP

First, we would like to clarify that code explanation results in Tables 1-3 are accompanied with self-debugging. The evaluation of code explanation without self-debugging is not our primary focus, but we presented the results in Figure 1 of Appendix E.1, where code explanation consistently improves the performance by 2-4%.

On the other hand, due to the availability of unit tests, we acknowledge that the improvement with explanation feedback on Trancoder and MBPP is generally less significant than on Spider. However, one benefit of explanation feedback is to provide more interpretable rationales on how LLMs infer the code correctness, and how the fixes are conducted. Such natural language feedback helps us to investigate the models’ weaknesses, and potentially can be used to finetune the model and improve its overall debugging and coding ability.

Furthermore, we would like to emphasize that one core contribution of our work is to conduct a comprehensive evaluation of self-debugging performance with different LLMs, applications, prompting methods and feedback formats, and demonstrate the importance of each factor. Specifically, both code explanations and traces are also commonly used for human debugging, which do not require any extra input from other human programmers. We show that stronger LLMs for code (code-davinci-002 with few-shot prompting and gpt-4) benefit more from such richer self-generated feedback, which can motivate future work on improving LLMs for investigating and understanding the semantic meaning of code.

Importance of unit test execution

As discussed in Section 5.2.2, we acknowledge that the improvement of self-debugging is less significant without unit test execution. However, even without unit tests, sometimes LLMs can still improve their performance with self-generated feedback. For example, as shown in Table 3b:

• With code-davinci-002, self-debugging without unit test execution improves the performance by up to 5%;
• GPT-4 without unit test execution improves the MBPP accuracy by 3.6%.

Furthermore, we would like to note that utilizing unit tests whenever applicable is also part of our self-debugging approach, and we consider the ability of leveraging execution of past generated programs as a key advantage of our self-debugging approach.

Comments on paper writing

Thanks for the feedback about Section 4’s readability. We have added Figure 2 and 3 in the main paper to show the self-debugging prompt formats, which are abbreviated versions of the corresponding figures in the appendix. We hope this makes the section easier to read.

Regarding other questions:

• Codex in Table 1 represents the baseline without self-debugging. We put it under the section “Self-Debugging (this work)” to denote that the following self-debugging results are built upon initial programs generated from this baseline. We added a divider around “Codex (baseline)” in Table 1 to make it clearer that these are the baseline results before self-debugging.
• The Codex results in Table 2 were produced from greedy decoding, which demonstrate that self-debugging improves the performance with a higher sample efficiency. On the other hand, Codex baseline results in Table 1 were obtained with multiple samples, which demonstrate that self-debugging outperforms other baseline methods that leverage multiple samples. We updated the captions of Table 1 and 2 to clarify the settings.

We hope our paper revision and above discussion address your concerns. Please do not hesitate to contact us for any further questions or clarifications.

I think you may have misunderstood my comment regarding 2-3% improvements. What I am wondering about is whether simple prompt changes may result in improvements of similar range.

We know that prompt changes affect results. We don't know if baselines and self-debugging use best possible prompts, so-so prompts, or not very good prompts. It is possible that the result ranges of baseline and self-debugging overlap if the prompt ranges are considered. We don't know how the results would compare if the best "oracle" prompts would be used.

Determining reasonable prompt variability range may not be simple. Authors naturally pick the best prompt that they have seen so far. Different authors may spend less or more effort to find the best prompt.

Have you tried different prompts for self-debugging? What variability range did you see?

Thank you

Thanks for your followup response! We agree that changing the baseline prompt can also change the performance. However, we have done our best to optimize the prompts and construct strong baselines. Please let us know if you have any better baseline prompts in mind, and we are more than happy to evaluate them.

On the other hand, the improvement from self-debugging is complementary to the improvement from baseline prompt changes. As supporting evidence, for text-to-SQL generation, we evaluated a variant of our full prompt where we removed the demonstration of one data row per table; i.e., we deleted the INSERT INTO statements in Appendix H. With greedy decoding, the accuracy of this baseline prompt is 75.6%, which is 1.9% lower than our full prompt. From this baseline, self-debugging also improves the accuracy by 2.1%. Note that removing INSERT INTO statements weakens both the baseline and the self-debugging prompt, but we still see a consistent improvement from the self-debugging process.

Regarding different self-debugging prompts, we have shown results of integrating different feedback information in Section 5, especially Table 2. During the development of our approach, we have also done extensive evaluation on different wordings of each feedback type, but we did not observe much performance differences that change the relative order when comparing different feedback types.

We hope the above discussion and followup experiments address your concerns. Please let us know your thoughts, and we are more than happy to answer any further questions.

For the overhead of the self-debug, I believe that we can simply consider each interaction with LLM can increase one running time for code generation. However several researchers have pointed out this overhead is worthwhile since it can largely increase code generation effectiveness. While I think the key problem of this paper is that Self-Debug is not new for code generation, previous strategies, e.g., Self-Edit, Self-Refine, and Self-Collaboration are used for code generation, but the authors have not compared with even one baseline in their experiments?