Dear Reviewer wpuL:

Thank you for your thorough and thoughtful review. Below, we have provided detailed responses to the identified weaknesses and questions, with similar points grouped together for clarity.

1. Weak benchmarks: the evaluated benchmarks are not complex enough to demonstrate the utility of the LEGO-Compiler.

To clarify, CoreMark is not a toy benchmark. Regarding ExeBench, although it is not very complex, it is a benchmark widely used for neural compilation tasks. For more details, please refer to the 1st response in the general reply. In short, we are actually pioneers in scaling up the capability of current LLMs for both neural compilation and neural code translation tasks. While the complexity of ExeBench itself is somewhat limited, we evaluate it primarily for fair comparison to related works, which also use ExeBench for evaluation.

Although ExeBench is not complicated in length, it covers almost all C language features, such as system calls and inline assembly usage, Achieving high accuracy on ExeBench is a foundation for evaluating more complex cases. Therefore, we also evaluate our LEGO-Compiler with a case study of CoreMark, which is significantly more complex and has been acknowledged as such by other reviewers. Additionally, we introduce the LongFunction benchmark to demonstrate translation scalability, which we explain in detail in the response to Question 6.

To further address your concerns, we are currently conducting experiments on more real-world C codebases to showcase the utility of LEGO-Compiler. We will update the results once the evaluation is completed!

2. The language features and library usage in the benchmarks are limited. System calls, such as fork and execve, require special handling in the assembly code. How does the LEGO-Compiler handle complex language features and library calls?

There seems to be a misunderstanding here. System calls and library calls are simply function calls from the compiler's perspective (including LEGO-Compiler). In the resulting assembly output, they are just calls with some additional calling conventions (for syscalls). We have not encountered significant issues caused by library function calls, as it is the programmer’s responsibility to handle the return values of these functions. Most errors we identified are analyzed in Appendix D.2, the ExeBench breakdown section. As for complex language features, some do require special handling, like the one we did for struct, union and array should be similar. Many features can be handled well by LLMs due to their pretraining, such as function calling ABIs, instruction-level translation, pointer operations, and indirect jumps. However, there are features with which LLMs still struggle. For example, handling expressions with very high depth, like a+(b*d/(c-e))/(f+g)-h, or architecture-specific knowledge like AVX intrinsics, poses challenges. We have discussed them in Appendix E, the limitation section. The link below is an example of how fork-exec example code can be neural-compiled correctly with CoTs, which can be easily reproduced:

https://www.perplexity.ai/search/i-want-you-to-act-like-a-compi-OPQbOBCXQkq2e2TvTwzuGA

3. Lack of comparison with other state-of-the-art methods.

Answer: Please refer to our 3rd response in the general reply, where smaller non-LLM neural compilers are performing worse than foundation LLMs, therefore we think it’s not necessary to compare with. We will later update our paper to include direct comparisons with Zhang, et.al, which wasn't available during our initial submission.

4. Without an understanding of the complexity of the benchmark, it can be hard to draw a conclusion on the effectiveness of the LEGO-Compiler. How does the LEGO-Compiler perform on benchmarks with over 1000 lines of code?

To address this question, first, as illustrated in Figure 4, the main function of CoreMark serves as a representative example with 2092 tokens, 215 LOCs, 48 basic blocks, and 535 instructions. This complexity surpasses the capabilities of all previous work; even our method without LEGO translation cannot compile it. Therefore, the effectiveness of LEGO translation is demonstrated by its capability to handle long and complex functions with lengthy code patterns.

For the question on how LEGO-Compiler performs on benchmarks with over 1000 lines of code, we need to clarify two things:

• 1)If the question refers to benchmarks with over 1000 lines of code per file or repository, CoreMark already qualifies, as it contains over 1000 lines of code across 40 functions, all of which are compiled correctly by LEGO-Compiler. We are also conducting further experiments on more real-world codebases, as mentioned in Question 1.

• 2)If the question refers to functions with over 1000 lines of code, it’s important to note that functions of this length are very rare in modern programming paradigms. Besides, we also conduct lengthy function experiments with LongFunction dataset that can be up to 37k LOC, which has shown that LEGO translation can reduce the complexity of function-level translation into finer granularity, which is the core contribution of our methods. The LongFunction dataset is provided in the supplementary materials, within the datasets folder.

5. Insufficient correctness evaluation on the generated assembly code, think we need SMT solver to guarantee full translation equivalence.

Please refer to our 4th response in the general reply.

6. The improvement with the LEGO-Compiler compared to baselines is not significant. Why is the improvement of the LEGO-Compiler over the baseline insignificant?

We would like to clarify that the LEGO translation method does not appear significant in ExeBench evaluation because the former CoT and error-feedback methods are able to handle most of the complexity in ExeBench, and it shows power when the code length is relatively long. However, calling this improvement insignificant is not fair, as the prior methods have already pushed the baselines to very high levels. We present the results on harder cases and show more significance for both CoT and LEGO translation methods in Table 2.

Another important point is that improvements in neural compilation capabilities are not linearly reflected in accuracy metrics, as different code cases have varying difficulties in compiling correctly. Therefore, it is more meaningful to assess in another view: how complicated cases can LEGO-Compiler translate correctly. If LEGO-Compiler can translate a very complicated case, LEGO-Compiler can translate simpler similar cases with high possibility. From this perspective, we believe the improvement is indeed significant.

Additionally, the significance of LEGO translation method is majorly tested through LongFunction evaluation, where we find that without the LEGO translation method, code translation task fails at 5k token input and compilation task fails at 2.6k token, even with powerful o1-preview model. However, equipped with the LEGO translation method, all code translation and compilation cases are passed, where the maximum function length can be over 238k. Although it is more like a needle-in-the-haystack experiment because the code patterns are simple in LongFunction, and each part translation is easy and somehow redundant, it showcases the scalability of LEGO translation because direct translation method can not handle such lengthy input. LEGO translation, on the other hand, significantly reduces the translation complexity and can handle lengthy translation. The details are provided in Appendix D.3.

7. Missing results for small LLMs and codestral.What are the limitations of applying LEGO-Compiler to small LLMs?

In our scenario, LLMs need to follow instructions provided in the prompt to perform complex tasks such as CoT translation and LEGO translation. We have observed that smaller LLMs struggle to follow the instructions required for LEGO translation. We are continuing to refine the LEGO translation framework to make it more robust and applicable across different LLMs. This is why we have not fully tested smaller LLMs.

As for Codestral, the situation is similar. It has difficulty following instructions in the CoT method, resulting in worse CoT outcomes compared to its pass@5 + feedback results.

Finally, thank you again for your detailed and thoughtful review. We hope the above responses address most of your concerns.

Sincerely,

Submission931 Authors

How many test cases are generated?

We actually try a bunch of different settings as we are also learning the usage of Csmith with your suggestions to evaluate with randomly generated code. In our current version, we generate 40 cases with the setting described in question2.

What settings do you use when generating programs with Csmith?

We use the following settings in the results of Figure 9 and Figure 10:

csmith --quiet --concise --no-volatile-pointers --no-builtins --no-volatiles --no-unions --no-checksum --no-jumps --no-bitfields --no-packed-struct --no-safe-math --no-global-variables --max-expr-complexity 2 --max-block-depth 3 --max-block-size 2 --max-funcs 1 --max-array-dim 2 --max-array-len-per-dim 10

We majorly limit settings, like union, global variables, volatile pointers, because Csmith tends to make the code significantly harder which is way beyond how well-formed C code is usually written. For example, Csmith will generate nested structs with unions and unions with structs, which can cause the reasoning in CoT fail and therefore, cannot help us evaluate the effectiveness of LEGO translation.

How many test cases does the LEGO compiler pass in total?

We pass 25 out of 40 cases, where all cases are generated in the same settings described in 2. The code generated by Csmith can vary hugely in its complexity as depicted in Figure 10. The failed cases are not presented in Figure 10 as the purpose of Csmith evaluation is to showcase the effectiveness of LEGO translation. Besides, the failures are mostly not caused by LEGO translation but the complexity of statement level translation. However, it is helpful as it does help us to reveal the challenging edge cases and patterns LLMs struggle to handle, like the assignment of overflow values (we discussed in Appendix D.5, line 1764-1771).

4. Compilation time analysis:

We acknowledge the importance of evaluating compilation time performance. Let us provide a quantitative analysis using a representative example. For a source program of 200 tokens, traditional compilers like GCC complete compilation in approximately 20ms. In contrast, our LLM-based approach using Llama-3.1-8B (deployed with vllm on an A100 80GB GPU) generates approximately 600 tokens of assembly output at a rate of 40 ms per token, resulting in a total processing time of about 20 seconds. For commercial LLMs like GPT-4o and Claude 3.5, the process is more effective, allowing them to compile code more quickly. The performance of function-level code compilation for these models is typically at the second level, which is 2-3 orders of magnitude slower than traditional compilers like GCC.

While this computational overhead is substantial, it aligns with the expected performance characteristics of decoder-only LLMs in translation tasks. Our primary objective is to explore and understand the fundamental capabilities of LLMs in program translation, rather than to compete with the efficiency of traditional compilers.

The use of iterative processes like pass@k and pass@k@fix_round and the inclusion of Chain-of-Thought reasoning could further increase the output token count and processing time. We now report the time analysis in ExeBench using GPT-4o:

In conclusion, the average pass time across all cases was 19.2 seconds per case, showing that our methodology is generally efficient, especially considering the complexity of neural compilation tasks. While the more advanced techniques, such as CoT and LEGO translation, took more time per case, they were necessary only for a small subset of challenging cases, and the majority of cases were handled swiftly. We will later reflect the above analysis to the manuscript.

5. More detailed results on different hardware architectures:

Performing full scale results on different hardware architectures would require substantial additional work, as the majority of our evaluation is based on x86_64. However, we have performed ARM and RISC-V experiments on the LongFunction benchmark, where the architecture related complexity is not influencing the results and the LEGO translation method performs all well on them. As for detailed results on ARM and RISC-V with more complicated cases like in ExeBench and CoreMark, we do find some interesting findings. For example, there are new semantic challenges like the restricted bit width of immediate values, so not all numbers can be expressed in the way the current model generates, and the calling convention of different architectures will cause some related errors as well.

We hope the above responses help address your concerns. Thank you again for your valuable review!

Sincerely,

Submission931 Authors

Dear Reviewer vsfE:

Thank you for your in-depth review. We appreiciate the time you spent assessing our work. Please find our responses to the weaknesses and questions below:

1. Contribution concern:

We believe there may be some misunderstandings regarding our contributions. Our work presents three key novel points: the use of CoTs in neural compilation, the implementation of error feedback mechanisms, and most importantly, we are the first to reveal and leverage the composable nature of code translation, proposing the LEGO translation method which can be applicable not only to neural compilation but to broader code translation tasks. It requires theoretical support to translate in finer granularity and then combine the results together, which we formalize as translation composability. We believe this is a significant novel contribution that can benefit the broader code translation community, enabling many LLM-based code translation tasks to scale up for production-level usage.

2. Standalone ablation study on LEGO translation method:

To clarify, the pass@k column in Table 1 only contains k times generation, and the block-by-block translation is applied only on the LEGO-Compiler column in Table 1, after CoT. Your suggestion on using different ablation orders when applying methods is helpful. Actually, ExeBench may not be complex enough to showcase the capability of our block-by-block LEGO translation method. Therefore we try to showcase the LEGO translation capability through the evaluation on the LongFunction benchmark we proposed. In this benchmark, without the LEGO translation method, code translation fails at 5k token input, and compilation fails at 2.6k tokens, even with powerful models like o1-preview. However, when equipped with the LEGO translation method, all code translation and compilation cases pass, with the maximum function length exceeding 238k tokens. Although this may seem like a "needle-in-the-haystack" experiment due to the simple and somewhat redundant code patterns in LongFunction, it demonstrates the scalability of LEGO translation. The method significantly reduces translation complexity and can be combined to form a full, lengthy translation. Details can be found in Appendix D.3. Additionally, we are currently extending our experiments to broader real-world applications and will ablate LEGO translation alone.

3. Comparison with previous work:

Please refer to our 3rd response in the general reply, where smaller non-LLM neural compilers are performing worse than foundation LLMs, therefore we think it’s not necessary to compare with.

As for comparison with Guo, et.al in C-LLVM IR experiments, we think the comparison with them using exactly matched setup is not necessary. However, we will conduct such experiments later by testing the meaningful C-LLVM IR generation results with ExeBench.

Dear reviewer vsfE:

We have now updated our manuscript and conducted more real-world C program evaluation and randomly generated program evaluation, with method ablations. We have updated the results in Appendix D.4 and Appendix D.5.

The results show the effectiveness of LEGO translation method as it scales up the complexity of code which LLMs are capable to compile by near an order of magnitude. We revise the ablation designs, where we think both the pass@k and the n round self-correction with error feedback are applicable to direct translation, CoT translation and LEGO translation, as they are internal capabilities within LLMs themselves, so we think they are better to be applied.

We hope the above updates address your concerns. If it is the case, we kindly invite you to reconsider your initial rating.

Sincerely,

Submission931 authors

3. Error Feedback and Self-Correction Limitations

Yes, the error feedback surely needs some development to support LLMs to do so, but not much, we will discuss the details: The assembler error can be easily captured by assembler toolchain and is the best self-correction type. As for runtime errors (usually a core dump), we need to use assembler to assemble with debug info(-g) to get the executable and capture its traceback from gdb execution, which can also help locate a range of possible errors. The silent error(mismatch output) is the one that LLMs are more struggling with, as even for humans, identifying silent errors is also challenging. Possible solutions include using binary search recursively in the code, and LLM simulating execution with the code, to locate the error position. This is an interesting topic waiting for further investigation.

4. Ablation of LEGO Translation Granularity

We use an adaptive algorithm (Algorithm 1) with error feedback to decide a suitable size for LLM to translate. From our investigation, translation with finer granularity will help the translation accuracy as it lowers the difficulty of each translation step, and it is the reason for us to use an adaptive way, from coarse-grained functions(no split) to fine-grained blocks or even statements(current split limitation).

5. Limited Discussion of Edge Cases and Limitations

Please refer to Appendix E for detailed limitation discussions. Besides, we will soon add more edge case failure examples to the manuscript to showcase the boundary of current methods with LLMs for clarity.

6. Potential for Fine-Tuning or Model Customization

Yes, compared to using ICL, fine-tuning the model with a compilation dataset should boost the results by learning the translation into parameters when performing direct translation. As it is already reported by [1], which is the SOTA in neural compilation task before our method. However, with only fine-tuning in raw dataset may not boost its accuracy in our LEGO-Compiler method as the instruction-following ability of the model may decrease during the fine-tuning process. Actually we prefer another perspective, where we don’t rely on existing compilers, we actually let LLMs reason the translation process and perform the translation as a “compiler”. Ultimately, we believe LLMs can reason the translation for other languages and architectures, as long as their monolingual corpora are pretrained, and few or no aligned corpora would be required. However, this is just for discussions and needs future work to further prove its correctness.

[1] Introducing Compiler Semantics into Large Language Models as Programming Language Translators: A Case Study of C to x86 Assembly (Zhang et al., EMNLP Findings 2024)

7. Comparison with Non-LLM Neural Compilers

Please refer to our 3rd response in the general reply, where smaller non-LLM neural compilers are performing worse than foundation LLMs, therefore we think it’s not necessary to compare with.

We are sorry the response is a bit lengthy. We hope these clarifications not only address your concerns but also give you greater confidence in your assessment of our paper's contributions to the field.

Sincerely, Submission931 Authors

Dear Reviewer KZKL:

We sincerely thank you for your thorough and insightful review of our paper. Your comprehensive summary and analysis demonstrate a precise understanding of LEGO-Compiler's core contributions. We are particularly impressed by your accurate articulation of our key innovations: the LEGO translation approach—a novel method that decomposes complex code into semantically composable blocks—and the annotation-based Chain-of-Thought mechanism, which, for the first time, introduces reasoning capabilities into neural compilation. Furthermore, to the best of our knowledge, LEGO-Compiler is the first work to enable LLMs to handle real-world, industry-grade code translation. These techniques are not only applicable to neural compilation but are also relevant to broader code translation tasks, particularly the LEGO translation method.

We note that while you've given our paper a strong accept rating (10), you've indicated a lower confidence level (2) in your review. We believe we can address this confidence gap, as your detailed comments demonstrate a solid grasp of our paper's central contributions and technical details, and a clear understanding of where our novelty lies. Your questions and constructive feedback are precise and well-targeted, suggesting a better understanding than you might have credited yourself with. We will address each concern:

1. Computational Efficiency and Scalability

Although our goal is not to replace traditional compilers, knowing its translation cost is indeed important. We will make a qualitative discussion here and later revise it in our manuscript. Using a 200 token program as example, GCC compiles in ~20ms, while generating 600 token assembly output for a LLM, say Llama-3.1-8B serving with vllm, its Time-Per-Output-Token is roughly 40ms/token for one A100 80GB GPU, therefore, will take 20 second for output, which is 2-3 orders of magnitude slower, using CoT will further increase the token output and therefore, being slower. This does show that using LLM to compile is costly compared to using modern compilers, however, as other machine translation tasks are also costly using decoder-only LLMs, we cannot blame them because knowing the translation capability with LLMs is also important. As for resource efficiency optimization, since LEGO translation is modular and can be translated separately, there could be optimizations by serving independent translation steps into different LLM chats, which could batch the translation steps together and speed up the generation. Functions can naturally be parallelized, and in our work, we further break down tasks into control blocks. However, this may require the frontend analyzer to be more carefully prompted or non-LLM implemented to support robust finer-grained translation parallelism. We believe this could be an interesting topic for future work.

2. Evaluation Breadth and Context

Currently we evaluate our LEGO-Compiler on C programming languages, and we also discuss the potential with other programming languages in Appendix E, the limitation section. There will be new challenges for different programming languages, however, we believe using similar CoTs and the LEGO translation method will definitely be helpful. We target more detailed study on different language compilation and translation as future work.

Dear Reviewer KZKL:

We have currently updated our manuscript and conducted both more real-world C program evaluation and randomly generated program evaluation, with method ablations. We have updated the results in Appendix D.4 and Appendix D.5, where the results do show the effectiveness of LEGO translation method. We also perform larger scale evaluation on ExeBench which shows consistent results.

Additionally, we find more edge cases and error types through the newly conducted evaluations, we discuss them in Appendix D.4.

We hope you find these newly updated results interesting and that they strengthen your confidence in your assessment of our work!

Best regards,

Submission931 authors

Dear reviewer KZKL:

Thank you again for your great review, and we would like to remind you that the discussion period is ending, and we haven’t heard from your responses where we have added several important experiments and have already addressed reviewer wpuL’s concerns.

We would really appreciate it if you could spent some time to review the rebuttal we have updated and consider raising your confidence score if more details are explained clearly from our clarifications. We will be happy to clarify any questions related.

Sincerely,

Submission931 authors

5. What do you mean by "semantically composable control blocks"?

We mean control block by full code patterns like if(){} and for(;;){}, etc, they are naturally encapsulated code patterns in modern PLs, and are LLM-aware since LLMs can reason their scope from bracket pairing, they are proved to be composable during translation, and they are a natural granularity level between strict basicblock and function. Function level neural translation is the majority and we think it is too coarse-grained for LLMs, making it hard to scale up. As for basicblocks defined in the compiler community, they are intuitively fine-grained and perform well in LEGO translation. However, we think control blocks are better semantic units as they match the full control keyword statements, naturally encapsulated code patterns, and LLMs are pretrained well to translate these control blocks.

6. What is the difference from how a traditional compiler works to "translate" each block separately, while maintaining a symbol table? What part of the compiler are you really replacing? The syntax-directed translation rules? What is the value in replacing these?

The translation of each block is inspired by how O0 compiler compiles, therefore, philosophically similar, the major differences are that we directly handle with source code and do the renaming, mapping, allocation ahead in the symbol table, which may differ from traditional way that usually operates at lower level like IRs or ASTs. We bypass the compiler part of compiler by using LLMs to translate from source code to assembly code. We treat neural compilation as a special case of neural code translation, therefore, both the AST building, traversing, syntax-directed translation are replaced because we fully rely on a LLM to compile/translate. The whole process is illustrated in Figure 2.

The values of these processes are:

• 1)We prove LLMs can follow carefully designed CoTs to do compiler tasks, which means the traditional compiler development knowledge can be taught to LLMs.
• 2)We are the first to realize that neural translation of function level code can be further split up into finer granularities (control block, basic block, or even statement), because of the translation composability nature, and therefore, be capable of handling reasonably sized programs by translating them separately.
• 3)We theoretically prove the translation composability, which is actually not a totally new theorem but an intuitive formalization of how compilers handle programs.

7. Who is responsible for allocating the addresses? For example, who assigns blksize to -36(%rbp) in Figure 1(b).

LLMs allocate the addresses, it is a reasoning process to sequentially allocate the stack addresses from the first variables to the last, it is majorly integer add/sub/mul mathematical computations to calculate an offset, and we found LLMs can handle them effectively as the CoT approach outperforms direct generation approach. The details of such process is illustrated in the Variable Mapping part in Figure 2.

Additionally, In our implementation, the whole CoT process is done by LLM, however, we agree that either the frontend part to form the translation rules and intermediate results or the backend part that do the translation of each part can be done by traditional methods, so there are exploration space for more suitable designs. Currently we implement a LLM-only approach, majorly to showcase LLMs’ capabilities in complex code translation tasks(compilation). We believe we study the hardest neural code translation tasks to the best of our knowledge.

8. Line 270-272, can you provide more details about this renaming pass? Is it aware of scoping rules?

The renaming pass first needs another analyzer pass that prompts LLMs to reason about every variable with their living scope. For scopes in C, LLMs majorly need to reason about every variable about its definition, since every variable has its own living scope, and the innermost definition will mask the rest. Therefore, for each identifier, the LLMs match the closest definition (from the current depth, cascading out to the global scope, similar to symbol resolution in compilers). Later after analysis, we rename them, eliminating duplicate names so every entry in the symbol table (shown on the upper right side in Figure 2) will be unique for later allocation. This process is done purely at the source code level, and therefore, can be easily checked for equivalence through unit tests.

It is aware of scoping rules. These rules are generally pretrained in LLMs, as we have found that LLMs can handle most renaming cases correctly in 0-shot. We further consolidate their understanding of scoping rules by providing a 1-shot ICL example.

We hope the above responses help address your concerns and give you a clear impression. Thank you again for your valuable reviews!

Sincerely,

Submission931 Authors

Dear Reviewer Mneu:

Thank you for your in-depth review. Please find the responses to the weaknesses and questions below:

1. What is the motivation for this work?

In short, our work is primarily a natural extension of neural code translation, with neural compilation being a special case. We first found that LLMs struggle to obtain very high compilation accuracy, so we experimented with various methods to improve this, and the methods (except the LEGO translation) are outcomes of these efforts. However, even with these improvements, we could not handle lengthy programs, even when the code consisted of simple repetitions, such as 100 for(i) loops handling simple statements. This motivated us to study the properties of programming languages. We also had an intuitive sense of the composability property we formalize, as this is the foundation of how compilers work. Then, we realized that the translation process is composable. Unlike natural languages, code may be complex in design, but translating well-designed code is not necessarily difficult. As a result, we proposed both the LEGO translation methodology and the LEGO-Compiler system. The former is applicable not only to neural compilation but to a broader range of code translation tasks (and possibly more), while the latter is a significant step forward for neural compilation, as it is the first to successfully translate real-world codebases like CoreMark.

2. 99% accuracy on a testset of 500 examples from ExeBench is very different from the "over 99% on ExeBench" (40k examples, or 23k examples after filtering). A different claim at a factor of x46.

We apologize for the simplification here. However, we believe using a subset of ExeBench is already representative because we find the difficulty distribution and accuracy results similar across ExeBench. A medium-sized subset can represent ExeBench well and allows us to perform more experiments efficiently. Since testing the full 23k dataset using commercial LLMs would require significant time and financial resources, we chose the current 500-sample subset for experiments with more LLMs and to ablate our methods. We are confident that the results will be consistent between the subset and the full 23k ExeBench. We are performing full scale ExeBench evaluation and we will report the results later.

3. What are the assumptions you are making about the programming language and the translation?

We need to clarify that our LEGO-Compiler can handle the scenario you provided easily, as it compiles each line one by one. The correctness of the code itself (where aliasing complicates programming and optimization) is the user’s responsibility, and since our method currently does not target optimizations, the translation of cases with pointer aliasing will still work correctly. The assumptions we make to programming languages are stated in Appendix B.5 and a few more in Appendix E.

int *p = malloc(sizeof(int));
int *q = p;
*p = 10;
*q = 20

The provided code example will translate to the following correct output by Claude 3.5, with similar outputs for other LLMs:

    pushq    %rbp
    movq    %rsp, %rbp
    subq    $16, %rsp        # Space for two pointers p and q
    # malloc call
    movl    $4, %edi         # sizeof(int) = 4
    call    malloc@PLT
    # p = malloc result
    movq    %rax, -8(%rbp)   # Store malloc result in p
    # q = p
    movq    -8(%rbp), %rax
    movq    %rax, -16(%rbp)  # Store p in q
    # *p = 10
    movq    -8(%rbp), %rax
    movl    $10, (%rax)      # Dereference p and store 10
    # *q = 20
    movq    -16(%rbp), %rax
    movl    $20, (%rax)      # Dereference q and store 20

4. How do you get the examples for ICL? What is "compiler knowledge guided CoT"?

The 1-shot example is carefully crafted to demonstrate the principles of CoT and to align the output. “Compiler knowledge guided CoT” refers to subtasks such as variable capturing, reasoning, renaming, stack allocation, etc. We call it compiler knowledge guided because we are inspired by compiler design experience. However, the CoT process itself does not depend on any specific compiler implementation, so it remains independent of existing compilers.

Dear reviewer Mneu:

We have just updated our pdf version with new evaluations showcasing the effectiveness of LEGO translation method. Besides, we also perform a 10x larger scale evaluation on ExeBench with Deekseek model, where the results are as expected, consistent between the 500 subset and the newly evaluated 5000 subset. Due to time constraint we cannot perform full scale 23k ExeBench evaluation, however, we can make this update in later times after the rebuttal period.

We hope the above updates address your concerns. If our answers are more in line with your expectations, we kindly invite you to reconsider your initial rating.

Sincerely,

Submission931 authors

Dear reviewer Mneu:

We would like to remind you that the discussion period is ending, and we haven’t heard from your responses where we have added several important experiments and have already addressed reviewer wpuL’s concerns.

We would really appreciate it if you could spent some time to review the rebuttal we have updated and update your review. We will be happy to clarify any questions related.

Sincerely,

Submission931 authors