Compiler-R1: Towards Agentic Compiler Auto-tuning with Reinforcement Learning

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.

Q1: How do the trained GRPO models compare to the initial decision tree strategy used in SFT dataset construction in OverOz and time?

Thank you for this insightful question. The trained GRPO models significantly outperform the initial decision tree strategy used during the SFT dataset construction, both in terms of OverOz (IR instruction count reduction) and time efficiency.

Key differences:

• OverOz Improvement: The GRPO models, trained with Reinforcement Learning (RL), adapt and optimize the pass sequence based on feedback from the environment, whereas the decision tree strategy is deterministic and lacks dynamic exploration. As a result, the GRPO models, such as GRPO-7B, achieve an 8.46% average reduction in IR instruction count, significantly outperforming the decision tree-based approach, which has more limited optimization capabilities.
• Time Efficiency: The GRPO-7B model is able to complete each program in approximately 26 seconds, making it considerably more time-efficient than the decision tree strategy and traditional autotuners. This demonstrates that the GRPO models are not only better at optimizing pass sequences but also faster in execution, thanks to their dynamic optimization approach.

This comparison clearly illustrates the advantages of GRPO models in both optimization performance and computational efficiency.

Q2: How do the trained GRPO models compare to SFT and RL ablations in OverOz and time?

Thank you for this important question. The GRPO models consistently outperform both SFT-only and RL-only models in terms of OverOz (optimization performance) and time efficiency.

• OverOz Comparison:
  • The GRPO-7B model achieves an 8.46% average OverOz improvement, significantly outperforming the best-performing SFT-only model (which achieves up to 4.66% OverOz improvement).
  • RL-only models, without the SFT initialization, show weaker performance because they lack the structured reasoning framework provided by SFT.
• Time Comparison:
  • The GRPO-7B model completes each program in about 26 seconds, which is comparable to the runtime of SFT-only models (29-55 seconds), and is much faster than traditional autotuners like OpenTuner (over 200 seconds).
  • The RL-only models, due to the lack of SFT pre-training, require more time to converge as they rely solely on trial-and-error interactions, which makes them less time-efficient.

In conclusion, the GRPO models offer a balanced trade-off, delivering both high optimization performance and low runtime, making them highly efficient for practical compiler auto-tuning tasks.

Q3: Some experimental details need to be clarified. The repetition penalty is not explained anywhere in the paper. Is it applied at the level of tokens or some higher level? How were the weights of the reward components (1.5 and alpha) set and how sensitive is the method to them?

Thank you for the opportunity to clarify these details:

• Repetition Penalty:
  • The repetition penalty is applied at the token level during the generation of pass sequences in the GRPO model. This penalty discourages the model from generating repetitive or redundant tokens, promoting more diverse and effective optimizations. By applying the repetition penalty, we prevent the model from getting stuck in suboptimal cycles and encourage exploration of a broader range of possible solutions.
• Reward Component Weights:
  • The reward function is a composite function consisting of two components: the format reward (Rformat) and the answer reward (Ranswer).
  • The weight for the format reward is set to 1.5, which prioritizes the correct reasoning format and tool-invocation structure. This weight was empirically chosen to ensure the model learns to correctly structure its reasoning and interactions.
  • The weight for the answer reward is represented by alpha, which controls the importance of IR instruction reduction in the reward function. Alpha is set to 30, reflecting the emphasis on optimization quality (instruction reduction) while balancing the importance of reasoning and tool usage.
• Sensitivity to Reward Component Weights:
  • Our experiments show that the method is relatively stable to small changes in the weights of the reward components. We conducted sensitivity analyses by adjusting 1.5 and alpha within a reasonable range (±20%) and found that the model's performance remained robust. However, large deviations from the chosen values led to slight performance degradation, highlighting the importance of these weights in shaping the agent's behavior without making the model overly sensitive to them.

We believe that these clarifications provide a deeper understanding of our experimental setup and the factors that contribute to the effectiveness of Compiler-R1.

Q4: Are the RL-only baselines instructed (prompted) with the same interaction protocol? Do the RL-only baselines collect their own on-policy data?

Thank you for these excellent questions. We are happy to clarify the specifics of our ablation studies:

1. Interaction Protocol:
   • Yes, the RL-only baseline was prompted with the exact same instruction prompt as our full Compiler-R1 models. This prompt defines the task, the available tools (instrcount, find_best_pass_sequence), and the required think -> tool_call -> response interaction protocol. We ensured that the RL-only agent had all the necessary information about the interaction framework from the start.
2. On-Policy Data Collection:
   • Yes, the RL-only baseline follows the standard on-policy reinforcement learning procedure. The agent generates trajectories based on its current policy, and this data is used to update the policy. This process is identical to the RL stage of our full Compiler-R1 framework.

The poor performance of the RL-only baseline, despite being prompted with the correct instructions and following the on-policy learning loop, highlights the difficulty of learning both complex interaction protocols and semantic reasoning for compiler optimization simultaneously. The agent struggles to find a meaningful policy due to the large, sparse-reward search space. This underscores the importance of SFT pre-training, which provides a "warm start" for the model, making the subsequent RL fine-tuning much more tractable and effective.

Q5: Why not simply include the FBPS results in the initial prompt to the LLM rather than require a separate tool call?

Thank you for this excellent question. The suggestion of pre-computing FBPS results and including them in the prompt is a valid approach, but we believe our interactive framework offers significant advantages:

• Efficiency: Pre-computing FBPS results for every program would enforce a rigid workflow and potentially lead to inefficient use of resources. Our approach allows the agent to dynamically decide when to invoke FBPS, ensuring that this computationally expensive step is only used when necessary.
• Flexibility and Extensibility: Our framework is designed to be extensible to include multiple optimization tools in the future. The FBPS tool is just one example, and we envision a future where the agent can choose from a variety of tools based on the specific characteristics of the program being optimized. This would make the system more generalizable and powerful for a wide range of optimization tasks.

We aim to create an adaptive, problem-solving agent that can intelligently select the right tool for the task at hand, rather than simply relying on pre-computed results. This flexibility is key to making our approach more robust and capable of handling complex, real-world optimization tasks.

We hope these clarifications may help to address your concerns. At last, we sincerely appreciate your valuable feedback, and we will carefully consider all your suggestions to further improve our paper. We would be deeply grateful if you could kindly reconsider raising the score. Thank you very much!

Q1

Thank you for your thorough follow-up. Here is a concise analysis to clarify these points.

The Source of Improvement

The final 8.46% improvement is a composite of(We will include the specific proportions of each category in the revised version.):

• The agent correctly applies the efficient SFT decision-tree logic in most cases, including when the initial sequence is good (No.1; and improves further as training progresses) and returned directly, as well as when the initial sequence is poor and FBPS is invoked as a fallback to ensure baseline performance (No.2).

• A crucial subset of cases where the agent, deviating from the SFT decision tree, independently learns emergent, adaptive strategies from scratch through RL training—strategies like "Maximization" and "Robustness" that were not present in the training data—demonstrating its intelligence (No.3).

Without RL, No.3 would not exist, and other potentially better deviation strategies would be unlikely to emerge during training. RL also enables the gradual improvement of initial sequences (No.1), which is difficult to achieve with a purely hardcoded approach.

Why Hardcoding Fails

Unlike a fixed hardcoded rule, we agree that alternative simpler strategies could be considered—for example, deciding the threshold based on program features, or letting the LLM only generate the initial pass guess while subsequent steps follow hardcoded logic. However, some individual deviation strategies might be hardcoded in isolation, but the very capacity to produce diverse deviation strategies in the first place comes from the agent’s learned intelligence—and that property itself cannot be hardcoded. After all, it would be impractical and rigid to hardcode every possible deviation strategy; what we seek is precisely the adaptability and intelligence that a large model provides, enabling adaptation beyond any fixed set of rules.

OverOz Thresholds

We apologize for not having explored retraining the model with thresholds other than 0 yet, due to limited time. If needed, we will address this in the revised version.

Q3

Of course. We ran an ablation study by setting the repetition penalty to 1.0, which effectively disables it. This serves as our baseline for "without penalty."

The average performance across our validation benchmarks without the penalty was as follows:

Q5

Thank you for the insightful question.

1. Why the agent does not always call FBPS

The agent's decision is governed by maximizing $E[R_{final}] = 0.5 * R_{format} + 15 * OverOz$.

• Path A (Efficient - No FBPS): This simple path is very reliable, meaning the probability of correctly executing the format, $P(format_{ok_A})$, is close to 1. Its expected reward is high and stable: $E[R_{final_A}] ≈ 0.5 * R_{format_{max}} + 15 * OverOz_{initial}$
• Path B (Maximizing - Call FBPS): This path is longer and more complex, introducing a higher risk of a format error, so $P(format_{ok_B})$ < 1. Its expected reward has higher variance: $E[R_{final_B}] = 0.5 * R_{format_{max}} * P(format_{ok_B}) + 15 * OverOz_{FBPS}$

The agent learns to choose the "risky" Path B only if the expected gain in performance $15 * (OverOz_{FBPS} - OverOz_{initial})$ is large enough to compensate for the potential loss of the format reward $0.5 * R_{format_{max}} * (1 - P(format_{ok_B}))$. This inherent calculation prevents the agent from developing a degenerate policy of always calling FBPS.

2. The Strong Behavioral Prior from SFT:

Additionally, the SFT phase instills a powerful "early-exit" behavioral prior. The RL agent starts with this efficient default and only learns to deviate for specific problem types where the reward signal for a more complex strategy is consistently strong.

Our RL training uses a discount factor γ = 1, so preference for shorter trajectories comes from the reward’s risk-reward trade-off, not discounting. We will include all RL hyperparameters and this analysis in the revised paper.

Concern

Thank you for your valuable suggestion. We acknowledge the importance of including runtime performance and compute cost in evaluating Compiler-R1. While our current work focuses on IR instruction count reduction, we plan to investigate runtime metrics and computational costs such as energy consumption and FLOPs in future research to provide a more comprehensive assessment.

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.

W1: The paper focuses on reducing instruction count, but how desirable of a quality is this?

Thank you for this insightful comment. We agree that IR instruction count does not always correlate directly with runtime or energy efficiency. However, it remains a widely used optimization targetwith practical value, as reducing instruction count often leads to leaner code, faster compilation, and better optimization opportunities downstream:

• Code Size Reduction: This is crucial for resource-constrained environments such as embedded systems, mobile devices, and IoT applications, where minimizing code size can save memory and storage, as well as reduce the cost of over-the-air updates.
• Compile Time: A smaller IR typically leads to faster compilation times, which can significantly improve developer productivity in large-scale projects.
• Performance Proxy: Although it is not a perfect metric, reducing instruction count often leads to simpler control flow and fewer redundant computations, which in turn frequently improves the runtime performance.

We will clarify this in the revised manuscript by discussing the empirical correlation between instruction count reduction and runtime performance. Additionally, we plan to extend our future work to consider multi-objective optimization, where runtime, energy efficiency, and correctness will be incorporated into our reward design.

W2: The comparisons in Table 1 do not seem entirely fair. Traditional autotuners are indeed given more time, but how much does the compute differ?

Thank you for bringing up this point. We understand that the comparison between Compiler-R1 and traditional autotuners, especially in terms of time and compute, requires further clarification. Here are the key aspects of the computational cost comparison:

• Compute Costs: Our method, Compiler-R1, relies on inference with LLMs. A single forward pass with a 7B model involves dense matrix multiplications on GPUs, which, while computationally intense, is massively parallelizable and completes in seconds.
• Traditional autotuners, in contrast, use heuristic search algorithms on CPUs. While the per-compilation cost is relatively lower, these autotuners perform hundreds or thousands of evaluations sequentially, leading to much longer total runtimes (e.g., 200 seconds for OpenTuner, and over 9000 seconds for CompTuner in our experiments).

We will update the paper to clarify this difference in computational paradigms and discuss how time-to-solution is often the most critical metric for developers. While FLOPs comparison is difficult, our approach delivers an order-of-magnitude faster time-to-solution using parallelizable GPU computations.

Q1: It is surprising to me that with a very simple SFT data, the model is capable of thinking via RL to a fairly large improvement over just SFT.

Thank you for this excellent and nuanced question. You are right to point out that the SFT data appears structurally simple, but we'd argue this simplicity is by design and is crucial for the overall success of the framework.

The perceived simplicity lies in the consistent think-tool-answer format. This structure is intentionally kept straightforward to ensure the LLM can reliably learn the fundamental interaction protocol. The true complexity and value of our SFT data lie not in the format, but in the reasoning logic embedded within the trajectories. Each target sequence is derived from our synergy-graph analysis, representing a high-quality, non-obvious optimization solution. More importantly, the data explicitly teaches the agent a critical, non-trivial skill: how to interpret tool feedback to make a conditional decision—either accept its own solution or escalate to the FBPS tool.

Therefore, the SFT stage provides a robust foundation, not a simplistic one. It solves the difficult "cold-start" problem by equipping the agent with a solid grasp of the task mechanics and a strong baseline reasoning strategy.

With this foundation in place, the RL stage can focus entirely on refining and transcending that strategy. The agent is no longer struggling to learn how to interact; it is now learning how to optimize better. It explores variations of the strong patterns learned during SFT and fine-tunes its policy based on live feedback, for instance, learning to be more confident in its initial guesses for certain program types. This focused, high-level learning is what accounts for the significant performance improvement over SFT alone.

In response to your request for an example of model reasoning, we will include an example of the RL-tuned model's thought process in the appendix of the final version of the paper. This will illustrate how RL improves the model's reasoning beyond simple imitation.

Assistant: 
<think>
Based on the provided autophase features, I will try the following pass sequence:
['--loop-simplifycfg', ...]

I'll use the instrcount tool to check if this sequence provides better optimization than -Oz.
</think>


[Tool Call] Function: instrcount
Arguments:
{
  "filename": "xxx",
  "optimization_flags": [
    "--loop-simplifycfg",
    ...
  ]
}

Tool: {"filename": "xxx", "status": "success", "improvement_over_oz": -8.695652173913043%}

Assistant: <think>
The pass sequence did not provide better optimization than -Oz, as improvement_over_oz is negative (-8.695652173913043%).

I will now call the find_best_pass_sequence tool to find a better sequence.
</think>



[Tool Call] Function: find_best_pass_sequence
Arguments:
{
  "filename": "xxx"
}

Tool: {
  "status": "success",
  "best_pass_sequence": [
    "--loop-interchange",
    ...
  ],
  "improvement_percentage": -0.79%
}

Assistant: <think>
The find_best_pass_sequence tool returned a new pass sequence, but the improvement_percentage is -0.79%, indicating a decrease in performance.

Since neither the initial sequence nor the newly found sequence provide positive improvement over -Oz, I'll recommend the standard -Oz optimization as my final answer.
</think>
<answer>
['-Oz']
</answer>

Q2: Could you make a comparison between your Compiler-R1 and traditional autotuners with respect to FLOPs?

Thank you for this sharp and important question. A direct FLOPs comparison is exceptionally challenging due to fundamentally different computational paradigms. Here is our analysis of the cost structures:

• Difficulty of Direct Comparison: A precise, apples-to-apples FLOPs count is difficult. Our method's cost is dominated by a few highly parallelizable LLM forward passes on a GPU. Traditional autotuners, conversely, perform thousands of less-intensive but often sequential program compilations on a CPU. These distinct processes do not map cleanly for a direct numerical comparison.
• Contrasting Computational Patterns:
  • Compiler-R1 (Inference-Heavy): Executes a predictable burst of billions of FLOPs that completes in seconds on modern hardware.
  • Traditional Autotuners (Search-Heavy): Their cumulative, latency-bound search process, involving thousands of evaluations, results in very long runtimes (200s to over 9000s), despite lower per-evaluation intensity.
• A More Holistic View of "Cost":
  • Time-to-Solution: From a developer's perspective, our method offers an order-of-magnitude speedup, which is often the most critical factor.
  • Amortized Cost: Our model has a high one-time training cost but becomes a reusable asset for fast inference. Traditional tuners incur their high search cost for every new program.

While our method requires GPU access, the rapidly improving accessibility of LLM inference makes this approach increasingly viable. In conclusion, Compiler-R1 makes a favorable trade-off, leveraging a higher burst of parallelizable computation to achieve a dramatically lower and more practical time-to-solution. We will add a detailed discussion of this cost analysis to the paper.

Q3: What are the takeaways from this work?

Thank you for this excellent question. Here are the key takeaways from our work:

1. Reframing the Phase-Ordering Problem: We present a novel approach to compiler optimization by framing the phase-ordering problem as a reasoning task solvable by an interactive LLM agent. This represents a paradigm shift from traditional search-based methods to adaptive, intelligent decision-making.
2. Value of Instruction Count Reduction: While not a perfect measure, reducing IR instruction count offers several real-world benefits:
   • Smaller binary size for resource-constrained systems (e.g., IoT, mobile devices).
   • Reduced compile time, improving developer productivity.
   • Potential runtime improvements, as reducing instruction count often leads to more efficient program execution.
3. Practical Impacts of Auto-Tuning: We demonstrate that auto-tuning is not only valuable in terms of performance but also in terms of developer efficiency. Our framework offers faster time-to-solution and can be adapted to other multi-objective optimization goals in future work.

We will also add a discussion of real-world applications and future directions in the revised paper to clarify the broader impact of our work.

We hope these clarifications may help to address your concerns. At last, we sincerely appreciate your valuable feedback, and we will carefully consider all your suggestions to further improve our paper. We would be deeply grateful if you could kindly reconsider raising the score. Thank you very much!

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.

W1: Novelty in terms of training

We appreciate the reviewer’s recognition of our work. While the combination of SFT followed by RL is not novel in isolation, the novelty in our paper lies in the integration of these techniques within the specific domain of compiler auto-tuning, a domain where such methods have not been explored previously. Traditional methods for compiler pass optimization rely on fixed heuristics or simple rule-based systems. Our framework leverages RL-driven LLMs, moving beyond static strategies to adaptive, autonomous, context-aware optimizations.

The novel contributions of our work can be summarized as:

• Creation of a high-quality reasoning dataset: Our dataset is specifically curated for compiler auto-tuning tasks, combining Chain-of-Thought (CoT) reasoning and tool-assisted feedback for effective LLM training. This dataset represents the first attempt to build a comprehensive reasoning dataset for this domain.
• First end-to-end RL framework for compiler pass optimization: While the combination of SFT and RL has been applied in other areas, our work is the first to integrate RL with LLMs in compiler auto-tuningtasks, offering an innovative framework to autonomously discover optimized pass sequences beyond static or heuristic-based methods.

We believe that the combination of these components provides a fresh perspective and contributes significantly to the advancement of compiler optimization. To address your suggestion, we will make the novelty of our work more explicit in the revised manuscript.

W2: Reward design is not deep

Thank you for highlighting this aspect. While IR instruction count serves as a useful proxy for optimization and is widely used in compiler optimization research, we fully acknowledge the importance of a broader reward design for real-world relevance.

• Current reward design: Our choice of IR instruction count is intentional due to its deterministic nature and wide acceptance in compiler research. It provides a clear, reproducible metric for assessing the optimization quality of the agent’s output.
• Future plans for multi-objective optimization: We recognize the importance of multi-objective optimization, and as part of ongoing work, we are exploring how to integrate additional metrics such as runtime, energy consumption, and code correctness into our framework. These can be easily incorporated by adjusting the reward function and replacing the current tools with ones that track the desired metrics.

By incorporating these objectives, we aim to improve the real-world applicability of Compiler-R1 and provide a more comprehensive optimization framework.

W3: Details about the RL training pipeline

We appreciate your request for further details about the training pipeline. We are happy to provide more details of the two-stage training process. As outlined in Figure 2, we first use SFT to initialize the model’s reasoning ability. This is followed by RL, where the model further refines its strategy based on feedback from interactions with the compilation environment.

Key aspects of the RL training include:

• RL algorithms: We use GRPO, PPO, and RPP to fine-tune the model. These methods are applied to iteratively improve the model’s policy, enabling more efficient optimization strategies.
• Training Convergence: Our RL training generally converges within 40 steps, showcasing the efficiency of the pipeline. We will provide convergence metrics and additional details on the training dynamics in the revised manuscript.

We will also include a more detailed analysis of the training dynamics, and other important metrics in our revised manuscript.

Q1: How errors or noise in tool execution are handled?

Thank you for raising this crucial point. The handling of errors and noise in tool execution is a vital component of ensuring the robustness of Compiler-R1.

Our approach to error handling is as follows:

• Error Detection: If a tool (e.g., instrcount) encounters an invalid pass sequence, it returns an "error" status. The model is trained to recognize this and take appropriate corrective action.
• Penalization for errors: Instead of ignoring errors, the RL reward function penalizes any trajectory that leads to errors or tool failures. This negative feedback helps the model avoid repeating mistakes and promotes more robust interactions in future iterations.

Rather than dropping errors, we treat them as negative feedback and use them to improve the robustness of the agent’s decision-making process. We will add a more detailed explanation of this error-handling mechanism in the final version paper.

Q2: Comparison of matched performance between SFT and RL

We acknowledge the importance of a fair comparison between SFT and RL models. To address the concern, we refer to Figure 3 in our paper, which shows that RL models significantly outperform SFT models:

• RL models (e.g., GRPO-1.5B) achieve higher OverOz% improvement compared to SFT models when both are allowed only a single interaction trace (as shown in Figure 3, at N attempt = 1), demonstrating the effectiveness of RL-based exploration.
• As highlighted in Figure 3, the SFT models have already converged, yet they still underperform compared to RL models with just one interaction round, indicating that further gains from SFT are unlikely, while RL continues to improve through active exploration.

In summary, our results highlight the clear advantages of RL models over SFT models, achieving better optimization with fewer interactions. As shown in Figure 3, RL-based exploration is more efficient and effective in optimizing compiler pass sequences, underscoring the value of RL in complex tasks like compiler tuning.

Q3: Can the system adapt to newer LLVM versions (e.g., v17)?

Thank you for your query regarding forward compatibility with newer LLVM versions. We are confident that Compiler-R1 can be adapted to newer LLVM versions, such as LLVM v17, with minimal effort.

Here’s how we ensure adaptability:

• Version-Agnostic Principles: The fundamental "phase-ordering problem" we solve persists and even grows in complexity with newer compilers. Our framework's core value—learning program-specific strategies to outperform standard heuristics like -Oz—remains highly relevant and is not tied to a specific LLVM version.
• Modular design: Our framework is designed for modularity. To adapt to a newer version like LLVM 17, the only required step is to update the underlying toolchain. We would simply replace the opt binary in our execution environment with the new one. The core learning pipeline remains unchanged and can immediately leverage the updated compiler, including its new passes and improvements, without modification.

We will emphasize the modularity and discuss the adaptability of our framework in the final version paper to highlight these strengths.

Q4: Consideration of curriculum learning

We appreciate the suggestion to explore curriculum learning. While our use of AutoPhase features mitigates some of the complexity variations across programs, we acknowledge that curriculum learning could be useful, particularly in early RL training stages.

• Current approach: The AutoPhase features provide a uniform input representation, reducing the need for curriculum learning.
• Future work: We plan to investigate the use of curriculum learning to better handle low-complexity programs in future experiments.

We will include this idea as a future direction in the revised manuscript.

Q5: IR instruction count as a proxy for optimization

Thank you for raising this important point. We agree that IR instruction count as a sole metric has limitations. We chose it because it is a deterministic, fast-to-measure, and standard proxy in compiler research, enabling rapid and reproducible experiments.

However, a key strength of our framework is its flexibility regarding the optimization objective. Compiler-R1 is not fundamentally tied to IR count and can be readily adapted to target metrics like runtime or power efficiency. This adaptation requires three straightforward changes:

• Re-construct the Dataset: Generate new target sequences for the SFT data based on the new objective (e.g., best runtime).
• Update the Reward Signal & Tools: Replace the instrcount tool with a runtime measurement tool and adjust the RL reward accordingly.
• Modify the LLM's Prompt: Change the agent's instructions to reflect the new goal (e.g., "minimize execution time"). The core agentic framework and its two-stage training pipeline remain unchanged, highlighting its versatility.

We acknowledge the importance of real-world performance, and in future work, we plan to extend our framework to include multi-objective optimization, incorporating runtime and energy efficiency alongside instruction count.

We hope these clarifications may help to address your concerns. At last, we sincerely appreciate your valuable feedback, and we will carefully consider all your suggestions to further improve our paper. We would be deeply grateful if you could kindly reconsider raising the score. Thank you very much!

Thank you very much for your time and effort in reviewing our paper. We sincerely appreciate your thorough feedback and insightful suggestions. Below, we respectfully provide our detailed responses to address your concerns.

W1 & Q1: Contributions of the paper wrt the dataset, algorithmic ideas and empirical improvements over previous SOTA on CompilerGym.

Thank you for this question. We are happy to clarify our novel contributions:

• On the Dataset: We present the first high-quality, reasoning-centric dataset tailored specifically for compiler auto-tuning tasks. Our key innovation lies in the systematic methodology used for constructing the dataset:

  • We curate a global knowledge graph by identifying synergistic pass pairs, which then serve to guide the generation of optimal pass sequences.
  • We introduce a Chain-of-Thought (CoT) framework, teaching the model not only what the answer is but also how to reason and use tools effectively to derive it.

  This structured approach addresses the long-standing lack of reasoning data in complex, tool-intensive domains like compiler auto-tuning.

• On Algorithmic Ideas: The core innovation of our work is the introduction of Compiler-R1, the first LLM-based agent specifically designed for compiler auto-tuning. While the two-stage approach of SFT followed by RL is not novel in isolation, our application of this combination within the domain of compiler optimization is the novelty. Traditional approaches to compiler pass optimization typically rely on heuristics or static strategies, whereas our agent uses interactive reinforcement learning to autonomously refine optimization strategies, thus moving beyond imitation learning. This makes Compiler-R1 the first interactive agent in this domain, marking a significant leap forward.

• On Empirical Improvements: Our results demonstrate state-of-the-art performance for LLM-based auto-tuning. Specifically:
  • Compiler-R1 (GRPO-7B) achieves an 8.46% average reduction in IR instruction count compared to the opt -Oz baseline. This performance significantly surpasses the strongest SFT-only baseline (4.66%) and traditional autotuners like OpenTuner (6.09%).
  • We also highlight the computational efficiency of our framework: Compiler-R1 completes the task in ~26 seconds, much faster than traditional methods that can take over 200 seconds.

These contributions not only stablish Compiler-R1 as a leading framework in LLM-driven compiler auto-tuning, but also provide a blueprint for creating interactive agents capable of solving complex, tool-assisted tasks in diverse domains.

W2 & Q2: How well does Compiler-R1 generalize to programs outside its training distribution?

Thank you for this excellent question, which addresses a critical aspect of our work. Our design choices have been made to ensure Compiler-R1 is inherently more generalizable, even to large programs, and we believe our approach offers strong potential for generalization outside of its training distribution.

1. Rationale for Focusing on Programs Under 10k Instructions

First, it’s important to clarify our decision to focus on programs with fewer than 10k instructions. This was primarily driven by the need for a fair and feasible comparison with traditional auto-tuning baselines such as BOCA and CompTuner. These methods often require extensive time to tune even a single moderate-sized program, and the time cost becomes prohibitive when scaling to larger programs. Limiting the size to <10k instructions allowed us to perform a rigorous and direct comparison across all methods, ensuring the comparability of results. However, we acknowledge the importance of testing on larger programs and are exploring this as a next step.

2. Generalization to Larger Programs with AutoPhase Features

Second, Compiler-R1 is uniquely positioned to handle larger programs, thanks to its use of a fixed-size, 56-dimensional AutoPhase feature vector. This design choice has profound implications for generalization and scalability, offering two key advantages:

• Decoupling Program Size from Input Size: Whether a program has 1,000 instructions or 1,000,000 instructions, it is represented by the same compact 56-element vector. The program’s size only affects the numerical values within this vector, meaning the model does not rely on processing raw code directly. This decoupling bypasses context window limitations that typically hinder models based on raw code representation, enabling better scalability.
• Encouraging Scale-Invariant Learning: The model learns to make decisions based on structural patterns and proportional relationships reflected in the features, such as the ratio of branch instructions to memory instructions. These structural signatures tend to be shared across programs of different sizes. By focusing on the "shape" of the program rather than its absolute length, the model can learn generalizable optimization heuristics, which makes it more robust to varying program sizes and complexities.

In essence, the task of generalization is simplified from understanding arbitrary-length code to mapping points within a structured, low-dimensional feature space. This approach inherently promotes generalizable and scalable learning, and while we acknowledge the need for empirical validation on larger program benchmarks, we are confident that this feature representation will provide a strong foundation for out-of-distribution robustness.

We will add this clarification to our paper to emphasize how our feature-based design enables generalization and why we believe it positions Compiler-R1 well for scalability to larger programs.

We hope this response addresses your concerns and demonstrates the novelty and robustness of Compiler-R1. We truly value your feedback and will incorporate your suggestions to further improve our paper. Thank you once again for your careful and constructive review!

I thank the authors for their detailed response, which have alleviated my main concerns about the approach. After reading the other reviews and comments, I continue to recommend acceptance of the paper, maintaining my current score.