Question 1: Other ways of encouraging exploration and approaches to maintain the correct format for math or code

A-1: Other ways of encouraging exploration

Thank you for this insightful question. Indeed, we did experiments with several alternative sampling strategies. we tested various temperature settings (e.g., T=0.8, 1.0, 1.2) with top-p sampling and also explored more recent methods like min-P[1]. Empirically, the default configuration of top-p sampling(T=1.0, top-p=0.95) paired with our method yielded consistently good performance.

Reference list:

[1]. Nguyen, M. N., Baker, A., Neo, C., Roush, A., Kirsch, A., & Shwartz-Ziv, R. (2024). Turning up the heat: Min-p sampling for creative and coherent llm outputs

A-2: maintenance of correct format for math or code

Compared to fixed structured output or limiting token vocabulary, our primary mechanism is to integrate system prompt template and the precisely aligned reward mechanism.

Inspired by DeepSeek-R1's practices, we use the system prompt o guide the overall output format. Correspondingly, we explicitly define a ``format reward'' derived from successfully parsing specific delimiters such as <model>...</model> and <python>...</python>, alongside an ```execution reward'' for runnable code. This design directly encourages the policy to maintain and improve the correct format through active learning and feedback.

This reward-based approach, combined with our selective KL penalty for policy control, provides a more general and adaptable solution for LLMs in optimization modeling, accommodating the diverse programming language styles of various solvers, e.g., Gurobi, Cplex, COPT.

---

Question 2: Effectiveness of Val_sc

Response:

Thank you for this insightful question. The key reason "inst_sc" outperforms "val_sc" is that it moves beyond comparing only the final objective value. "Val_sc" is a "black-box" method that can be misled when multiple, structurally different (and often incorrect) model formulations happen to produce the same numerical answer by coincidence. "Inst_sc" provides a more discerning evaluation by enforcing consensus on the underlying mathematical structure itself, using features extracted from the solver's LP file.

A primary mistake made by top-scoring solutions chosen by "val_sc" is incorrect variable types. LLMs often struggle with the semantic nuance between a binary choice (e.g., "to build a facility or not," requiring a binary variable) and a general discrete quantity (requiring an integer variable). A solution might incorrectly use an integer variable, but if the optimal value happens to be 0 or 1, "val_sc" would incorrectly validate it based on the objective value. Other errors include using the wrong optimization direction (e.g., maximization instead of minimization).

We identified "inst_sc" as the appropriate strategy after observing these specific failure modes. Since the LP file provides objective, structured statistics of the model, we hypothesized that incorporating its structural features (like binary/integer variable counts and optimization direction) into the consensus check would directly target these common LLM errors. Our experiments confirmed this, showing "inst_sc" effectively filters out these structurally flawed models and improves accuracy.

---

Question 3: Two-stage RL Training Implementation

Response:

The training process involves two distinct stages, both utilizing the exact same training dataset.

• In stage 1, we utilize the specific 'stage-1 reward' signal to guide the training.
• For stage 2, training commences from the checkpoint obtained at the conclusion of Stage 1.
• Both stages required approximately 24×8 H100 GPU hours for completion, totally 48×8 H100 hours.

---

Weakness1 and Weakness2: Motivation and Limited Novelty

Response:

Thank you for your constructive and essential feedback. We appreciate your perspective on the novelty and motivation of our work. We want to clarify that our core contribution lies not in proposing entirely new RLVR mechanisms in isolation, but in critically adapting and refining existing RLVR techniques to overcome fundamental limitations of LLMs when tackling complex optimization problems that necessitate external solver interaction.

While LRMs (e.g., DeepSeek-R1) excel at sequential reasoning for tasks like AIME mathematics, optimization modeling presents a unique challenge. It inherently demands a global understanding of highly interconnected objective functions, constraints, and decision variables. Furthermore, it requires complex computational steps (e.g., executing the simplex method for linear programming, branch-and-bound for mixed-integer problems), which can be difficult to handle through purely textual, step-by-step reasoning.

Below are early results using purely textual step-by-step reasoning, without external solver assistance:

As these results demonstrate, relying solely on an LLM's sequential reasoning leads to extremely low performance across all benchmarks. The established practice within the optimization community is to utilize mature solvers precisely because they offer the global view and computational power necessary to resolve these intricate interdependencies.

By leveraging LLMs for accurate problem understanding and initial mathematical formulation, we can delegate the computationally intensive aspects of optimization to robust, specialized solvers through the generation of solver code blocks. This crucial division of labor is fundamental to achieving high performance in optimization tasks.

Building on these findings, one of the main challenges is to achieve both diverse reasoning exploration (which LLMs are good at) and strict accuracy and validity (which solvers demand). The selective KL-penalty with the tailed solver-based reward design, offers a robust and generalizable solution, makes it highly adaptable for extending LLM assistance across a variety of optimization solvers.

---

Weakness3: Error Bars of the Results

Response:

We sincerely apologize for the oversight in not including this crucial information in our initial submission. We have now conducted ten runs using top-p (Nucleus) sampling (temperature=0.5, top-p=0.95). Here are the mean and standard deviation results:

As the table shows, our results are stable and consistent with those presented in the initial submission.

---

Weakness4: Writing of the Problem Description

Response:

Thank you for this very constructive feedback. We fully agree that a concrete, motivating example would significantly improve the manuscript's readability and accessibility, especially for readers without a background in Operations Research. Your suggestions are extremely helpful for strengthening the introduction.

In our revision, we will restructure the introduction to better frame the core challenge. We will incorporate a classic motivating example—such as a logistics or resource allocation problem—to illustrate the practical journey from a real-world business scenario to a formal mathematical model. This will help ground the generic phrase "optimization across diverse domains" by providing a tangible application.

Furthermore, as you astutely pointed out, the description of the modeling process at the beginning of Section 3.2 is better suited for an earlier section. We will move this high-level overview—which describes the stages of problem analysis, mathematical formulation, and code implementation—into the introduction. This will provide readers with a clear, upfront understanding of the complex task our work aims to automate.

We believe these changes will make the problem statement clearer from the outset and the motivation for our SIRL framework more compelling to a broader audience. Thank you again for your valuable guidance.

---

Weakness5: Experimental Details Clarification

Response:

We acknowledge that the role and implementation of the "10 LLM roles" were not sufficiently detailed in the main text, and we appreciate the opportunity to clarify.

The primary rationale for using 10 distinct LLM roles is to enhance the diversity of the generated outputs. By prompting the model with different expert personas (e.g., an "operations research scientist" versus a "Python engineer"), we encourage it to explore varied reasoning paths and formulation strategies for the same problem. This diversity is crucial for the robustness of any self-consistency method; it mitigates the risk of all generation attempts converging on the same plausible but incorrect solution due to a shared systemic bias in the model. A more diverse pool of candidate solutions increases the probability that the correct formulation is present and can be identified by our consensus mechanism.

Due to page limitations, we placed the specific prompts that define each of these roles in the Appendix. We recognize our oversight in not explicitly linking to this from the main text. In our revised manuscript, we will amend Section 3.1 ("Instance-enhanced self-consistency") to briefly explain the purpose of these roles and add a direct reference to Appendix C.1 for the detailed prompts.

---

Weakness 1: Open Source Commitment

Response:

We appreciate the reviewer’s question regarding our open-source commitment. We confirm our plan to open-source the full training codes and a significant portion of training data.

• To demonstrate this commitment, we have already released two checkpoints (SIRL-Qwen2.5-7B-Gurobi and SIRL-Qwen2.5-7B-COPT) for immediate community use, and we plan to release more.
• In addition, we are actively preparing to release the complete training codebase and a significant portion of our training dataset. Our goal is to empower the community to build their own frameworks for diverse optimization problems or different optimization tools based on our work.

---

Weakness 2: Contribution of the Training Data

Response:

Thank you for this insightful question. We acknowledge that our data synthesis pipeline builds upon foundational works like ORLM. Our primary motivation stems from the distinct data requirements of our Reinforcement Learning (RL) approach compared to traditional Supervised Fine-Tuning (SFT). While SFT relies on (question, CoTs, code) style solution trajectories, our RL framework requires high-quality (question, answer) pairs to generate a reliable reward signal.

Therefore, our contribution is not a completely new synthesis framework, but a crucial enhancement focused on maximizing the correctness of the final answer. We achieve this with new emerging techniques for the LLM community(LLMs as a judge, reflection) and also our novel instance-enhanced self-consistency ("inst_sc") method. The instance-enhanced self-consistency goes beyond traditional majority voting on the final value by leveraging the solver's LP file to enforce consensus on the underlying mathematical structure (e.g., variable types). As shown in our experiments, this provides more robust verification, ensuring the high-quality data essential for our RL training paradigm.

---

Weakness 3: Novelty of RL Training Novelty may be a little limited in terms of RL training: RL has been widely used in post-training LLMs for many domains such as math / competitive programming. So, one can argue that the contribution of RL training proposed in this work is limited.

Response:

We appreciate the reviewer’s comment regarding the novelty of RL training within our framework, especially given its broader application in LLM post-training. We acknowledge that RL has indeed been widely adopted for enhancing LLMs in various domains, such as mathematics and competitive programming.

However, optimization modeling presents unique challenges. While large reasoning models (e.g., DeepSeek-R1), are great at step-by-step thinking for tasks like AIME math problems, optimization modeling requires a global understanding of highly interconnected objective functions, constraints, and decision variables, followed by computationally intensive steps (e.g., simplex method for linear programming, branch-and-bound for mixed integer programming). These complex computational burdens are inherently difficult for purely textual, step-by-step reasoning.

Below are results for models trained via step-by-step reasoning, without external solver assistance:

As these results demonstrate, relying solely on an LLM's sequential reasoning leads to extremely low performance across most optimization benchmarks.

The established practice within the optimization community utilizes mature solvers because they provide the global view and computational rigor necessary to resolve these intricate interdependencies. Our novelty stems from effectively integrating RL to teach LLMs to accurately understand problems and formulate them into solver-executable code, thereby delegating the heavy computational lifting to these specialized tools.

---

Weakness 4: Baseline Comparisons

1. Baseline comparisons lack fairness; LLMOpt (self-correction) and OptMath (14B/32B) results omitted.

2. Compare against top reasoning models (e.g., GPT-o3, Deepseek-r1).

3. Include ablation results for RL training, supervised fine-tuning, and their combination.

Response:

A-1: Baseline Evaluation Protocol
We appreciate the insightful question about our baseline comparisons, specifically regarding LLMOpt and the role of self-correction. We acknowledge that self-correction is a powerful strategy that can boost the performance of LLMs in various domains. However, our primary goal was to provide a rigorous and consistent comparison of the core generation capabilities of different models and training methodologies.

Therefore, our main results are based on the OptMath paper's rigorous and well-defined evaluation protocol, which uses Pass@1 accuracy, where a solution is considered valid only if the relative error is less than 1e-6. We believe this strict, single-pass evaluation is crucial for the optimization community as it provides a clear comparison of how well a model can generate a correct and solvable optimization formulation on its first attempt, without iterative refinement or external strategic guidance during inference.

A-2: Comparison with Stronger Baselines and SOTA Reasoning Models
We are grateful to the reviewer for the excellent suggestion to include comparisons with stronger OptMath baselines and leading reasoning models like DeepSeek-R1 and OpenAI-O3. We also include results from our 32B SIRL-model, ensuring equivalent parameter size across the evaluated models . This is a valuable addition that clarifies the landscape. We have now run these experiments and present the updated Pass@1 comparison in the table below:

It is interesting to note that while large reasoning models (LRMs) exhibit surprisingly strong performance on some of the harder, more complex benchmarks such as IndustryOR and OptMATH-193, they generally perform poorly on simpler questions.

Our initial analysis shows that for "simpler" optimization questions (where the problem structure might be straightforward but requires precise, direct solver invocation), these reasoning models tend to "overthink." They tend to generate unnecessary verbose reasoning steps or fail to precisely generate the correct solver code, as they are not explicitly optimized for concise, error-free tool-use.

A-3: RL vs. SFT and SFT+RL
We appreciate the reviewer’s crucial question regarding the comparison of our RL model with supervised fine-tuning (SFT) baselines. To thoroughly address this, we have conducted the requested experiments and present the Pass@1 comparison in the table below, clearly demonstrating the benefit of our online RL training:

As shown, our method consistently outperforms SFT across all benchmarks. Notably, the performance gain is most significant on more challenging datasets like MAMO Complex (+24.1%) and IndustryOR (+9.0%), demonstrating the necessity of RL for solving complex problems.

---

Limitation

The conclusion section lacks a thorough discussion of the work's limitations. The authors should more openly and explicitly address the limitations of their approach.

Response:

Thank you for pointing out the insufficient discussion of limitations. In our understanding, the limitations an further work can be broadly categorized as:

*Robustness of Reward Mechanism with Solver Feedback:

While our framework benefits from utilizing rich feedback signals directly from classical solvers (e.g., LP files and optimal values), extracting more signals and precisely translating this information into a perfectly robust reward mechanism for LLMs remains a significant challenge, there is still scope for enhancing performance and mitigating 'reward hacking.

Depth of Error Analysis: Our current evaluation primarily relies on quantitative metrics, and our error analysis is largely qualitative, focusing on observed failure cases. We also recognize that even the LRMs (e.g., Deepseek-R1) continue to face significant challenges in achieving high scores on difficult benchmarks such as IndustryOR and OptMATH—an open problem that warrants further investigation and the effort of all optimization community. In future work, we plan to conduct a more systematic and comprehensive analysis of error patterns, and to implement targeted model improvements aimed at addressing the specific weaknesses identified.

Thanks for the rebuttal. I have a follow-up question regarding the "results for models trained via step-by-step reasoning, without external solver assistance". In this case, how is the reward constructed without solving the problem? I do not think the reward signal used in the paper (i.e. comparing the objective obtained by the solver and the ground truth objective) is particularly novel, as many coding / competitive programming RL methods invoke an external compiler / solver to construct the reward, so I'd like the authors to further clarify the novelty in this regard.

Another comment / question is, in my experience, many of the existing test benchmarks are wrong. For example, it seems like the "ground-truth" answer to all traveling salesman problem in MAMO complex are wrong, and TSP actually consists of a significant portion in MAMO complex. In this case, a boost in the test accuracy in MAMO complex may not actually reflect the stronger model behavior, but rather making the same mistake as the wrong test benchmark, and hence, the performance improvement in the benchmark numbers may not be trustworthy. I would like to confirm with the authors if they have thoroughly cleaned up the test benchmark and made sure that the test results are correct. I believe this is very crucial in assessing the actual performance improvement of the method.

Question 1: Novelty about reward design

Response:

Thank you for your follow-up question. We appreciate the opportunity to clarify the novelty of our method. We fully agree that using external tools like compilers to construct rewards is now a common practice in modern RL for competitive math and programming problems, as we cited in our related work section.

The primary novelty of our work lies in how we address the specific challenges of optimization modeling by leveraging the unique affordances of modern solvers.

• Leveraging the Unique Richness of Solver Output:

A well-known challenge in applying RL to complex reasoning is the sparsity of outcome-based rewards. Methods relying on a simple outcome-based reward—such as comparing a solver's final objective to a ground truth—suffer from this well-documented problem. This is a fundamental challenge in domains like mathematics and coding, where a correct final answer provides limited insight into the quality of the reasoning process. As our experiments show, relying exclusively on this sparse Stage-1 reward (format + objective function verification) is insufficient because it provides limited corrective signal for flawed intermediate steps.

Optimization solvers provide a wealth of structural information about the formulated model. We leverage the solver's .lp file output, which is a symbolic and implicit representation of the LLM's mathematical reasoning. Here is one example,

\ Model AdvertisingOptimization

\ LP format - for model Browse. Use MPS format to capture full model detail.

Maximize

60500 RadioAds + 50000 SocialMediaAds

Subject To

BudgetConstraint: 5000 RadioAds + 9150 SocialMediaAds <= 250000

MinRadioAds: RadioAds >= 15

MaxRadioAds: RadioAds <= 40

MinSocialMediaAds: SocialMediaAds >= 35

Bounds

Generals

RadioAds SocialMediaAds

End

This allows us to create a granular, process-aware reward that can diagnose flaws in the mathematical representation. It enables us to move beyond a simple final score and instead reward the structural integrity of the model itself, In this sense, our method synergistically combines the tractability of outcome-based signals with the granularity of process-based rewards, which our ablation studies confirm is necessary for high performance.

• Targeting Mathematical Modeling Techniques:

Meanwhile, our reward function is explicitly designed to reinforce crucial modeling principles. For instance, it actively rewards the correct application of linearization techniques (e.g., the Big M method) needed to transform logical constraints into a solvable linear program. This teaches the LLMs how to model, rather than just what the final answer is, addressing the core bottleneck for LLMs in this domain. This type of approach leads to significant improvements on hard benchmarks (e.g.,IndustryOR,OptMath)

---

Question 2: Error in testing benchmark

Response:

Thank you for raising this critical point. We completely agree that benchmark integrity is fundamental for assessing true performance improvements in optimization modeling with LLMs. Your observation regarding the flawed Traveling Salesman Problem (TSP) instances within the MAMO Complex dataset perfectly highlights a significant challenge for the entire research community. We want to clarify the we take concrete steps to mitigate this issue in our work and future work. Our evaluation datasets were primarily inherited from the OptMATH project, which provided a partially verified foundation by the OptMATH team. Building upon this, we conducted our own verification and correction process. For instance, we have performed a revision of the ground-truth solutions for the NL4Opt and IndustryOR benchmarks to improve their accuracy for our evaluation. As part of our commitment to advancing research in this area, we have open sourced our refined test sets to the community. We believe our work makes a contribution to this by identifying and correcting issues in several key benchmarks.

We fully acknowledge that a comprehensive and definitive cleanup of all public benchmarks is a substantial undertaking that requires sustained community-wide effort. Regarding the specific issue you raised concerning the TSP problems in the MAMO Complex dataset, we have identified this as a critical area for future work. We are committed to continuing our efforts to verify and correct these instances and will continue to open-source our work to provide a more robust foundation for the community.

---

We sincerely appreciate your thorough and meticulous review, especially your proactive engagement with the benchmark integrity.

Given your deep experience with these benchmarks, we are confident you have valuable insights. If your team has already performed additional work to identify and clean other flawed instances, we would be incredibly grateful if you could share the link to this cleaned dataset. This would allow us to report our results on the corrected benchmarks as soon as possible, ensuring our work meets the highest standards and contributing a validated dataset back to the community.

Weakness 1: Compute Efficiency Reporting

Response:

• GPU Running Time:
  Our model was trained from the Qwen2.5-7B-Instruct base, following a two-stage process using the same training dataset throughout.

  • Stage 1: Training is guided by a specific 'stage-1 reward' signal.
  • Stage 2: Training resumes from the checkpoint at the end of Stage 1.
  • Total Compute: Each stage required approximately 24 × 8 H100 GPU hours, totally 48×8 H100 GPU hours.

• Solver Runtimes:
  For each problem, we set a 100-second timeout for the solver during both training and inference. This duration was found sufficient for the textbook-level problems in our benchmark. Preliminary experiments with longer timeouts (200–300 seconds) showed diminishing returns, supporting our choice of 100 seconds.

• Comparison: Base, SFT, and RL Approaches
  To highlight the advantage of our online RL approach over a strong supervised fine-tuning (SFT) baseline, we present the following pass@1 results:

  Models	NL4OPT	MAMO Easy	MAMO Complex	IndustryOR	OptMATH-193
  BaseModel	75.1%	81.3%	22.7%	13.0%	4.1%
  SFT	83.3%	86.4%	38.0%	24.0%	20.8%
  RL	96.3%	90.0%	62.1%	33.0%	29.0%

  As shown, our method consistently outperforms SFT across all benchmarks. The performance gain is especially pronounced on more challenging datasets, underscoring the necessity of RL for complex problem solving.

---

Weakness 2: Clarity of Curated Seed Data for Training Data Synthesis

Response:

Thank you for highlighting the importance of seed data clarity. The quality and composition of the seed data are indeed crucial for our data synthesis pipeline and final model performance.

To clarify, the seed data used in our work is identical to the dataset established and utilized in the foundational ORLM work. For brevity and due to space constraints, we did not repeat the detailed curation process in our manuscript, as it is thoroughly documented in the original ORLM paper. Based on your feedback, we have revised Section 3.1 ("Data synthesis framework") to explicitly state that our process builds upon the ORLM seed data, and we have added a clear citation to the original paper for readers seeking a comprehensive description. We believe this revision improves the clarity and self-containedness of our methodology.

---

Q1: What is the curated seed data used for training data synthesis? Is it exactly the same as the seed data used in ORLM and OR-Instruct (e.g., 686 cases in their study)? If yes, is there a risk of bias or overfitting when evaluating on the IndustryOR benchmark?

Response:

As stated in the appendix, the seed data is identical to that used in ORLM (686 cases). While there is a potential risk of overfitting on IndustryOR, our model demonstrates robust generalization across other test datasets. Furthermore, the data synthesis process significantly alters the data distribution. This is evidenced by the fact that SFT alone could not replicate our results, highlighting the effectiveness of our approach.

---

Q2: Is reliance on external optimization solvers a limitation, given that verifiable reward mechanisms are only applicable to a narrow set of problem domains? How might this approach extend to tasks without access to classical solvers, or are there alternative strategies to overcome this constraint?

Response:

We appreciate this insightful question regarding the role of external solvers. Rather than a limitation, we view this as a core design principle.

The results of training via textual step-by-step reasoning without external solver assistance are shown below:

Here, Math4Opt-RL is trained via the RLVR approach without using solvers as verifiers. As these results demonstrate, relying solely on an LLM's sequential reasoning leads to extremely low performance across most optimization benchmarks.

While large reasoning models (e.g., DeepSeek-R1) excel at sequential reasoning for tasks like AIME mathematics, optimization modeling presents unique challenges. It requires a global understanding of highly interconnected objective functions, constraints, and decision variables, as well as the execution of complex computational steps (e.g., the simplex method for linear programming, branch-and-bound for mixed integer programming). This computational burden is difficult to handle through purely textual, step-by-step reasoning.

The optimization community relies on mature solvers because they provide the global perspective and computational power needed to resolve these intricate interdependencies. By leveraging LLMs for accurate problem understanding and initial mathematical formulation, and delegating the computationally intensive aspects to specialized solvers via generated code blocks, we achieve high performance in optimization tasks. This division of labor is fundamental to our approach.

Weakness 1 and Question 1: Experimental Setup and Reporting of Standard Deviations

Response:
Thank you for your insightful question regarding the experimental setup and reporting of standard deviations. We apologize for omitting this crucial information in our initial submission.

We have now conducted additional experiments and report the following performance metrics using top-p (nucleus) sampling (temperature = 0.5, top-p = 0.95):

As shown, the results are highly stable and consistent with our initial findings.

---

Weakness 2 and Question-3 : Reward Design With/Without Solver Answers

Response:
We agree that using actual optimization solvers for the final step is highly effective for interpretability and accuracy. We have investigated the scenario where solvers are used exclusively during data generation, and the LLM is required to perform all reasoning and solution steps at inference—acting as the "solver." To assess this, we trained models to reason step-by-step through text, without solver assistance at inference.

Optimization modeling differs fundamentally from tasks like AIME mathematics, as it requires a global understanding of interconnected objectives, constraints, and variables, as well as complex computational steps (e.g., simplex method for linear programming, branch-and-bound for mixed-integer problems). These are challenging to resolve through purely textual, step-by-step reasoning.

To illustrate, we present early results from models trained solely via textual reasoning, without solver assistance:

(Math4Opt-RL is trained via RLVR without solver verification.)

These results demonstrate that relying solely on LLM sequential reasoning leads to very low performance. The optimization community uses mature solvers because they provide the global view and computational power needed for these tasks. Moreover, deriving precise decision variable values for optimal solutions remains extremely difficult for LLMs using only step-by-step textual reasoning. In summary, although it is possible to rely on the LLM as the "solver" at inference, this approach currently results in substantially lower accuracy and reliability.

---

Weakness 3: Insufficient Checklist Completion

Response:
Thank you for highlighting the need for more detailed explanations and explicit references in our checklist. We acknowledge that our initial submission lacked clarity in this regard. In the revised version, we confirm that we will:

• Carefully review each checklist item.
• Provide direct references to the relevant sections or appendices for every point.
• Ensure that all previously unexplained parts (e.g., the Claims section) are now explicitly addressed.

---

Weakness 4: Minor Corrections and Clarifications

Response:
We appreciate your detailed suggestions, which have improved the clarity and accuracy of our manuscript. The following changes will be made:

• Definition of LP:
  We now define Linear Programming (LP) as:
  “Linear Programming (LP) is a mathematical method for optimizing a linear objective function subject to linear equality and inequality constraints.”
  We also clarify that the .lp file format, used in data synthesis, is a standard format for solvers to write and process models.

• Line 33:
  Corrected “information, generate” to “information, and generate”.

• Line 103:
  Added the conjunction “and” in “rstar [43],Phi [44]” → “rstar [43], and Phi [44]”.

• Line 117:
  Inserted a missing period before “For example”.

• Line 340:
  Corrected the typo “simpltasks” to “simple tasks”.

---

Question 2: why using only stage-2 reward boosts performance on OptMATH but diminishes performance on NL4OPT

Response:
The stage-2 reward, with its $R_{bonus}$, encourage advanced optimization techniques, including reformulation methods that are native to and leveraged by mature solvers for efficient processing of complex SDP, MILP, and SOCP problems, which is critical for solving challenging complex problems in hard benchmark dataset such as OptMATH.

When the model is primarily incentivized for these advanced optimization techniques, it may tend to over-explore or involve redundant exploratory steps, leading to diminished performance for tasks where a simpler approach would have been more accurate and efficient.

Weakness-1: Unclear Reward for Advanced Modeling

Response:

Thank you for highlighting this point. We fully agree that, in general, simpler models are preferable.
Our advanced modeling bonus, R_bonus, is not meant to encourage unnecessary complexity. Rather, it is designed to reward the use of essential reformulation techniques from Operations Research—methods that are often required to make a problem tractable for high-performance MILP solvers.

For instance, problems involving logical "if-then" conditions or fixed charges often lead to non-linear or logically invalid initial formulations. While LLMs may generate simple, approximate solutions for tasks like the 0-1 knapsack or certain TSP instances, consistently obtaining optimal solutions requires these advanced methods. A typical example is the Big-M formulation, which linearizes such conditions by introducing binary variables, making the problem solvable as a MILP. Without these reformulations, the resulting model would be either incorrect or unsolvable by standard solvers.

Therefore, the bonus is awarded only when these techniques are indispensable for producing a correct and solvable model. Our model automates this critical (and often non-trivial) reformulation step, providing human modelers with a mathematically rigorous model that is ready for high-performance solvers.

---

Weakness-2: Limitations of using "solvers" as verifiers.

Response:

Thank you for your question about the limitations and computational cost of using solvers as verifiers. This is indeed a crucial aspect.

To illustrate the challenges throughout the process, we present the error types for the Qwen-2.5-7B-Instruct model and SIRL-Qwen2.5-7B, with a maximum execution time of 100 seconds.

Error Types (Qwen-2.5-7B-Instruct)

Error Types (SIRL-Qwen2.5-7B)

As shown above, the proportion of "Timeout" errors—where the solver becomes stuck—is very low, occurring only for the most challenging OptMATH-193 dataset (0.5%) in the Instruct model. This indicates that, for these textbook-level optimization problems, solver execution is generally not a computational bottleneck. The main error types are "Execution Error" (where the LLM generates unexecutable code) and "Wrong Answer" (where the model is logically incorrect).

While solver execution is not currently a primary bottleneck for our tested problems, we acknowledge that for truly large-scale, real-world optimization challenges, the computational cost of solver verification could indeed become a limiting factor.

---

Weakness-3 The cost of training and value position.

Response:

First question:
Our training process consists of two distinct stages, both using the same training dataset.

• In Stage 1, we use a specific "stage-1 reward" signal to guide training.
• In Stage 2, training resumes from the checkpoint obtained at the end of Stage 1.
  Each stage required approximately 24 × 8 H100 GPU hours, totally 48×8 H100 GPU hours.

Second question:

• We have open-sourced and released checkpoints that achieve state-of-the-art performance among models with 7B–14B parameters,. Any member of the community can download these checkpoints to build their own LLM model assistants. We also plan to release more checkpoints.
• Additionally, we introduce a new RLVR framework for optimization modeling and will continue to release corresponding codebase and a portion of training dataset , empowering the community to build their own LLM assistants for diverse optimization problems.

---

Weakness-4: The paper confuses Linear Programming (LP), the LP file format, and Mixed-Integer Programming (MIP), needing clarification on whether the method is LP-specific or also applies to MIPs.

Response:

Thank you for your insightful feedback, which has greatly improved the clarity of our manuscript. We apologize for any confusion caused by our earlier writing and appreciate the opportunity to clarify.

Our method is not limited to Linear Programming (LP); it supports a broad range of optimization problems, including Quadratic Programming (QP), Mixed-Integer Programming (MIP), and Second-Order Cone Programming (SOCP). During data synthesis and the reinforcement learning stage, we utilize the .lp file format, which is a standard for solvers to represent various optimization models. This format is not exclusive to LP but is widely compatible with models such as QP and MIP. Our strong performance on the OptMATH test set, which includes MIP and SOCP instances, further demonstrates that our method generalizes effectively beyond LP.

In the revised manuscript, we will clarify that the .lp file format supports multiple optimization models, not just LP, and we will revise relevant sections to emphasize the method’s applicability across diverse problem types.

We sincerely appreciate your thorough review and welcome further suggestions to improve our work.

---

Weakness-5 Missing References: The paper omits key references like Ner4Opt, Enhancing Decision Making (AAAI 2025), Holy Grail 2.0, Text2Zinc, and Constraint Modeling with LLMs, which are relevant for comparison, benchmarking, or complementary approaches.

Response:

Thank you for these highly valuable references. We confirm that we will revise our Related Work section to address all of your suggestions.

---

Reply to Questions

Q-1: For the comparisons, how did you obtain their results (copied from their papers, re-implemented their methods, re-ran them, or otherwise)? This refers to the agent-based and offline learning methods in Table 2.

Response:

The results in Table 2 are from the paper "OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling." We used their figures directly because of their rigorous evaluation protocol, which validates solutions only if the relative error is less than 1e-6. Further details on their evaluation methodology and results are publicly available on the paper's GitHub repository.

---

Q-2: Why are there no results for Chain-of-Experts (CoE) on these benchmarks?

Response:

Thank you for pointing this out. The absence of results for Chain-of-Experts (CoE) on these benchmarks is due to a fundamental mismatch in the required input data format, which poses significant challenges for direct and fair comparison.

Our evaluation benchmarks, consistent with most recent work in this area (e.g., ORLM, OptMATH), consist of problems presented as natural language descriptions paired with a ground-truth optimal value.

In contrast, the CoE framework, as presented in its original paper, operates on pre-processed, structured data inputs rather than raw natural language. The proprietary data-processing pipeline used to convert natural language problems into this required structured format is difficult to adapt to other benchmark datasets. Reimplementing this pipeline for each of our diverse benchmarks would require significant and non-trivial effort, with no guarantee of faithfully replicating their methodology. Furthermore, to the best of our knowledge, we are not aware of any subsequent work that has benchmarked the CoE framework on these standard datasets.

---

Q-4: How does your method handle ambiguous or incomplete inputs?

Response:

Our current research primarily focuses on well-defined, textbook-level optimization problems, which allowed us to rigorously validate the core methodology of training LLMs for optimization modeling.

For ambiguous or incomplete inputs, one promising direction is to develop interactive clarification modules, where the LLM can query the user for necessary missing details, potentially through multi-turn conversational frameworks. An even more advanced approach could involve a multi-agent system framework. In this setup, an "analyzer agent" would first attempt to autonomously infer and supplement the content of an incomplete natural language input before passing it to the core modeling agent. To effectively judge and improve the quality of its inferred content, this analyzer agent would require a robust reward system, potentially combining our RLVR methodology with Reinforcement Learning from Human Feedback (RLHF).

This will require concerted efforts from the entire research community.