Thank you for your valuable comments. We sincerely hope that we could properly address your concerns.

W1: Focuses on exact objective matches. Deeper insight with measuring near-optimality, solver tolerance, or robustness.

We provide additional results on modeling accuracy beyond just the objective value. However, many benchmarks lack labeled models in their datasets. Therefore, we conducted experiments on a subset of the OptMATH training set (100 problems) that includes annotated models. Since our method and the baselines were not trained on this dataset, the comparison is both safe and fair.

• Optimality and Tolerance. We present the solver execution time and code pass rate in Table 1. The code pass rate indicates the ratio of code without bugs. In the existing optimization modeling datasets, we solve the optimization models to optimal solutions within seconds, so we do not set solver tolerances for these experiments.
• Robustness. We ask the LLM to randomly rewrite the problem and test performance on these rewritten versions. The rewriting process alters 10% of the sentences by changing their order and rephrasing another 10% while maintaining their meaning. Results in Table 2 demonstrate the model's robustness to perturbations in the problem.
• For deeper insights into the optimization modeling, we provide more metrics on the correctness of the optimization models. To further address your concerns, we find the datasets with annotated models and provide the proportion that LLM-generated models match with the ground-truth models in some statistical information, such as the number of variables, binary variables, integer variables, and constraints of the optimization models. As shown in Table 1, OptiTree achieves the highest matching ratio with the ground-truth model, underscoring its reliability.

Table 1: More metrics.

Table 2: Rubustness of OptiTree.

Q1: Results when OptiTree is applied on top of reasoning LLMs

We conducted experiments to build OptiTree on various LLMs. The results are shown in Table 3, demonstrating that OptiTree generalizes well across different LLMs and significantly improves their performance.

Table 3: Results on more LLMs.

W3&Q2: Setup for efficiency comparison is unclear.

The efficiency comparison setup involves measuring the average time, in seconds, for each method to solve a single problem from the various benchmark datasets.

• The result in Table 2 of the main text represents the inference time required to generate the final optimization model for each new problem. We (1) use tree search to find an appropriate problem and (2) perform optimization modeling, combining the modeling thoughts from the problem and we solve the model. The total time includes both the tree search and modeling phases.
• The total runtime for each prompt-based baseline (like CoE, OptiMUS, and MCTS) and for the OptiTree pipeline was recorded for every problem attempted. These times were then averaged across each dataset.
• We know you may wonder about the solver execution time for the solver code of each problem. This time is included in the modeling time. As shown in Table 1, the solver time is under a second, which can be considered negligible.

Q2: Reason for OptiTree to speed up. Time complexity analysis. How does OptiTree scale for complex problems?

• The high efficiency of OptiTree.The speed of OptiTree, especially compared to other prompt-based methods, can be attributed to two main factors.

  • OptiTree explores a much smaller search space. Existing methods involve iterative searches through variables, constraints, and objective functions, which can exponentially enlarge the search space as problem complexity increases. In contrast, OptiTree searches predefined candidate subproblems, requiring only the identification of the most suitable subproblem within a finite set, leading to greater efficiency.
  • OptiTree has fewer iterations in the workflow. Existing methods often involve repeated calls to certain agents in a multi-agent system for optimization modeling. For example, for complex problems, existing methods may call the terminology interpretation repeatedly. These methods use a management agent to control the workflow automatically. However, the agent's hallucination can lead to suboptimal decision chains. In contrast, OptiTree avoids this multi-agent system, resulting in a more streamlined and efficient workflow.

• We try to estimate the time complexity of OptiTree. Given a problem $\mathcal{P}$, we analyze its two main components: Tree Search and Modeling.

  • The complexity of tree search is influenced by the depth of the search path $H$ and the number of children at each node for comparison $NumChild_h$ in each step $h$.

  • The modeling time is influenced by the complexity of the original problem description $Com(\mathcal{P})$ and the length of the modeling prompt. The promplt length can be roughly esitimated by $l(\mathcal{P})+l(\mathcal{T}(\mathcal{P}))+C$, depending on the length of problem description $l(\mathcal{P})$ and the modeling thoughts $l(\mathcal{T}(\mathcal{P}))$. The notation $C$ is the fixed length of the prompt template.

  • We suppose further that the time for LLM generation is $\alpha\cdot Com\cdot l$, where $\alpha$ is a constant, $Com$ is the reasoning complexity ratio measuring the reasoning complexity of the task, and $l$ is the prompt length. For subproblem identification, the reasoning complexity is low and set to 1 for simplicity. Suppose further that $l(child)$ is the text length for each child in the tree. Thus, the total time estimation is given by

    $\alpha\sum_{h=1}^H NumChild_h\cdot l(child)+\alpha\cdot Com(\mathcal{P})\cdot(l(\mathcal{P})+l(\mathcal{T}(\mathcal{P}))+C)$

• The scalability of OptiTree. OptiTree is designed to scale for problems with longer question lengths or higher complexities. For longer question lengths, we avoid implementing a complex multi-agent workflow that calls agents repeatedly for specific tasks. Instead, we search within a finite subproblem space, significantly reducing the number of requests to the LLM. As shown in Table 2 of the main text, the search process is efficient.

W2&Q3: The link between diagnosed variable-definition errors and the chosen decomposition remedy. Quantify how much OptiTree lowers this specific error.

We explain the link between them.

• Variable Definition Challenge. Following the paper of OptiMUS, the errors mainly come from three aspects: missing or wrong constraints, incorrect modeling and coding errors. The incorrect models mostly come from the wrong variable definition, e.g., defining the wrong binary variables in the model. Almost half of the errors of OptiMUS come from the incorrect modeling (62.5% for ComplexOR). Our initial analysis in Motivation 1 pinpoints that this is also the main error for CoT. For the complex OR problems, directly defining variables is a challenging task for LLMs. This observation motivated us to enhance the variable definition process using step-wise guidance.

• Our approach is addressing the incorrect modeling from the perspective of variable definition. We do so in the three aspects.

  • Step-by-Step Decomposition. Instead of defining complex variables for the entire problem at once, OptiTree decomposes the task into modeling a series of simpler subproblems. Modeling a subproblem and correctly defining its variables are less challenging tasks for an LLM. Our experiment in Motivation 3 confirms this, showing a significant accuracy improvement when providing the LLM with a ground-truth model of a subproblem first.
  • Enhancement with Modeling Thoughts. The process is further enhanced by retrieving modeling thoughts for the identified subproblem. As shown in the schema examples, these thoughts provide explicit, step-by-step guidance, starting with how to define variables. This provides a strong, reliable template that the LLM can adapt.

• We present the percentage of all kinds of mistakes for methods in Table 4. OptiTree reduces the errors of variable definitions.

Table 4: Failure cases information.

Thank you for your valuable comments. We sincerely hope that we could properly address your concerns.

W1: The pseudo-code and prompts

• We first explain the pseudocode.

  • Global thoughts synthesis process. We apologize for not including this prompt in the Appendix. We generate modeling thoughts by integrating information from the maximal subproblem’s modeling thought and the current problem description.

    This is an OR problem with the following problem description

    {problem description}

    The problem contains a subproblem of {subproblem name}. The modeling thoughts to model this subproblem are as follows. Please refer to the thoughts and list the modeling thoughts for the original OR problems.

    {subproblem modeling thoughts}

  • Modeling fails means that the final objective value of the generated optimization model does not equal the ground-truth value for that problem. In the tree update, if the generated answer for the current problem is incorrect, the system determines that the current modeling tree is insufficient for that problem type and updates the current problem into the tree.

  • The distillation prompts are listed in Appendix H.4. The first block of prompts is used for statement thoughts distillation, and the second block of prompts is used for modeling thoughts and schema distillation.

  • Yes. The maximal subproblem is the same as Algorithm 1.

• Two prompts for modeling thoughts distillation. We apologize for mislabeling. These blocks of prompts are used for statement thought distillation.

  • The first prompt is used for situations when we have to find subproblems in the previous search. It asks the LLM to take the specific problem and its current basic subproblem type as input and then determine a more precise subproblem.
  • The second prompt is used for situations when the search process stops at the root node. Thus, we cannot refer to any prior subproblem for thought construction.

• Code correction. We will revise the manuscript to include it for clarity.

  • The code correction module is a standard component of the prompt-based optimization modeling methods, such as CoE and OptiMUS. It is applied after the solver codes generated by the LLM raise errors. The prompt instructs the LLM to analyze the bugs in the code. We use one iteration of this correction step.
  • The code correction is not a core contribution and a trick in our work. We find in practice that the trigger of the code correction module is less than 3%. This implies that the successful execution rate for the initial codes is high, and the benefits of the correction prompt are not significant.

• The Subproblem Identification Prompt is used during the tree search to find the single best subproblem to proceed with at each level. It returns the single best matching subproblem and a JSON object containing a similarity score.

  Add New Tree Nodes Prompt is used during the tree update phase to correctly position a new node within the tree's hierarchy. The LLM uses the prompt and identify all parent-child relationships between a new problem and its potential siblings. It returns a list of all problem types that are identified as subtypes of the primary one; it does not output a similarity score.

• Yes, the "constraints" information is used, and the entire JSON object is used for the subproblem matching.

W2: When their tree modeling fails.

Following the paper of OptiMUS, the errors mainly come from three aspects: missing or wrong constraints, incorrect modeling and coding errors. Almost half of the errors of OptiMUS come from the incorrect modeling (62.5% for ComplexOR), e.g., defining the wrong binary variables in the model. The results for OptiTree are presented in Table 1. OptiTree reduces the errors of incorrect models and the constraint errors are the main error for OptiTree.

• First, accurately formulating constraints requires a deep and nuanced understanding of the specific problem scenario and its domain knowledge. For highly specialized or novel OR problems, existing background knowledge may not fully capture the intricate details necessary for precise constraint definitions.
• The second limitation is the intrinsic reasoning ability of the base LLM. This framework may struggle with problems requiring significant logical or relational reasoning. For instance, in the Staff Rostering problem, the model must effectively allocate staff for 24-hour shifts to meet coverage constraints (the workers work for 8 hours a day). If it fails to recognize that the demand period from 1:00 to 2:00 AM is covered by workers who began their shifts at 8:00 PM, it will result in an incorrect constraint and ultimately a flawed model.

Table 1: Failure cases information.

W3: Performance on recent benchmarks

We include the fine-tuned baselines, Evo-Step and OptMATH. We also include two new benchmarks, the OptiBench and OptMATH datasets for testing. The fine-tuned models for Evo-Step and OptMATH have not yet been released, so results are drawn from the original papers. The results demonstrate the superior performance of OptiTree over the baselines across benchmarks.

Table 2: The comparisons with more datasets and baselines.

Q1: Improvement of (1) stronger models like reasoning models, (2) weaker models like Qwen and llama.

We conduct experiments to build OptiTree on (2) stronger models, including GPT-o1 and DeepSeek-R1; (2) weaker models, including Qwen-2.5 7B and Llama3-8B. The results in Table 3 demonstrate that OptiTree can significantly improve the performance of the LLMs.

Table 3: Improvement on more models.

Q2: Sensitivity to different datasets used to build the tree and different seeds.

• First, we show that the pipeline is not sensitive to the different datasets used to build the tree. We conduct the following experiments. (1) We randomly select 400 different problems in the OR-Instruct dataset for tree construction. (2) We randomly select 100 problems for tree construction. (3) We select problems from the NL4Opt and MAMO ComplexLP datasets for tree construction. We found that OptiTree maintains stable performance across different datasets.

• Second, we run the code with different seeds for tree construction and testing. We find that OptiTree achieves a stable performance across the random seeds.

Table 4: Sensitivity analysis.

Q3: Line 221: Is this a typo?

$P$ is a subproblem of $P_k^{(M)}$, and thus $P$ is the parent as the deeper node represents a more complex problem, i.e., $P \subseteq P_k^{(M)}$.

Q4: Figure 2: Examples of the in IndustryOR

We provide a simplified example due to the space limit.

• Problem: A store plans to purchase and sell the goods for the first quarter of next year. The warehouse capacity is $M$ units, and the initial goods in stock are $M_0$ units. Determine the units to be purchased and sold each month to maximize total profit, with purchase and sales prices for the $i^{th}$ month denoted as $p_i$ and $s_i$.

• Analysis. OptiTree can reduce the errors in incorrect models.

  • Without OptiTree. Define $x_i$ as the units purchased at the beginning of each month and $y_i$ as the units sold each month.

    $max_{x,y}\sum_i s_iy_i-p_ix_i$

    $x_0-y_0+M_0\le M$

    $x_i-y_i\le M,i>0$, inventory constraints

  • OptiTree identifies this problem as an inventory problem, with $I_i$ representing the inventory at each month.

    $max_{x,y}\sum_i s_iy_i-p_ix_i$

    $I_0=M_0$

    $I_i=I_{i-1}+x_i-y_i$

    $I_{i-1}+x_i\le M$, inventory constraints

    $y_i\le I_{i-1}+x_i$, sale constraints

Q5: The code for the paper

Thank you for your interest in our work and your valuable suggestions. However, we are sorry that authors cannot include links to external pages (including anonymous repositories) during the rebuttal period. We will publicly release all code and final modeling trees shortly.

Dear Reviewer vacN,

Thank you very much for taking the time to read our rebuttal and for promptly acknowledging it. We truly appreciate your engagement with our work during the review process.

To further address your concerns, we provide the revised pseudo-code following your suggestions as follows.

---

Algorithm 1 Tree Search for Subproblem Decomposition

---

• Require: Target problem $\mathcal{P}$, modeling tree $\mathbb{T}$

• Ensure: Hierarchical modeling thoughts $\mathcal{T} (\mathcal{P})$, and the maximal subproblem $\tilde{\mathcal{P}}$

• function TreeSearch($\mathcal{P}$, $\mathbb{T}$ ):

  • Initialize current node $\mathcal{N}\leftarrow$ The root of the modeling tree $\mathbb{T}$

  • # prompts block 1 and 2 in Appendix H.4

  • Extract statement thoughts $\mathcal{C}_\mathcal{P}$

  • # Identified subproblems

  • while $\mathcal{N}$ has children do

    • Get the children of $\mathcal{N}$
    • # prompts in Appendix H.2
    • Identify the subproblem of $\mathcal{P}$ among the children with similarity scores
    • If $\mathcal{P}$ has a subproblem among the children then
      • $\mathcal{N}\leftarrow$ Subproblem with the max similarity score
      • The maximal subproblem $\tilde{\mathcal{P}}\leftarrow \mathcal{N}$
    • else
      • $\mathcal{N}\leftarrow$ None
    • end if

  • end while

• Synthesize $\mathcal{T}(\mathcal{P})$ from $\mathcal{T}(\tilde{\mathcal{P}})$

• return $\mathcal{T}(\mathcal{P})$, $\tilde{\mathcal{P}}$

• end function

---

---

Algorithm 2 Modeling Tree Update

---

• Require: Target problem $\mathcal{P}$, modeling tree $\mathbb{T}$

• Ensure: Updated Modeling Tree

• function TreeUpdate($\mathcal{P}$, $\mathbb{T}$ ):

  • $\mathcal{T}(\mathcal{P})$, $\tilde{\mathcal{P}}\leftarrow$ TreeSearch($\mathcal{P}$, $\mathbb{T}$)
  • # prompts in Appendix H.3
  • Model the problem $\mathcal{P}$ using the modeling thoughts and solve the model for optimal value $y$
  • If $y$ does not equal to the ground-truth objective value then
    • # prompts block 1 and 2 in Appendix H.4
    • Distill schema (problem type,$\mathcal{C}_\mathcal{P}$,$\mathcal{T} (\mathcal{P})$) from $\mathcal{P}$
    • Find the children of the maximal subproblem $\tilde{\mathcal{P}}$
    • for child $\tilde{\mathcal{P}}_k$ in the children do
      • # prompts block 3 in Appendix H.4
      • if $\mathcal{P}\subset_{\mathcal{S}}\tilde{\mathcal{P}}_k$ then
        • Insert $\mathcal{P}$ as parent of $\tilde{\mathcal{P}}_k$ and children of $\tilde{\mathcal{P}}$
      • else
        • Insert $\mathcal{P}$ as sibling of $\tilde{\mathcal{P}}_k$ and children of $\tilde{\mathcal{P}}$
      • end if
    • end for
  • end if

• return $\mathbb{T}$

• end function

---

If you have any further thoughts, questions, or suggestions---either regarding our current response or the broader direction of the work---we would be genuinely grateful to hear them. We highly value your perspective and would welcome any opportunity for continued discussion or clarification.

Best,

Authors

Thank you for your valuable comments. We sincerely hope that we could properly address your concerns.

W1: Explanation of constructing the modeling tree

The modeling tree construction is fully automated. To construct a modeling tree, we need a dataset (OR-Instruct) with various problems containing the problem description and the ground-truth optimization models.

• First, we explain the tree search process to find an appropriate subproblem.

  • OptiTree uses an LLM to distill the description of the target problem into a set of statement thoughts, which summarize the problem's key features and requirements into a schema. The statement thoughts are used for subproblem identification (Appendix H.1 for an example).

  • The search begins at the root node of the modeling tree. Optitree evaluates the children of the current node to find the best match for the target problem. The search halts when it reaches a node where none of its children qualify as a subproblem for the target problem. The last subproblem found in this process is called the maximum subproblem, and its modeling thoughts are used to guide the final solution.

    The simplified prompt for subproblem identification in tree search.

    Input problem: {input_problem_statement_thoughts}.

    You are given a list of defined subtypes and the statement thoughts of subproblems: {subproblems_statement_thoughts}.

    Your task is to determine if the specific problem belongs to one of the given subtypes. Return only the final answer in the following JSON format:

    ‘‘‘json{

    ​ matching_subtype: "",

    ​ reasons: "",

    ​ similarity: [0,1,2]

    }‘‘‘

    The simplified prompt for modeling using modeling thoughts.

    You are a mathematical formulator to formulate the given OR problem as an optimization model and generate Gurobi code to solve the model.

    {problem description}

    The problem has a subproblem of {subproblem name}. During the modeling process, we can refer to the subproblem model: {subproblem model}

• Second, we explain the evaluation process whether a problem should be updated into the tree. For each problem in a given dataset, we perform a tree search to find the maximum subproblem and use the modeling thoughts to enhance the modeling of the original problem. We solve the generated optimization model and obtain the final answer of the objective value. If the final answer matches the ground-truth answer, no update is necessary, indicating that the tree can already address this type of problem. Otherwise, the unsuccessful problem is used to expand the tree.

• Third, we explain the adding process of the tree nodes.

  • Distill a Schema: First, a new modeling schema (including problem type, statement thoughts, and modeling thoughts) is automatically distilled for the failed problem. The prompt is as follows.

    Your task is to summarize the type of this specific problem and provide a detailed statement thoughts of this type. You must classify the problem into more precise scenarios, such as the Traveling Salesman Problem (TSP), the facility location problem, and so on.

    Specific Problem: {problem description}

    Here is an output format example ...

  • Position Among Siblings: The new node is placed as a child of the maximum subproblem. The system then uses an LLM to verify the relationship between the new problem and the existing children of the maximum subproblem.

    • If P is a subproblem of an existing child, P is inserted between the parent and that child.
    • Otherwise, P is inserted as a new direct child, becoming a sibling to the others.

    You are given a primary problem type and its statement thoughts.

    {problem_statement_thoughts}

    You are also provided with a list of other problems with their statement thoughts. Your task is to determine which problem types in the list are subtypes of the given primary problem type.

    {list_of_problem_types_and_statement_thoughts}

W2: What is the greatest search depth?

We report the greatest search depth across different datasets. For the harder datasets, the search depth tends to increase, indicating that more complex problems feature more sophisticated subproblem structures.

Table 1: The greatest search depth.

W3: Performance evolution when thoughts are aggregated along the search path/ cost–benefit trade-off between prompt length and accuracy.

• We have conducted the related experiments in Section 5.4, Impact of the Search Depth. We compared the performance of OptiTree with an unrestricted search depth against variants with maximum depths of 1 or 3. The results in Table 3 demonstrate that performance improves with deeper searches.
• We analyze the prompt length for the modeling step, which includes the fixed prompt template, the problem length, and the length of the modeling thoughts. Table 2 shows that the prompt length does not significantly increase as the search goes deeper.
• Given that we only use the thoughts from the final node, the cost-benefit trade-off is not about a growing prompt length, but rather about search time vs. accuracy. The search time for different search depths is presented as follows. Table 3 shows that the cost is a small component of the overall efficient process.

Table 2: The prompt length for the problems (characters).

Table 3: The comparisons of trade-off between search depth (search time) and the performance.

W4: Comparisons with the latest methods, such as Evo-Step and OptMATH

We include Evo-Step and OptMATH in our baselines. The Evo-Stop and OptMATH are fine-tuned models. We also include two new benchmarks, the OptiBench and OptMATH datasets for testing. As the fine-tuned models for Evo-Step and OptMATH have not yet been released, results are drawn from the original papers. The results illustrate the superior performance of OptiTree compared to these baselines across benchmarks.

Table 4: The comparisons with more datasets and baselines.

Q1: Analyzing scenarios where OptiTree may underperform and discussing the potential reasons.

• Failure cases analysis. Following [4], the errors mainly come from three aspects: missing or wrong constraints, incorrect modeling and coding errors. In [4], almost half of the errors come from the incorrect modeling (62.5% for ComplexOR), e.g., defining wrong binary variables in the model. We analyze the failures of OptiTree and normalize the failure rates to sum to 1. We find that most errors are from the missing constraints. OptiTree can significantly reduce the error of incorrect modeling, which mainly comes from the wrong variable definitions.
• Limitations. The constraint errors are the main error for OptiTree.
  • First, accurately formulating constraints requires a deep and nuanced understanding of the specific problem scenario and its domain knowledge. For highly specialized or novel OR problems, existing background knowledge may not fully capture the intricate details necessary for precise constraint definitions.
  • The second limitation is the intrinsic reasoning ability of the base LLM. While OptiTree provides a structured framework and knowledge, the final synthesis depends on the LLM's inherent reasoning capabilities. This framework may struggle with problems requiring significant logical or relational reasoning. For instance, in the staff rostering problem, the model must allocate staff for 24-hour shifts to meet coverage constraints (with workers working 8-hour days). If it fails to recognize that the demand period from 1:00 to 2:00 AM is covered by workers who began their shifts at 8:00 PM, it will result in an incorrect constraint and ultimately a flawed model.

Table 5: Failure cases information.

[1] Evo-Step: Evolutionary Generation and Stepwise Validation for Optimizing LLMs in OR.

[2] OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling.

[3] OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling.

[4] OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models.

Dear reviewer,

Could you please read the authors' response, and accomplish the mandatory acknowledgement a.s.a.p.?

AC

Dear Reviewer MotJ,

You noted that the limitations of our method could be discussed more thoroughly. To address this, we provide a detailed analysis of an example from the IndustryOR dataset.

This example demonstrates that while OptiTree successfully corrects fundamental structural errors (what [1] terms "Incorrect Modeling"), it still faces challenges with constraints that require deep, context-specific logical reasoning.

Problem. A factory requires a specialized tool for $n=10$ planning stages. $r_j$ specialized tools are needed. At the end of this stage, all tools used within this stage must be sent for repair. There are two repair methods: one is slow repair, which is cheaper (costs $b$ per tool) but takes longer ($p=3$ stages to return); the other is fast repair, which costs $c$ per tool and requires $q=1$ stages to return. If the tools cannot meet the needs, new ones must be purchased, with a cost of $a$ per new tool. Determine an optimal plan to minimize the cost.

• The Errors of the Standard Output Model. The standard output from DeepSeek-V3 produces a structurally flawed model. It tries to fulfill demand by combining new purchases with tools returning from repair, but it completely lacks any concept of inventory tools.

  The model given by the Standard output of DeepSeek-V3. Let $x_j \geq 0$ be the new tools purchased at stage $j$, $s_j \geq 0$ be the tools sent to slow repair at stage $j$, and $f_j \geq 0$ be the tools sent to fast repair at stage $j$.

  \min \quad &\sum_{j=1}^{n} \left(a x_j+b s_j+c f_j\right) &\\\\ &s_j+f_j= r_j\quad \forall j \in \\{1,\dots,n\\} &\text{(Repair Requirement)} \\\\ &x_j+s_{k-p}+f_{k-q} \geq r_j \quad \forall j \in \\{1,\dots,n\\} &\text{(Demand Fulfillment)} \end{aligned}$$

  Following the errors described in OptiMUS [1], the model output has the error type of Incorrect modeling, which fails to define the Inventory variables $I$.

• The Improvement of OptiTree. OptiTree correctly identifies that this problem contains an Inventory Problem subproblem. By applying the modeling thoughts, it generates a much-improved model that correctly introduces inventory variables $I$ and a proper inventory balance constraint.

  The model given by OptiTree (DeepSeek-V3). Let $x_j \in \mathbb{Z}\_+$ be the new tools purchased at stage $j$, $s_j \in \mathbb{Z}\_+$ be the tools sent to slow repair at stage $j$, $f_j \in \mathbb{Z}\_+$ be the tools sent to fast repair at stage $j$, and $I_j \in \mathbb{Z}\_+$ be the inventory of available tools at start of stage $j$. OptiTree finds a subproblem of the Inventory Problem for the original problem.

  \min \quad &\sum_{j=1}^{n} \left(a x_j+b s_j+c f_j\right) &\\\\&I_1=0&\text{(Initial Inventory)}\\\\&s_j+f_j=r_j \quad \forall j \in \\{1,\dots,n\\} &\text{(Repair Requirement)} \\\\ &I_{j+1}=(I_j+x_j - r_j)+s_{j-p}+f_{j-q}\quad \forall j \in \\{1,\dots,n\\} &\text{(Inventory Balance)} \\\\&I_j+x_j \geq r_j \quad \forall j \in \\{1,\dots,n\\} &\text{(Demand Fulfillment)} \end{aligned}$$

• The Errors in the Model Given by OptiTree. Despite the structural correction, the OptiTree model still contains a subtle error that requires common-sense logical reasoning. The model does not account for the finite 10-stage horizon. A human modeler would reason that it is illogical to pay for a repair that will finish after the tools are no longer needed.

  • Slow repairs (taking $p=3$ stages) are disabled for the last 3 stages ($j=8,9,10$) since tools sent for repair at these stages wouldn't return by stage 10, and fast repairs (taking $q=1$ stage) are disabled for the final stage ($j=10$) for the same reason. The repair requirement uses inequality

    $s_j+f_j \le r_j \quad\text{(Repair Requirement)}$

    rather than equality $s_j+f_j=r_j$, allowing flexibility to not repair tools when unnecessary---particularly near the end of the horizon. Thus, the final corrected model should be

    \min \quad &\sum_{j=1}^{n} \left(a x_j+b s_j+c f_j\right) &\\\\&I_1=0&\text{(Initial Inventory)}\\\\&s_j+f_j \le r_j \quad \forall j \in \\{1,\dots,n\\} &\text{(Repair Requirement)} \\\\ &I_{j+1}=(I_j+x_j - r_j)+s_{j-p}+f_{j-q}\quad \forall j \in \\{1,\dots,n\\} &\text{(Inventory Balance)} \\\\&I_j+x_j \geq r_j \quad \forall j \in \\{1,\dots,n\\} &\text{(Demand Fulfillment)} \end{aligned}$$

  • This demonstrates a limitation where OptiTree applies a general pattern correctly but misses a nuanced, context-dependent constraint. The error stems from a lack of OR knowledge patterns and a gap in the base LLM's logical reasoning about the problem's specific context. We believe this limitation will diminish as the underlying reasoning capabilities of LLMs continue to improve.

[1] OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models.

Best,

Authors

Dear Reviewer MotJ,

We sincerely thank you for your time and efforts during the rebuttal process. We are writing to gently remind you that the author-reviewer discussion period will end in less than 10 hours. We have responded to your further comments and eagerly await your feedback, and we sincerely hope that our response has properly addressed your concerns.

We eagerly await your feedback to understand if our responses have adequately addressed all your concerns. If so, we would deeply appreciate it if you could raise your score. We sincerely thank you once more for your insightful comments and kind support.

Best,

Authors

Thank you for your valuable comments. We sincerely hope that we could properly address your concerns.

W1: Details of motivating observations

• We use the IndustryOR dataset for three experiments. The "50 standard OR problems" refer to a collection of foundational OR problems in the OR textbook [1], including classical problems such as the standard TSP, Knapsack, set packing, assignment, and facility location problems.

• Providing the ground-truth model. (1) We first collect the optimization models in the textbook [1] as a ground-truth model for the 50 standard problems. (2) For a problem in IndustryOR to formulate, we use LLM to identify whether the problem contains subproblems in the 50 standard OR problems. (3) If a subproblem is identified, we augment the LLM's prompt for optimization modeling by integrating the relevant ground-truth model of the subproblem. The simplified prompt is as follows.

  You are a mathematical formulator to formulate the given OR problem as an optimization model and generate Gurobi code to solve the model.

  {problem description}

  The problem has a subproblem of {subproblem name}. During the modeling process, we can refer to the subproblem model: {subproblem model}

W1: Details of statement thoughts extraction and similarity function SimLLM.

The extraction of statement thoughts and the similarity function SimLLM are both processes handled by a LLM using specific prompts.

• Statement thoughts are concise, high-level statements that summarize the key features and requirements of the problem. They are generated to create a clear and comprehensible format for subproblem identification. We provide an example here (Appendix H.1).

  Diet Problem with Integer Constraint

  • The Diet Problem with Integer Constraint involves determining the quantity of each food item to meet nutritional requirements while minimizing cost.
  • Nutritional Constraints: The selected food items provide at least the required amount of nutrients.
  • Cost Minimization: The total cost of the selected food items should be minimized.
  • Integer Servings: Food items must be integer values.

  An example prompt to obtain statement thoughts is:

  Your task is to summarize the type of this specific problem and provide a detailed statement thoughts of this type. You must classify the problem into more precise scenarios, such as the Traveling Salesman Problem (TSP), the facility location problem, and so on.

  Specific Problem: {problem description}

  Here is an output format example ...

• SimLLM provides a similarity score directly from the LLM, used during the tree search to measure how closely a candidate subproblem’s statement thoughts align with those of the target problem. This involves a detailed prompt asking the LLM to compare a list of statement thoughts and return the most similar one in JSON format. The output includes a discrete score of 0, 1, or 2, where 2 indicates high similarity. SimLLM is used to select the best-matching subproblem at each step of the tree search.

  Input problem: {input_problem_statement_thoughts}.

  You are given a list of defined subtypes and the statement thoughts of subproblems: {subproblems_statement_thoughts}.

  Your task is to determine if the specific problem belongs to one of the given subtypes. Return in the following JSON format:

  json{

  ​ matching_subtype: "",

  ​ reasons: "",

  ​ similarity: [0,1,2]

  }

W1: Evaluation with partial credit.

The objective metric is standard in the LLM-based optimization modeling literature [2, 3], and in subsequent works ORLM, LLMOPT, OptMATH, OptiBench. However, defining a robust "partial credit" metric is a significant challenge.

• Most existing benchmarks only contain labeled optimal value and do not contain annotated ground-truth optimization model, making the partial credit metric difficult to design.
• Even with ground-truth models, simple text-based or structural comparisons can be unreliable. A model may have minor syntactic differences from the ground truth but be semantically equivalent. Conversely, a model might be nearly identical yet contain a critical error, such as a misplaced inequality sign.
• To further address your concerns, we find that the training set in OptMATH has annotated models. We use 100 problems for evaluation. As our method and the baselines were not trained on this dataset, the comparison is safe and fair. We present the percentages of LLM-generated models that align with the ground-truth models based on statistical information, including the number of variables, binary variables, integer variables, constraints and objective values. Note that all the statistical information is just a reference, as the mismatch of variables and constraints does not imply the model is incorrect. Notably, OptiTree exhibits the highest matching ratio with the ground-truth model, underscoring its reliability.

Table 1: Evaluation of partial credit.

W2: Tree construction seems to require human curation, and it's unclear whether the scalability and what happens when problems don't fit existing tree structures.

• We want to clarify that the tree construction and updating process is fully automated and does not require human curation for new domains. We construct the modeling tree using 400 randomly selected problems in the OR-Instruct dataset (training dataset for ORLM). We run across problems in the dataset to update the tree and use the single tree across the testing benchmarks.
• Scalability. We use just 400 problems from the OR-Instruct dataset for tree construction and test across a wide range of different problem domains. The superior performance demonstrates the scalability.
• If the tree search process cannot find any relevant subproblems for a given task (i.e., the search halts at the root node), OptiTree simply provides no modeling thoughts. In this case, the framework defaults to the LLM's baseline modeling capability without the enhancement from OptiTree.

W2 & Q: Analysis of tree maintenance costs over time and the results in Table 2 in the main text.

• Your understanding of the 1-time cost for tree construction is right. However, this part of the time is not included in Table 2 in the main text. The result in Table 2 represents the inference time required to generate the final optimization model for each new problem. We (1) use tree search to find an appropriate problem and (2) perform optimization modeling, combining the modeling thoughts from the problem. The two parts of time (Tree search and modeling) refer to these two steps.
• We provide the tree maintenance costs. We also provide the sum of tree construction and inference time (CPU time) as follows (in hours, and the reported Average Time in seconds). Compared to the inference time of the baselines or the time cost for fine-tuning LLMs, our method is the most efficient.

Table 2: The comparisons of the inference time.

W2: generalize to ML more broadly

While our work centers on optimization modeling within OR, the core principle of our approach—leveraging a hierarchical knowledge structure to guide an LLM in decomposing complex problems into simpler, solvable subproblems and retrieving contextual hints for each—represents a generalizable reasoning paradigm. We argue that hierarchical knowledge retrieval with tree search offers a versatile framework that could inspire advancements in other domains requiring multi-step, complex reasoning.

W2: Error cases and limitations

Following [2], the errors mainly arise from three aspects: missing or wrong constraints, incorrect modeling and coding errors. In [2], almost half of the errors come from the incorrect modeling (62.5% for ComplexOR), e.g., defining wrong binary variables in the model. We analyze the failures of OptiTree and normalize the failure rates to sum to 1. The results are presented in Table 3. We find that most errors are from the missing constraints, and the ratio of incorrect modeling is low. We have the following conclusions. (1) OptiTree can significantly reduce the error of incorrect modeling, primarily caused by wrong variable definitions. The results coincide with those in Table 3, where the number of variables is closest to the ground-truth model. (2) We can also see the limitations of OptiTree. While it substantially mitigates incorrect modeling through decomposition and modeling thoughts, it still relies on the base model's capacity to understand and interpret problems. The requisite OR knowledge for problem comprehension remains a limitation.

Table 3: Failure cases information.

[1] Optimization in Operations Research

[2] OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models

[3] Chain-of-Experts: When LLMs Meet Complex Operations Research Problems

Dear Reviewer JMeQ,

Thank you for your feedback and for acknowledging that our proposed revisions will improve the paper's clarity. We will ensure that all the details discussed during our discussions are fully incorporated into the final manuscript.

Please let us know if any points remain unclear or if you have any further questions. We eagerly await your feedback to understand if our responses have adequately addressed all your concerns. If so, we would deeply appreciate it if you could raise your score.

Thank you again for your time and constructive suggestions!

Best,

Authors