We appreciate the reviewer for the constructive comments and for recognizing our contributions as the first to interpret NCO models and to provide a tool that helps reveal the internal mechanisms of these models. This aligns with the core objective of our work, and we are grateful for your recognition. We understand that the main concern lies in the need for broader empirical evaluation, particularly in terms of scalability to larger problem sizes and applicability to diffusion-based models, as well as how to effectively leverage the findings and extend the probing method’s practical implications. We have conducted additional experiments and provided further discussion to address these concerns, as outlined below.

---

1. W1-Experimental setup is limited in scope.

While we believe analyzing representations on a smaller scale is a meaningful way to isolate complexities and yield insightful findings, we follow your suggestion and include experimental results on TSP-500 and TSP-1000 instances to validate our probing-based approach at larger scales. Since solving large-scale TSPs to optimality is often infeasible, we use approximate solutions as labels instead of exact optima, consistent with prior work. All results are averaged over 1,000 instances.

As shown in the tables below, our key findings persist at larger problem scales. The superiority of LEHD over two heavier encoder-based models becomes even more pronounced in probing results, mirroring its strong performance on large instances and reinforcing our original conclusions. Moreover, LEHD continues to depend on the same key embedding dimensions across scales. This experiment confirms that probing remains an efficient and reliable method for interpreting model behavior in large-scale settings, since it only requires training a lightweight linear probe and thus presents no scalability barrier. We will integrate these results into Table 9 of the manuscript.

---

2. W2-Are the behavioral differences due to architectural variations or training algorithm differences?

Your confusion is completely understandable, as we also encountered this question frequently before conducting this study. It is precisely because it is difficult to pinpoint where differences in model behavior originate that we introduced probing, along with our newly proposed probing methods, to help address this challenge. Below, we first provide a direct answer to your question based on the conclusions from the original manuscript, and then summarize our findings from the manuscript regarding the impact of model architecture and training methods, respectively.

• Directly answer your question: In our original manuscript, we provide an answer to this question: model architecture influences inductive bias, which in turn shapes how models capture decision-relevant information. Models with better generalization tend to encode information using a small number of key dimensions and consistently rely on the same dimensions across different problems. On the other hand, at least in the case of POMO, we find that the training method has relatively little impact on the model's internal behavior. We summarize the relevant findings from our manuscript for your reference below.

• Model architecture: Through the probing analysis, we verify the impact of differences in model architecture. This includes the contrast between heavy encoder models, such as AM and POMO, and heavy decoder models, such as LEHD, in how they capture decision-relevant information. Specifically, LEHD with better generalization tend to consistently utilize the same embedding dimensions across different tasks. In contrast, models whose internal knowledge becomes dispersed or disorganized across dimensions during generalization tend to suffer performance degradation. These findings are discussed in detail in Section 4.3 in page 7 of the manuscript. Furthermore, the additional experiments presented in our response to W3, the details of which are provided below, offer further evidence supporting this conclusion.

• Training methods: We compared different training methods for the POMO model in the manuscript. Using probing, we analyzed the ability of POMO models trained via supervised learning (SL) and reinforcement learning (RL) to capture decision-relevant information. The results show that the SL variant performs slightly worse than its RL counterpart (0.84 vs. 0.86 on Probing Task 2), which is consistent with their overall performance gap (0.571% for SL vs. 0.134% for RL on TSP100). The detailed results can be found in the manuscript, lines 795 to 799.

---

3. W3-The conclusions from CS-Probing lack practical implications and fail to show how they could inform the design of improved NCO methods.

To address the concern that CS-Probing lacks practical implications, we added a simple yet effective new experiment based on the insights derived from CS-Probing in the original manuscript. Specificly, we conducted an experiment inspired by our probing insight that LEHD generalizes more effectively when its decisions depend on a small subset of embedding dimensions, indicative of sparsity in its final layers. We therefore introduced a regularization term to further promote sparsity in LEHD’s final-layer embeddings, thereby demonstrating how our insights can inform model design.

Specificlly, we trained LEHD models with varying regularization strengths (λ) on TSP100 and evaluated their generalization on TSP200 and TSP1000. The table reports the performance gap to the best known solutions. The first column (origin, namely λ = 0) corresponds to the unregularized baseline. Moderate regularization (λ = 1e-6 to 1e-3) improves generalization. We will include this experiment to demonstrate the practical value of our probing approach. The current results are preliminary due to time constraints, and a more thorough analysis on more layers/models will be provided in the revised paper.

Beyond this, we will enrich our discussions in Section 5 to outline several promising avenues for NCO models to enhance practical implications, including compression, distillation, and pruning of their representation space. Probing NCO layers across CO tasks and training stages may reveal task-specific dimensional requirements, thereby providing a systematic alternative to heuristic selection. We note that while no single study can encompass all possible experiments, our proposed probing pipeline and CS-Probing will serve as powerful tools for future research on the design and analysis of NCO models.

---

4. W4&Question-The benchmarked methods are outdated. Could the proposed CS-probing technique be applied to analyze current Diffusion-based modals as well?

Yes, probing can also be applied to analyze the current diffusion-based NCO models. For example, the classic model in [1] uses a denoising network that is a GNN, and the node embeddings of the graph in this GNN can be analyzed in the same way as the node embeddings in Transformer-based NCO models for solving routing problems. Additionally, we demonstrate how probing can be applied to GNN models in Section 3.3 on page 5 of the manuscript.

To further address your question, we add a prelimiary experiment of DIFUSCO. Due to version conflicts in DIFUSCO’s code, we rebuilt the environment with alternative packages compatible with our hardware, which prevented us from using its pretrained models. Instead, we present probing results from our own DIFUSCO training (20 epochs), showing that diffusion-based NCO models can also be effectively analyzed with probing.

The table below presents node embedding results for DIFUSCO. As the number of GNN layers increases, the ability to capture Euclidean distances slightly declines, while the ability to identify optimal edges improves. This distinction between low-level and high-level feature encoding aligns with patterns observed in previous models in the manuscript. These results demonstrate that probing is also effective for analyzing diffusion-based models, where the GNN acts as the denoising network.

[1] Sun, Zhiqing, and Yiming Yang. "Difusco: Graph-based diffusion solvers for combinatorial optimization."

---

5. Minor comment-The paper relies heavily on the appendix.

Thank you for this helpful suggestion. We appreciate your understanding of the page limit constraints. We will carefully reread the paper, and if we identify areas where the flow would benefit from additional context, we will incorporate the relevant key results from the appendix into the main text to improve clarity and accessibility.

We thank the reviewer for the constructive and valuable feedback! We appreciate the recognition that our work addresses fundamental gaps, introduces a novel and well-designed framework, overcomes critical barriers, and provides powerful, actionable insights. We also especially thank you for your suggestion to investigate sparsity regularization, which we have found to be beneficial for enhancing generalization. Please find below our responses detailing the new experiments and expanded discussion; we trust these enhancements will further demonstrate the value of our work.

---

1. W1&Q1-Lack justification for using linear probing over non-linear probes and does not compare CS-Probing with other interpretability methods. What unique insights does it provide?

I.Why linear probe?

In representation analysis, linear probing tests whether a feature is encoded in a form that can be extracted by a simple affine transformation. If a property is linearly separable, then the network has organized its internal geometry to make that feature readily available downstream. While NCO models are globally non-linear, our interest lies precisely in whether certain combinatorial features are represented in a geometrically simple (i.e., linearly decodable) form, because the final output layers of NCO models use the embeddings in a linear manner.

For non-linear probes, consider Probing Task 1 (Euclidean distance): although non-linear models may yield better regression performance, it becomes unclear whether the gains stem from the NCO model’s embeddings or the probe’s capacity. Moreover, non-linear probing adds complexity that obscures the internal representations we aim to interpret. Therefore, we adopt linear probing to better align with our goal of understanding how NCO models encode information.

II. Compare to other interpretability methods

• Gradient-based attribution. It reveals output sensitivity to input features but does not indicate where or whether specific combinatorial concepts are encoded in hidden representations. It also lacks structural insight into how the model organizes or reasons over the solution space. In contrast, probing offers a systematic interpretability framework by evaluating the linear decodability of target properties from intermediate representations. Our CS-Probing further identifies the specific layers and embedding dimensions where each concept is encoded.
• Visualization. Attention visualization highlights which nodes or edges the model attends to, but it neither quantifies representational separability nor reveals what specific information is encoded for decision-making in NCO models. In contrast, probing provides a systematic and quantitative assessment of linear separability and information content within embeddings, offering a layer-wise view of representational structure and deeper insight into how decision-relevant information is encoded. Besides, at the beginning of Section 4, we already employ activation visualization, a commonly used interpretability method, as shown in Figure 4. We will also add a discussion of attention visualization in the revised manuscript.
• Neuron ablation. Eliminating units (by zeroing or randomizing) can help validate the importance of specific neurons, but it is not suitable for identifying which dimensions capture particular information. Zeroing a dimension reveals little about its meaning, and in high-dimensional spaces, it is impractical to test all dimensions or combinations exhaustively. However, we still include a new experiment based on the neuron ablation method, where we use zeroing to validate the key dimensions identified by CS-Probing. Please refer to the response to Q3 for details.

III. Unique insights the CS-Probing provides

Compared to existing probing methods, CS-Probing offers a deeper level of analysis. As detailed in the manuscript, it not only assesses whether certain information is captured by the deep model's embeddings, but also reveals how that information is encoded. For example, CS-Probing enables us to uncover why LEHD demonstrates better generalization, a capability that standard probing methods typically lack.

IV. Improvement of the manuscript

Additional results on neuron ablation (see our response to Q3) and attention visualization will be included in the revised manuscript.

---

2. W2&Q2-No multiple hypothesis testing.

Thank you for the suggestion. We have incorporated it and added new experiments accordingly. Specifically, we applied the Benjamini-Hochberg procedure to control the false discovery rate (FDR) at 0.05 across the 256 tested dimensions. We found that the key LEHD dimensions previously identified as significant remain so after correction, consistent with the results reported in Table 2 of the manuscript. The updated results and a description of the correction procedure will be included in the revised manuscript.

Notably, the original p-values of these dimensions were well below conventional thresholds (less than 1e-4, as shown in the sample_codes\3_probing_exp\example_probing_exp.ipynb in the anonymous link provided in the manuscript), and they remained significant after FDR correction, further supporting the robustness of our findings.

---

3. W3-Limited evidence of generality across architectures or problems.

To address your concern, we conducted a preliminary experiment on DIFUSCO [1], a diffusion-based NCO. We trained the model for 20 epochs and then applied probing to analyze its representations. The table below presents node embedding results for DIFUSCO. As the number of GNN layers increases, the ability to capture Euclidean distances slightly declines, while the ability to identify optimal edges improves. This distinction between low-level and high-level feature encoding aligns with patterns observed in previous models in the manuscript. These results demonstrate that probing is also effective for analyzing other architectures, e.g., diffusion-based models, where the GNN acts as the denoising network.

[1] Sun, Zhiqing, and Yiming Yang. "Difusco: Graph-based diffusion solvers for combinatorial optimization."

---

4. W4&Q4-Lack a demonstration of how the insights can inform the design of improved NCO models.

We thank the reviewer for the valuable suggestion on applying regularization to encourage sparsity. We found that, based on CS-probing insights, simply regularizing the final-layer embeddings of the LEHD decoder can directly improve its generalization.

In this new experiment, we simply regularize the final-layer embeddings of the LEHD decoder to encourage sparsity in its representations, without making any changes to the original LEHD model architecture, training hyperparameters, or code implementation.

Specificlly, we trained LEHD models with varying regularization strengths (λ) on TSP100 and evaluated their generalization on TSP200 and TSP1000. The table reports the performance gap to the best known solutions. The first column (origin, namely λ = 0) corresponds to the unregularized baseline. Moderate regularization (λ = 1e-6 to 1e-3) improves generalization. The current results are preliminary due to time constraints, and a more thorough analysis on more layers/models will be provided in the revised paper.

We will include this simple yet effective generalization-enhancing experiment in the manuscript to demonstrate the practical value of our proposed probing approach. It provides direct evidence that insights obtained through probing can lead to tangible improvements in model generalization performance.

---

5. Q3-How robust is the two-dimensional finding across runs, and have ablation studies been conducted to test its causal impact (by zero the two key dimensions out) on performance?

• Re-training. We retrained the LEHD model under different random seeds. We found that although the key dimensions vary across different runs, the conclusion still holds: the LEHD model consistently relies on a small number of key dimensions to make decisions across different scales. Specifically, we find that the retrained LEHD model still relies on two key dimensions to capture decision-relevant information. Although these two dimensions differ from those listed in Table 2 of the manuscript, all of our conclusions remain valid.
• Neuron ablation. We include an new experiment based on the neuron ablation method, where we use zeroing to validate the key dimensions identified by CS-Probing. As shown in the table below, zeroing LEHD’s key dimensions (31 and 97) results in a significant performance drop (over 60%). Even zeroing either one individually causes a larger gap than zeroing other dimensions. For comparison, we zeroed two non-key dimensions: 98 (with large activation in Figure 4 of the manuscript) and 126 (with the highest probing coefficient among non-key dimensions in Table 2). In addition to these new experimental results, the original results in Table 3 of the manuscript also show that LEHD continues to rely on the same two key dimensions when generalizing to larger problem sizes. These results further confirm the importance of the key dimensions identified by CS-Probing.

We will include this new experiment and observation in the updated manuscript to enhance the robustness of our findings.

We appreciate the reviewer’s constructive feedback and recognition of our contributions to interpreting NCO models, particularly our insights into representation learning, inductive biases, and generalization mechanisms. This was the core motivation of our study, and we appreciate your recognition. We understand the main concern is regarding how to leverage these findings and the broader probing applicabilitys, which we address with new experiments and discussions as follows.

---

1. W1&Limitations-The authors focus on analyzing representations in neural combinatorial models. I encourage them to propose representation learning methods to enhance solver performance and generalization.

Thanks for the suggestion! Before we provide new experiments to directly show how to translate our findings into actionable designs, we would like to first emphasize that:

• We introduce the first probing framework for NCO and offers the first comprehensive study of internal representations in existing NCO models. We regard our work as a meaningful and necessary step, as our interpretability framework can provide the foundation for more principled novel designs in the future.
• More importantly, we enhances transparency and address the key questions raised by the operations research (OR) community. We uncover what knowledge NCO models capture and how they are leveraged to solve CO problems. These represent foundational questions raised by the OR community concerning the soundness of NCO methods. Our work directly studies them: NCO models capture both low-level features such as Euclidean distance and high-level strategies like avoiding myopic decisions.
• We provide new analytical tools (CS-Probing). Through CS-Probing, we find that: (1) distinct inductive biases learned by different NCO models; (2) they encode decision rationales in specific embedding dimensions closely tied to generalization; and (3) in the subspace formed by these key dimensions, model decisions can be interpreted by examining only a few informative dimensions.

We believe our findings are worth sharing with the community, which aligns well with many related interpretability research in deep learning. For example: (1) In computer vision, [1] employs probing to evaluate CNN layer representations, revealing that deeper layers produce increasingly linearly separable features, which informs future representation evolution, skip connection design, etc; (2) In natural language processing, [2] demonstrates that probing enhances interpretability, guides architecture design, uncovers latent capabilities, and inspires representation manipulation and alignment.

We agree with the reviewer that designing methods based on our findings is exciting. To show such potential, we conducted an experiment inspired by our probing insight that LEHD generalizes more effectively when its decisions depend on a small subset of embedding dimensions, indicative of sparsity in its final layers. We therefore introduced a regularization term to further promote sparsity in LEHD’s final-layer embeddings, thereby demonstrating how our insights can inform model design.

Specificlly, we trained LEHD models with varying regularization strengths (λ) on TSP100 and evaluated their generalization on TSP200 and TSP1000. The table reports the performance gap to the best known solutions. The first column (origin, namely λ = 0) corresponds to the unregularized baseline. Moderate regularization (λ = 1e-6 to 1e-3) improves generalization. We will include this experiment to demonstrate the practical value of our probing approach. The current results are preliminary due to time constraints, and a more thorough analysis on more layers/models will be provided in the revised paper.

Beyond this, we will enrich our discussions in Section 5 to outline several promising avenues for enhancing NCO models, including compression, distillation, and pruning of their representation space. Probing NCO layers across CO tasks and training stages may reveal task-specific dimensional requirements, thereby providing a systematic alternative to heuristic selection. We note that while no single study can encompass all possible experiments, our proposed probing pipeline and CS-Probing will serve as powerful tools for future research.

[1] Alain, Guillaume, and Yoshua Bengio. "Understanding intermediate layers using linear classifier probes."

[2] Allen-Zhu, Zeyuan, and Yuanzhi Li. "Physics of language models: Part 3.1, knowledge storage and extraction."

---

2. W2-Large scale problems.

While we believe analyzing representations on a smaller scale is a meaningful way to isolate complexities and yield insightful findings, we follow your suggestion and include experimental results on TSP-500 and TSP-1000 instances to validate our probing-based approach at larger scales. Since solving large-scale TSPs to optimality is often infeasible, we use approximate solutions as labels instead of exact optima, consistent with prior work. All results are averaged over 1,000 instances.

As shown in the tables below, our key findings persist at larger problem scales. The superiority of LEHD over two heavier encoder-based models becomes even more pronounced in probing results, mirroring its strong performance on large instances and reinforcing our original conclusions. Moreover, LEHD continues to depend on the same key embedding dimensions across scales. This experiment confirms that probing remains an efficient and reliable method for interpreting model behavior in large-scale settings, since it only requires training a lightweight linear probe and thus presents no scalability barrier. We will integrate these results into Table 9 of the manuscript.

---

3. W3-The linear probing model and probing process are unclear.

The linear probing model is a linear model, specifically

$y=\beta_0+\beta_1 x_1 + ... \beta_n x_n$

where $y$ represents the information to be probed, $x_n$ is the activation value of the $n$-th dimension of the embedding of the NCO model, and $\beta$ is the regression coefficient.

The probing process is detailed in Section 3.1 and Appendix B. Briefly, probing tests whether a deep model encodes specific information by training a simple (typically linear) model to predict target labels from the model’s internal activations. Successful prediction indicates that the relevant information is present in the embeddings. This approach is standard in other domains as well.

We will revise the paper to more clearly describe the linear probing model. Please let us know if there are any specific points of confusion that we can further clarify.

---

4. Q1-Do the probing analyses also apply to diffusion-based models?

Yes, probing can also be applied to analyze diffusion-based NCO models. For example, the classic model in [1] uses a denoising network that is a GNN, and the node embeddings of the graph in this GNN can be analyzed in the same way as the node embeddings in Transformer-based NCO models for solving routing problems. Additionally, we demonstrate how probing can be applied to GNN models in Section 3.3 on page 5 of the manuscript.

To further address your question, we conducted a preliminary experiment on DIFUSCO. We trained the model for 20 epochs and then applied probing to analyze its representations. The table below presents node embedding results for DIFUSCO. As the number of GNN layers increases, the ability to capture Euclidean distances slightly declines, while the ability to identify optimal edges improves. This distinction between low-level and high-level feature encoding aligns with patterns observed in previous models in the manuscript. These results demonstrate that probing is also effective for analyzing diffusion-based models, where the GNN acts as the denoising network.

[1] Sun, Zhiqing, and Yiming Yang. "Difusco: Graph-based diffusion solvers for combinatorial optimization."

---

5. Q2-Can we use less embeddings for LEHD, while the performance does not decrease?

Insightful questions! Dimensionality selection in deep learning models is a complex and interesting problem. To first answer the question: the LEHD model as a whole does not achieve comparable performance with very low-dimensional embeddings (e.g., 8 or 16 dimensions). Although the LEHD model appears to rely primarily on just two dimensions of the final-layer embeddings to make near-equivalent decisions, this does not imply that the entire model operates using only these two dimensions. The earlier layers still require high-dimensional representations to effectively learn and process information.

We agree with the reviewer that developing a principled method for determining the necessary dimensions of NCO models while retaining perforamnce is an exciting future work. Notably, the additional experiments in our response to W1 have already provided preliminary evidence that insights from our probing analysis can improve LEHD’s generalization performance while employing fewer active embeddings via sparsity regularization. We consider these findings both compelling and worthy of extension to other layers in future work, where our proposed probing pipeline/tools will furnish a systematic approach for identifyin key dimensions of each layer.

We appreciate the reviewer for the constructive feedback and for recognizing the importance of our interpretability exploration in NCO models, as well as the interesting and valuable findings we present. We are also grateful for the acknowledgment that our study is extensive and supported by a large number of experiments and data. The probing methodology and the dataset we constructed reflect the core contributions we hope to bring to the NCO field, and we sincerely appreciate your recognition of their value. We understand that the main concern lies in the lack of actionable implications of the probing results in the current manuscript, which we address through new experiments and additional discussions as outlined below.

---

1. Cons-The probing idea with auxiliary tasks has unclear actionable implications.

To address the concern that CS-Probing lacks actionable implications, we added a simple yet effective new experiment based on the insights derived from CS-Probing in the original manuscript. Specificly, we conducted an experiment inspired by our probing insight that LEHD generalizes more effectively when its decisions depend on a small subset of embedding dimensions, indicative of sparsity in its final layers. We therefore introduced a regularization term to further promote sparsity in LEHD’s final-layer embeddings, thereby demonstrating how our insights can inform model design.

Specificlly, we trained LEHD models with varying regularization strengths (λ) on TSP100 and evaluated their generalization on TSP200 and TSP1000. The table reports the performance gap to the best known solutions. The first column (origin, namely λ = 0) corresponds to the unregularized baseline. Moderate regularization (λ = 1e-6 to 1e-3) improves generalization. We will include this experiment to demonstrate the practical value of our probing approach. The current results are preliminary due to time constraints, and a more thorough analysis on more layers/models will be provided in the revised paper.

Beyond this, we will enrich our discussions in Section 5 to outline several promising avenues for NCO models to enhance practical implications, including compression, distillation, and pruning of their representation space. Probing NCO layers across CO tasks and training stages may reveal task-specific dimensional requirements, thereby providing a systematic alternative to heuristic selection. We note that while no single study can encompass all possible experiments, our proposed probing pipeline and CS-Probing will serve as powerful tools for future research. We look forward to seeing how it can facilitate the discovery of further actionable insights to advance the field of NCO.

---

2. Q1-While probing helps to build a smaller model with similar performance, same outcome can be achieved by using hyperparameter tuning methods. What other practical implications?

First, we would like to clarify that regarding your point that "the same outcome can be achieved by using hyperparameter tuning methods," this is somewhat incorrect. We apologize for any misunderstanding. Next, we would like to further explain the practical implications of probing and our proposed CS-Probing method.

I. Why can't hyperparameter tuning methods achieve the outcomes obtained through probing? Specifically, what distinguishes the conclusions drawn from our probing methods from simply reducing the model's dimensionality while maintaining similar performance?

• Although LEHD relies on only two key dimensions, it cannot simply reduce its embedding dimensions at will. Although the LEHD model appears to rely primarily on just two dimensions of the final-layer embeddings to make near-equivalent decisions, this does not imply that the entire model operates using only these two dimensions. The earlier layers still require high-dimensional representations to effectively learn and process information.
• CS-Probing provides valuable insights that can be used to improve NCO models, which experimental methods like hyperparameter tuning cannot achieve. In our response above (under "Cons"), we demonstrate how insights obtained through CS-Probing can guide LEHD model improvements in generalization. This represents a concrete application that cannot be achieved through standard experimental practices such as hyperparameter tuning. Methods like hyperparameter tuning can only improve the model heuristically, rather than through the analysis and understanding of the model’s internal mechanisms, which allows for more evidence-based research.

II. Practical implications of probing and our proposed CS-Probing method.

• Probing enhance transparency and address the key questions raised by the operations research (OR) community. We uncover what knowledge NCO models capture and how they are leveraged to solve CO problems. These represent foundational questions raised by the OR community concerning the soundness of NCO methods. Our work directly studies them: NCO models capture both low-level features such as Euclidean distance and high-level strategies like avoiding myopic decisions.
• In the original manuscript, we have demonstrated how CS-Probing can be used to analyze and understand NCO models, providing actionable insights for improvement. Through CS-Probing, we find that: (1) distinct inductive biases learned by different NCO models; (2) they encode decision rationales in specific embedding dimensions closely tied to generalization; and (3) in the subspace formed by these key dimensions, model decisions can be interpreted by examining only a few informative dimensions.

III. Conclusion.

To the best of our knowledge, we are the first to introduce probing techniques in the context of NCO. Our proposed probing tasks and the CS-Probing framework not only provide a new method comparable to hyperparameter tuning for improving model performance, but also allow us to verify whether specific knowledge is encoded in NCO representations and to reveal how black-box models make decisions by identifying the most informative embedding dimensions.

This is significant for future research, as it opens up new directions for understanding the inductive biases of NCO models, evaluating the presence and form of knowledge encoded in their embeddings, and guiding the design of more efficient and interpretable representations. As illustrated in the additional experiment presented in our response above (under “Cons”), probing can directly inform actionable strategies. This provides supporting evidence for the practical utility of probing in both interpreting and improving NCO models.

---

3. Q2-For Probing Task 2, how does your study address the issue of non-uniqueness in optimal solutions, especially when drastically different solutions have similar optimization costs?

The answer to this question can be found in lines 624 to 631 on page 15 of the manuscript. Here, we summarize the relevant text from the manuscript to explain how we address the issue of non-uniqueness in optimal solutions, for your reference:

• We first follow the procedure described in page 614 to 623 of the manuscript to collect positive and negative labels for a given instance. Specifically, we extract an edge from the optimal solution as the positive label (label 1), and the edge connected to the same node with the closest Euclidean distance that is not part of the optimal solution as the negative label (label 0). For example, the positive edge might be between node a and node b, while the negative edge is between node a and node c.
• We then examine the edge labeled as 0. To verify its correctness, we add a constraint to the mathematical model that forces the connection between nodes a and c. The new optimal solution obtained under this constraint has a higher cost than the original unconstrained solution, indicating that this edge is not part of any optimal solution for this instance.
• Similarly, for the data labeled as 1, we add a constraint that prevents the connection between nodes a and b. The resulting solution is also worse than the original, confirming that both labels are valid.

Using the above method, we ensure that the routing instance dataset we collect does not contain instances with multiple optimal solutions, thereby preserving the rigor and validity of the probing dataset.