An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling

The presentation of this paper has several problems. Let me mention a few examples:

• Several acronyms are not explained: PDQP, LPs (line 51).
• A more detailed introduction is recommended for Section 2.2. This entire work is based on the so-called PDQP algorithm and the algorithm, its main intuition, and results have not been introduced thoroughly.
• In Algorithm 1, the IF condition in line 9 is not explained: the variable "residual" is not defined yet.
• In line162, please specify the dimension of $W^k_x$ and $W^k_y$. In general, I think it is a good habit to mention the dimensions of variables when they are first introduced.
• I suggest that Section 3.2 should be incorporated into Section 3.3: its content is not sufficiently rich enough to stand alone and its goal is to motivate why the authors propose an unsupervised loss function.
• In Section 3.3, the definitions of three loss functions $r_{primal}, r_{dual}$ and $r_{gap}$ should be introduced in the main text, and not the Appendix. Same remark for their normalized version.
• There is a double citation of the paper "A practical and optimal first-order method for large-scale convex quadratic programming" in lines 587 - 591.

Certain formulations and arguments in the paper need adjustment, in my opinion. Here are several examples:

• Proposition 3.2 and Section 3.2 are confusing. I think the authors want to convince us that the distance between the optimal solution and the predicted one is not a good training loss. This is debatable because in many situations such as image denoising, the distance is more important. Also, Proposition 3.2 does not seem to support their argument since it provides an upper bound.
• Theorem B.1 in Appendix B analyses a different formulation from (1). I do not read this proof carefully but the author should re-check it.
• In Appendix B, the training instances are generated by taking an instance from QPLIB, perturbing with Gaussian noise. However, the authors test with the same instances that they used to generate training data (see Tables 1 and 5). Is that fair?

1. The PDQP architecture, including deep unrolling, trainable parameters, projection operators, and channel expansion, and its use as a warm start to accelerate convergence closely mirrors existing work, PDLP. While effective, these techniques do not constitute novel contributions in the context of this paper, as they are directly aligned with the PDLP paper.
2. While the paper provides some foundational hyperparameter settings, such as the learning rate, optimizer, and training iterations, it lacks some details, such as the GNN architecture, the depth $K$, and the design of multi-layer perceptrons (MLPs) $f_x$, $f_y$, $g_x$, and $g_y$.
3. Although the experiments provide comparisons regarding acceleration as a warm start against traditional pure QDHG, the paper does not include a direct runtime comparison between the neural network model and the traditional QDHG. Without this data, it is difficult to evaluate the efficiency of the neural network and assess the trade-offs between efficiency and optimality.

• The technical contributions, such as the construction proof for neural network complexity, are incremental compared to previous work [1].
• While the construction proof demonstrates that a PDQP-net exists that can recover the PDQP algorithm and achieve linear convergence, the convergence guarantee for PDQP-net with trainable parameters remains unclear. Providing convergence guarantees for the training process is crucial for the unrolling-based L2O scheme [2].
• Given that the construction proof relies on the PDQP algorithm, which already achieves the optimal convergence rate among first-order methods, the authors should discuss the reasons behind the performance gain of PDQP-net. Theoretically, is there (constant factor) complexity improvement of PDQP-net compared with the PDQP algorithm?
• The author provides a complexity of PDQP-net as $O(log(1/\epsilon))$ in Prop. 3.1, but the dependence on the problem dimensions ($n$ and $m$) is unclear, which is also critical for understanding the scalability of this framework.

While the paper presents some interesting concepts, it feels somewhat incomplete.

1. The theoretical results are limited. Theorem 3.1 is straightforward: by setting parameters in the unrolled method to mimic PDQP, the conclusion follows. Additionally, Proposition 3.1 directly follows from the PDHG convergence results [1].

2. The experimental setup could be stronger. (i) For baseline comparisons, the proposed method is primarily compared with PDHG. A more recognized open-source solver is OSQP [2]. Additionally, as a first-order method, PDHG typically have very slow convergence for ill-conditioned problems (although its single-step complexity is cheap). More robust solutions like interior-point solvers (e.g., Gurobi) would be a stronger baseline. (ii) The dataset is insufficient to fully support the conclusions. For instance, the "QPLIB-8845" instance used in the paper is a single instance rather than a dataset [3], meaning the GNN is only trained on this one instance (along with some perturbed variations). If we want to solve another QP, we have to train another GNN for that. Typically, training GNN is more complex than solving a single QP, so we would want the trained GNN to generalize across multiple QP instances.

3. References on unrolling are lacking. Surprisingly, even the first paper on unrolling [4] is not discussed. A more comprehensive review of unrolling techniques would be beneficial.

[1] Chambolle and Pock. "A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging." Journal of Mathematical Imaging and Vision 2011.

[2] Stellato et al. "OSQP: an operator splitting solver for quadratic programs." Mathematical Programming Computation 2020.

[3] https://qplib.zib.de/QPLIB_8845.html

[4] Gregor and LeCun. "Learning fast approximations of sparse coding." ICML 2010.

The novelty of the proposed idea seems to be the main issue.

• as far as I understand the method "deep unrolling" refers to the unrolled operations of the operator splitting method for successive projection operations for primal and dual variables. As far as I can see, this is equivalent to the ADMM method. However, there is no mention of prior work on ADMM or other operator splitting methods.
• There are already existing works on differentiable or "unrolled" operator splitting methods as layers in deep neural networks for solving optimization problems in the learning to optimize setting
• the paper would benefit from a related work section highlighting similarities and differences between recent learning to optimize methods for parametric QP problems
• specifically, authors are encouraged to include recent advancements in differentiable optimization layers including QP-layers ADMM layers, and machine learning-based warm starting and operator splitting methods for QP and NLP problems
• This method looks very similar to that presented in the paper: "Differentiable Linearized ADMM" https://proceedings.mlr.press/v97/xie19c.html
• Other related work that uses differentiable Douglas-Rachford (DR) layers to warm-start QP is closely related but not mentioned in the paper "End-to-End Learning to Warm-Start for Real-Time Quadratic Optimization" https://proceedings.mlr.press/v211/sambharya23a.html
• limitations of the proposed method are missing