Enhancing Variational Quantum Algorithms: Effective Quantum Ansatz Design Using GFlowNets

I'll address each weakness and questions point by point:

While the experiments show that QAS does better than other models, the problems are relatively small and it is somewhat unclear how well this approach scales for larger problems.

While the presented experiments focus on standard benchmarks (e.g., 4 and 6 qubits) and we currently do not have access to a physical quantum computer, these are widely accepted in the community to validate new methods due to their analytical tractability and availability of baseline comparisons. However, the scalability of our approach is supported by the time complexity analysis in Section C, which demonstrates that our method scales polynomially with the number of qubits ($n$) and circuit depth ($l$). This is a significant advantage over methods like QuantumDARTS, which scale exponentially ($\mathcal{O}(4^n l T)$) with the number of qubits. We are actively working on extending our experiments to larger qubit systems, and results on these are planned for future work.

The use of GFNs doesn't seem to take into account much (if any) problem structure, such as symmetries. Many VQAs leverage permutation symmetry to reduce the size of the search space and achieve better results.

We appreciate this insightful observation. While our current implementation focuses on general-purpose quantum architecture search, we recognize the potential benefits of incorporating problem-specific structures, such as symmetries, to further optimize the search space and improve results. Notably, GFlowNets are well-suited for such enhancements, as they can seamlessly integrate these features, as demonstrated in [1,2].

Overall, this paper could be considered incremental as it simply uses a different ML model for VQAs to obtain improved performance. Among all the other papers along this line, there isn't much in this paper that particularly stands out.

We respectfully disagree with the characterization of this work as incremental. GFlowNets represent a significant departure from traditional gradient-based or reinforcement learning approaches to QAS. By learning a generative policy that samples solutions proportional to their rewards, GFlowNets address key challenges in quantum architecture search: diversity of solutions, efficient exploration of combinatorial spaces, and end-to-end learning without measuring the intermediate states. Our experiments demonstrate that this approach reduces circuit complexity by an order of magnitude while maintaining or improving accuracy. To further emphasize the novelty of our approach, we will add section B to highlight the limitations of existing methods and clarify the unique advantages of using GFlowNets for QAS.

Several minor grammatical issues and typos throughout the paper.

Thank you for pointing this out. We will thoroughly proofread the manuscript to address any remaining grammatical issues and typos to ensure clarity and professionalism.

For unweighted max-cut, why was conditional value-at-risk (CVaR) used rather than more common metrics, such as the average and maximum size of the cut found? CVaR seems to be a rather unusual metric, and it would be useful to also provide the operational meaning of this metric.

The reason we use this metric is to make direct comparisons with other methods, as the lack of publicly available implementations limits comparisons in other metrics.

We are grateful for the reviewer’s thoughtful feedback and constructive questions, which have helped us identify areas where additional clarification and discussion are necessary. If the reviewer finds our responses and proposed updates satisfactory, we kindly ask to reconsider the rating of our manuscript.

[1] Baking Symmetry into GFlowNets, arXiv:2406.05426

[2] Geometric-informed GFlowNets for Structure-Based Drug Design, MoML 2024 Spotlight

Dear Reviewer Tkhm,

Thank you for your insightful comments on our paper. As the discussion phase deadline approaches, we hope our responses have satisfactorily addressed your concerns. If there are any additional points that require clarification, we would be happy to provide further responses.

Best regards,

The Authors

I'll address the concerns and questions raised by the reviewer point by point:

This paper lacks a convincing explanation of how these two features contribute to the generation of effective quantum circuit ansatz. Also, this paper should provide circuit generated with GFlowNets using other objective functions.

While our current work utilizes the trajectory balance objective, we acknowledge the importance of exploring alternative objectives, such as flow matching loss and subtrajectory balance, as mentioned by the reviewer. These objectives offer valuable perspectives, particularly for cases involving long or incomplete trajectories. We plan to conduct a comprehensive comparison of these objectives in future work to evaluate their impact on the generation of effective quantum circuit ansatz.

Our choice to use the TB objective (https://arxiv.org/pdf/2201.1325) is based on its demonstrated empirical advantages, including faster convergence, improved sample diversity, more efficient credit assignment, and robustness to large action spaces—key considerations for our application. We will include this justification in the main text for greater clarity.

While ablating over different GFlowNet objectives would provide additional insights, it is not yet standard practice in the GFlowNet literature. Most papers adopt the TB objective as a default (e.g., arxiv:2310.08774, arxiv:2310.03579), which we have followed here. Additionally, the subtrajectory balance (subTB) objective, while valuable for very long trajectories (as shown in Learning GFlowNets From Partial Episodes For Improved Convergence And Stability), is less relevant for the relatively shallow quantum circuits considered in this work. However, we appreciate the suggestion and will consider these alternative objectives in future iterations of our research.

This paper highlights that the ansatz generation method could result in fewer parameters, gate counts, and depths and deals with scalability problems. However, the experiment circuits are relatively small, and the maximum circuit scale is only 10 nodes in the max cut problem. The authors should consider adding larger problems as related research has demonstrated its circuit search with more than 100 qubits (Wang, H., QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits).

We note that our experiments focus on small, standard benchmarks commonly used in VQA research for proof of concept and comparative analysis, as they provide a controlled environment for validating new methods. That said, we agree that scalability to larger systems is essential. While our theoretical time complexity analysis demonstrates polynomial scaling, empirical validation on larger systems remains a critical next step. Unfortunately, the lack of access to a physical quantum computer currently limits our ability to perform such validation. We would also like to clarify that the reference cited benchmarks problems with up to 27 qubits, which are already challenging to simulate on classical devices. Furthermore, their method exhibits exponential scaling with the number of qubits, as shown in their time complexity analysis, which fundamentally constrains its applicability to larger problems. In contrast, our method’s polynomial scaling offers a clear path toward addressing larger and more complex quantum systems.

This paper does not take into consideration hardware constraints, including hardware topology and noise, which are crucial to NISQ devices. These constraints may lead to an increase in circuit depth and gate counts.

We appreciate the importance of hardware constraints in practical quantum computing. While our current framework operates in a noise-free simulation environment, it is designed to be adaptable. By incorporating forward masks or hardware-aware reward functions, we can avoid those incompatible with specific topologies. This adaptability makes the approach suitable for deployment on NISQ devices. In future work, we plan to extend the method to explicitly account for noise and hardware topology, ensuring the generated circuits are not only efficient but also practical for implementation on real devices.

Thank you for acknowledging that our responses addressed your concerns. However, we would like to clarify and address the points raised:

More recent research, 'Élivágar: Efficient quantum circuit search for classification,' also demonstrates polynomial scaling, and they do consider the hardware limitations and constraints which seems more advanced compared to the results in this paper.

We respectfully disagree with the reviewer’s comparison. For the work “Élivágar: Efficient Quantum Circuit Search for Classification”, they focus on quantum machine learning (QML), they use the methods based on random sampling with two metric to post selecting circuits, Clifford noise resilience, and Representational capacity which is tailored to QML problems. The work focus on the QML problems, and they are not designed for the broader class of general quantum computing problems that our work addresses.

Moreover, rather than contradicting our approach, Élivágar supports the feasibility of incorporating hardware-aware features, as these can seamlessly be integrated into the sampling process or reward functions within the GFlowNet framework. This adaptability is a key strength of GFlowNets.

We also note that Élivágar includes comparisons with QuantumNAS. However, we would like to emphasize again that QuantumNAS is not considered a state-of-the-art method in this domain. More recent work, such as QuantumDARTS (ICML 2023) and CRLQAS (ICLR 2024), as highlighted by another reviewer, represents the current state of the art. Both of these works utilize benchmarks similar to ours, making them more relevant points of comparison. Notably, our method demonstrates superior performance in designing efficient circuits compared to these state-of-the-art approaches.

Apologize for my misleading, the scale is not meant for qubits but for the number of parameters

We agree that the reviewer’s initial comment misrepresented the scalability of the current research in this field. The clarification that the “scale” referred to the number of parameters rather than qubits highlights a misunderstanding that may have led to an inaccurate assessment of our work’s scope and contributions.

We appreciate the clarification but believe this initial misrepresentation should be taken into account when evaluating the validity and fairness of the comments.

• both of your technologies can be implemented on real quantum hardware no matter how many qubits you have - simulation results in terms of the number of qubits depend on your hardware memory side.

We would like to emphasize again that QuantumNAS relies on the unitary of the circuits to evaluate metrics for co-searching. This requirement inherently leads to exponential scaling with the number of qubits, which significantly limits its applicability to large systems. This limitation contrasts with our approach, which demonstrates polynomial scaling, making it more suitable for addressing larger and more complex quantum systems.

Furthermore, this limitation of QuantumNAS is also acknowledged and supported by the work Élivágar, as cited by the reviewer, which highlights the challenges of scaling such methods to larger qubit systems.

However, the paper still needs to improve a lot to be publishable at ICLR. I agree the idea of the paper could have more space to be improved, as I mentioned in my previous comments. However, I have to keep my negative score for the current version.

We are disappointed that despite recognizing the addressed concerns and the potential of the ideas presented in the paper, you have chosen to maintain a negative score without providing additional constructive feedback to justify this decision. If you believe there are further areas that require improvement to meet the standards of ICLR, we would appreciate specific feedback outlining these issues. A blanket statement that “the paper still needs to improve a lot” without elaboration does not provide actionable guidance, particularly when our responses have already addressed the points you raised.

We respectfully request a reconsideration of your evaluation, as we believe the paper presents a strong and original contribution that aligns with the expectations of the conference. Thank you for engaging in this discussion and for your time reviewing our work.

CRLQAS considers part of the hardware constraints of IBM_Mumbai in their paper, and they use a scaling weight on the noise model of IBM_Mumbai to provide evidence of their method could "adapt" the noise drift. QuantumDARTS is NOT using a noise model from the IBM backend but a readout error model with data obtained from Sycamore and Zuchongzhi. QuantumNAS consider multiple backends from IBM. Those are all you can easily find in their paper.

Based on your model and your rebuttal, I don't think it's too hard for you to demonstrate your model's results with the consideration of at least a noise model from any of the hardware inventor's backends. At least a runtime varies by the number of qubits with the same noise model could help reviewers re-evaluate the scalability of your backend.

My major point is all of those papers at least include parts of the hardware constraints into consideration, the mechanism shift on the IBM side is one of the interesting topics I recommend to you and is the topic what I think your model could demonstrate your model's advantage on.

I'll address each weakness point by point:

1. Regarding the scalability of our approaches:

Thank you for constructive suggestions. We acknowledge the importance of scalability in ensuring the practical utility of our method. For the QAS problem, the number of search possible gate arrangements grows exponentially with number of qubits and number of layers, which makes the search space large and combinatorial in nature. The motivation of our work is to address this challenge by developing a stochastic search algorithm. Our approach uses GFlowNets to learn stochastic policies to sample the vast search space, allowing us to find optimal quantum circuit architectures. Many works of GFlowNets demonstrate the numerical success of sampling in combinatorial space, including graph problem [1], language posterior inference [2], Molecule search [3]. We add a section to discuss about the complexity of our methods, which shows $\mathcal{O}\left(T L\left(n^3 l^3 E+n^2 l^2 E^2\right)\right)$, comparing to QuantumDARTS time complexity, $\mathcal{O}\left(4^n l T\right)$.

2. Concerning the results of bond lengths in Figure 2:

We appreciate this suggestion. In the revised manuscript, we will include the energy error along with the energy value (Figure 9 in the paper), similar to the approach in [4]. This addition will provide a clearer representation of our results' accuracy compared to the ground state.

3. About the current state-of-the-art in QAS:

Thank you for bringing this to our attention. We updated our comparison to include the curriculum reinforcement learning method mentioned in [5]. We will discuss how our approach compares to this more recent method in terms of performance on ground state energy estimation.

4. On the lack of theoretical guarantees:

We acknowledge the importance of establishing strong theoretical foundations. Notably, there is a growing theoretical research on GFlowNets, showing the connections to MCMC [6], reinforcement learning [7], and variational inference [8]. While this work primarily emphasizes numerical results, we plan to explore and expand upon the theoretical aspects in future studies.

We deeply value the reviewer’s thoughtful feedback and insightful questions, which have been instrumental in identifying areas for improvement and further clarification. If our responses and suggested revisions address the concerns effectively, we kindly invite the reviewer to reconsider their rating of our manuscript.

[1] Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets

[2]  Amortizing intractable inference in large language models

[3]Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation

[4] Experimental quantum computational chemistry with optimized unitary coupled cluster ansatz

[5] Curriculum reinforcement learning for quantum architecture search under hardware error

[6] Generative Flow Networks: a Markov Chain Perspective

[7] Generative Flow Networks as Entropy-Regularized RL

[8] GFlowNets and variational inference

Dear Reviewer B4xX,

Thank you for your insightful comments on our paper. As the discussion phase deadline approaches, we hope our responses have satisfactorily addressed your concerns. If there are any additional points that require clarification, we would be happy to provide further responses.

Best regards,

The Authors

Dear Reviewer,

The authors have provided their rebuttal to your comments/questions. Given that we are not far from the end of author-reviewer discussions, it will be very helpful if you can take a look at their rebuttal and provide any further comments. Even if you do not have further comments, please also confirm that you have read the rebuttal. Thanks!

Best wishes, AC

I'll address the concerns and questions raised by the reviewer point by point:

1. Regarding the computational cost and state space design:

The state space is designed as a directed acyclic graph where states represent partially constructed quantum circuits, and actions correspond to the selection of quantum gates as shown in Section 4.1. To handle the exponential complexity, we leverage GFlowNets, which effectively sample from a learned distribution proportional to the reward function. This allows efficient exploration of the combinatorial space without requiring exhaustive enumeration. Scalability is ensured through the use of transformers to the stochastic policy, which scale efficiently with the number of qubits and layers, as shown in our time complexity analysis.

2. On optimizing ansatz parameters and architecture simultaneously:

We appreciate this important point. The ansatz parameters and architecture are optimized sequentially ,not simultaneously, within the GFlowNet framework. During the forward pass, the architecture is sampled, and its parameters are optimized using a classical optimizer to compute the reward as shown in Section 4.1. Our reported time complexity $\left(\mathcal{O}\left(T L\left(n^3 l^3 E+n^2 l^2 E^2\right)\right)\right.$ ) demonstrates the efficiency of this approach compared to baseline methods like QuantumDARTS $\left(\mathcal{O}\left(4^n l T\right)\right.$ ). The computational costs are detailed in the manuscript (Section C).

3. Concerning the small size of example cases:

We understand the reviewer's concern about the limited size of our example cases. While the examples involve 4 and 6 qubit Hamiltonians, these are standard benchmarks in variational quantum algorithm research. These problems serve as proof of concept and validate the efficiency of GFlowNet-sampled circuits. Future work will focus on scaling the approach to larger systems and demonstrating its applicability to higher qubit counts.

4. About the number of iterations/circuit trials for convergence:

We apologize for overlooking this important detail. The framework typically converges within approximately 6000 iterations for the problems considered, as illustrated in the training rewards plot (Figure 5). This demonstrates the efficiency of GFlowNets in achieving convergence compared to traditional search methods.

5. Regarding the standard deviation in results:

We agree that including standard deviations would enhance the reliability of our results. In the revised version, we updated Tables 1 & 3 to include standard deviations for all reported metrics of our methods due to the lack of publicly available implementations of other methods.

6. On the limited comparison to other architecture search algorithms:

We appreciate this feedback. QuantumDARTS from ICML 2023 was chosen as it represented a state-of-the-art method at the time of our experiments. Based on feedback from other reviewers, we have added comparisons to an additional state-of-the-art method based on reinforcement learning. We would be glad to compare with more methods if the reviewer could provide specific suggestions.

7. Although the proposed generative model finds better ansatz than the QDARTS, I image QDARTS would be more efficient.

We respectfully disagree that QDARTS would be more efficient. Our method offers better scalability and efficiency compared to QDARTS, especially for larger problem sizes. The GFlowNet approach allows for more efficient exploration of the architecture space, resulting in faster convergence and better-quality solutions. Additionally, our method's ability to generate diverse circuit architectures provides more flexibility in adapting to different problem requirements and hardware constraints.

We appreciate the reviewer’s detailed feedback and constructive suggestions, which will undoubtedly strengthen our work. If the reviewer finds these updates satisfactory, we kindly request to consider revisiting the rating of our manuscript.

Dear Reviewer 7NCk,

Thank you for your insightful comments on our paper. As the discussion phase deadline approaches, we hope our responses have satisfactorily addressed your concerns. If there are any additional points that require clarification, we would be happy to provide further responses.

Best regards,

The Authors

Dear Reviewer,

The authors have provided their rebuttal to your comments/questions. Given that we are not far from the end of author-reviewer discussions, it will be very helpful if you can take a look at their rebuttal and provide any further comments. Even if you do not have further comments, please also confirm that you have read the rebuttal. Thanks!

Best wishes, AC