A Quantum Circuit-Based Compression Perspective for Parameter-Efficient Learning

Q2. Additionally, could the authors ... empirical evidence?

Response: We thank the reviewer for this insightful question. While a comprehensive theoretical analysis of QPA is planned for future work, this study primarily focuses on demonstrating the feasibility and potential of QPA for PEFT in large-scale machine learning tasks, particularly billion-parameter LLMs. In Appendix B, we outline possible directions for developing a theoretical framework for QPA, building on existing studies of VQC and QNN theories.

We also appreciate the reviewer highlighting tensor networks as a potential link to the theoretical foundation of QPA. Tensor networks are highly effective in decomposing high-dimensional tensors into a network of smaller interconnected tensors, capturing local correlations while discarding irrelevant degrees of freedom. Given that quantum circuit states can also be approximated by tensor networks (with accuracy depending on the rank), the quantum circuit used in QPA can be viewed as a more general and flexible representation of high-dimensional tensors. This flexibility enables a richer representation of the parameters in PEFT methods.

While the development of a full theoretical framework remains an important goal for future work, the empirical results presented in this study provide a solid foundation. These findings demonstrate the practical viability of QPA and its scalability to large models, serving as an essential step toward understanding and optimizing QPA for long-term effectiveness.

[1] Jarrod R McClean, et al. Barren plateaus in quantum neural network training landscapes. Nature communications, 9(1):4812, 2018.

[2] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249–256. JMLR Workshop and Conference Proceedings, 2010.

[3] Laia Domingo, G Carlo, and F Borondo. Taking advantage of noise in quantum reservoir computing. Scientific Reports, 13(1):8790, 2023.

[4] Chuhan Wu, et. al. Noisytune: A little noise can help you finetune pretrained language models better. arXiv preprint arXiv:2202.12024, 2022.

We sincerely thank Reviewer KJS2 for the invaluable feedback and suggestions, which have significantly enhanced the quality of our work. We greatly appreciate the time and effort dedicated to providing such thoughtful insights, which have been crucial in helping us refine and improve our paper.

W1. Efficiency Concerns: ... landscapes typical of QNN.

Response: We thank the reviewer for their thoughtful comment. In response to the concerns regarding challenges like barren plateaus, which can significantly hinder the training process, we have added a new appendix H section titled “On Gradient Variance of Quantum Circuit Parameters” to address this issue.

In the original study [1], barren plateaus are associated with loss functions defined as the expectation value of Hermitian operators. In contrast, QPA evaluates the loss of a target classical model (an LLM in this study), with updates generated using PEFT methods. The QNN output provides measurement probabilities rather than operator expectations, which distinguishes our approach.

Our analysis of QNN gradient variance, $\partial \theta$, in QPA-LoRA results for GPT-2 (Fig. 10) reveals no significant downward trend as the number of qubits increases. This stability is likely due to the backward propagation of gradients through the mapping model, mitigating the exponential vanishing trends typically seen in barren plateaus. However, we observed a slight downward trend with increasing QNN repetitions $L$, resembling patterns seen in deep feedforward networks and barren plateaus [1,2], though to a much lesser extent. These findings suggest that QPA’s gradient dynamics share characteristics of both quantum and classical paradigms without exhibiting severe barren plateau effects.

W2. The inputs of ... sufficiently explored.

Response: To explore the impact of limited measurement shots on the accuracy of estimated probabilities and their subsequent effect on fine-tuning LLMs, we have added Appendix G, “Effects of Finite Measurement Shots and Noise,” to the revised version. As illustrated in the newly added Fig. 8, increasing the number of measurement shots improves performance, eventually converging toward exact simulation results. While practical implementations must carefully balance measurement budgets and performance, our method demonstrates effectiveness even with a finite number of shots, highlighting its utility beyond idealized strong simulations.

Additionally, in this appendix, we incorporate noise models provided by IBM, based on real hardware data from ibm_torino and ibm_fez, to investigate the potential behavior of QPA in real-device environments. Interestingly, introducing noise from ibm_torino and ibm_fez yields mixed results: while performance decreases slightly in some cases, it improves in most instances. This observation is consistent with prior quantum computing studies, where quantum noise has been shown to enhance performance in specific paradigms [3].

In a similar way, classical machine learning studies have found that small amounts of noise can improve LLM fine-tuning [4]. Since the quantum circuit in QPA generates measurement probabilities that are input into a classical MLP mapping model to produce PEFT parameters, it is plausible that quantum noise plays a similar role to the “small noise” effect observed in these studies. Such noise may facilitate exploration of a broader parameter space, ultimately aiding in the fine-tuning of LLMs.

Q1. The question is that ... unavailable quantum computing?

Response: We thank the reviewer for raising this intriguing question. From the perspective of quantum-inspired methods, fast quantum simulation or approximation techniques, such as tensor networks, offer a feasible way to represent the state of a quantum circuit. However, these methods typically require circuits with limited entanglement or shallow depths to ensure that the quantum state remains sufficiently simple for approximation.

In contrast, the quantum circuit proposed in this study is designed for greater generality, enabling the construction of complex and deep quantum neural networks. At the current stage, we believe that tensor networks may be a viable alternative in scenarios where the dimensionality of the tuning target is inherently low. For instance, if the “update” of the weight matrix during model training—the tuning target—is simple enough to be represented by low-dimensional structures, tensor networks could be a valuable tool to explore within the QPA framework. This remains an interesting avenue for future investigation.

Dear Reviewer,

With the rebuttal phase nearing its conclusion, we would like to gently remind you of our responses.

We have thoughtfully addressed all your comments and questions, and we hope our revisions meet your expectations.

If there are any unresolved concerns, we are more than willing to discuss them further. Otherwise, if you find our updates satisfactory, we kindly invite you to consider raising your score.

Thank you again for your time and valuable feedback!

Q5. Best choice of ... hyperparameters for other models?

Response: We thank the reviewer for providing an insightful question. The choice of block size $(n_{mlp})$ presents an interesting trade-off. Larger block sizes increase the number of trainable parameters and improve performance (lower testing perplexity), while smaller block sizes result in higher compression but require more qubits. Striking the optimal balance between these factors is likely an open question, as resource requirements vary depending on the user’s constraints and objectives.

Regarding the best number of QNN repetitions ($L$) for different PEFT methods, the required qubit count can be determined once the target PEFT parameters are identified. Empirical studies suggest that $L$ typically scales proportionally with the number of qubits, either as $O(N)$ or, in some cases, $O(N^2)$ [4,5,6]. From a theoretical perspective, while studies on the overparameterization behavior of QNNs [7] indicate that QNNs can be overparameterized with a polynomial number of parameters ($O(poly(N))$), the structure of QPA differs from traditional QML approaches that use the expectation value as a loss function. Instead, QPA focuses on generating the parameters for PEFT methods. As such, the choice of $L$ within $O(N)$ or $O(N^2)$ should strike a proper balance between expressivity and avoiding overparameterization.

[1] Laia Domingo, G Carlo, and F Borondo. Taking advantage of noise in quantum reservoir computing. Scientific Reports, 13(1):8790, 2023.

[2] Chuhan Wu, et. al. Noisytune: A little noise can help you finetune pretrained language models better. arXiv preprint arXiv:2202.12024, 2022.

[3] Andrea Mari, et al. Transfer learning in hybrid classical-quantum neural networks. Quantum, 4:340, 2020.

[4] Marco Cerezo, et al. Variational quantum algorithms. Nature Reviews Physics, 3(9):625–644, 2021.

[5] Sukin Sim, et al. Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms. Adv. Quantum Technol., 2: 1900070

[6] Marcello Benedetti, et al. Parameterized quantum circuits as machine learning models. Quantum Science and Technology, 4(4):043001, 2019.

[7] Larocca, Martin, et al. "Theory of overparametrization in quantum neural networks." Nature Computational Science 3.6 (2023): 542-551.

Q1. Lack of comparison ... rather than theoretical support?

Response: We thank the reviewer for this question. As mentioned in our response to w2, we have added Appendix F, “Effects of Different Circuit Ansatz,” to the revised version to explore this topic further.

Q2. Performance in noisy ... be considered.

Response: We thank the reviewer for this question. As noted in our response to w1, we have added Appendix G, “Effects of Finite Measurement Shots and Noise,” to the revised version to investigate this topic further. While these results are based on simulations rather than actual hardware, the noise model used is derived from real hardware data and should provide a reasonable approximation of QPA’s behavior on real devices.

Q3. Lack of discussion on ... and computing resources.

Response: We appreciate the reviewer’s comment regarding computational time, as it highlights an important aspect of our work. In response, we have added Appendix I, “On Computational Time,” to the revised version to investigate this topic in detail.

While reducing the number of training parameters in QPA does require additional computations for quantum operations and the mapping model, we anticipate that improvements in quantum hardware and operation speed will help mitigate this challenge in the future. To compare the computational cost of QPA with classical approaches at the current stage of simulation, we have included execution time data in the newly added Appendix I.

From the LoRA results in Fig. 2, we observe that the average execution time of QPA is approximately three times longer than that of classical LoRA. This means that while QPA completes one epoch, LoRA can complete three epochs. To evaluate the performance of both methods under comparable computational time constraints, we present results where QPA with one epoch is compared to LoRA with three epochs for GPT-2, and QPA with one and two epochs is compared to LoRA with five epochs for Gemma-2 (Fig. 11).

The findings show that, while QPA does not outperform LoRA to the same extent when both methods are trained for the same number of epochs, it demonstrates better performance in regions with smaller parameter sizes under similar total execution times.

Q4. In more complex ... issues in such scenarios?

Response: We thank the reviewer for raising this important question. To compare QPA with traditional QML methods, as discussed in Appendix A.2, it is indeed possible to generate parameters for tuning tasks using traditional QML approaches (e.g., by utilizing the expectation value of an operator as the output). However, a key distinction lies in the size of the output and the corresponding qubit requirements.

For an output of size $n$, conventional QML relies on the expectation value of the Pauli-z operator, $\langle \psi (\theta) | \sigma_z^{(i)} | \psi (\theta) \rangle$, for each qubit. This approach necessitates $n$ qubits, as outlined in Eq. 14 of [3]. In contrast, QPA generates parameters by leveraging the measurement probability $|\langle \phi_i | \psi (\theta) \rangle|^2$ for each basis, which significantly reduces the qubit requirement to $\lceil \log_2 n \rceil$.

In practical terms, the PEFT experiments presented in this study utilized between 4 and 11 qubits. If traditional QML were employed, the required number of qubits would scale exponentially, ranging from $2^4$ to $2^{11}$. This exponential scaling would result in an impractically large qubit requirement, rendering conventional QML infeasible for PEFT applications. Thus, in terms of scalability, the qubit usage of QPA demonstrates a significant advantage over traditional QML.

We are truly grateful to Reviewer Y4nw for the invaluable suggestions, which have greatly contributed to improving the quality of our work. We deeply appreciate the time and effort invested in providing thoughtful feedback, which has been instrumental in helping us refine and enhance our paper.

W1. The study assumes ... of the proposed QPA.

Response: We thank the reviewer for the insightful comment. Indeed, to consider noisy environments and finite measurement shots can better reflect the behavior of QPA method in a more realistic way. Thus, in the revised version, we have added Appendix G “Effects of Finite Measurement Shots and Noise” to address this topic.

In this appendix section, we utilize the noise model provided by IBM, based on real hardware data from ibm_torino and ibm_fez to investigate the potential behavior of QPA in real-device environments.

Interestingly, introducing noise from ibm_torino and ibm_fez leads to mixed outcomes: while performance decreases slightly in some cases, it improves in most instances. This aligns with previous findings in quantum computing studies, where quantum noise has been shown to enhance performance in certain paradigms [1].

Similarly, in classical machine learning, studies have demonstrated that small amounts of noise can improve the fine-tuning of LLMs [2]. Given that the quantum circuit in QPA generates measurement probabilities, which are then fed into a classical MLP mapping model to produce PEFT parameters, it is plausible that quantum noise serves a similar role to the “small noise” effect observed in prior studies. This noise may help explore a broader parameter space, ultimately aiding in the fine-tuning of LLMs.

W2. The paper mentions ... of the current choice.

Response: We sincerely appreciate the reviewer’s thoughtful question, which has provided valuable insights and helped us improve the context of our work. The choice of $R_Y$ and CNOT gates is motivated by the ultimate goal of generating parameters for PEFT methods, which are typically real numbers. This rationale underpins our decision to use the $R_Y$ circuit in the main content.

In response to the reviewer’s suggestion, we have added Appendix F, “Effects of Different Circuit Ansatz,” to the revised version to explore this topic further. Alongside the $R_Y$ and CNOT combination, we examined $R_X$ + CNOT , $R_Y R_Z$ + CNOT, and $R_X R_Z$ + CNOT configurations for generating LoRA parameters in the QPA task (Fig. 7). The results show no significant performance differences between these circuit constructions. However, when a finite number of measurement shots is considered in a noiseless environment (Fig. 9), the $R_Y$ + CNOT combination demonstrates superior performance compared to the $R_X$ + CNOT combination, suggesting that $R_Y$ + CNOT is indeed a reasonably effective choice.

W3. The method uses ... measurement noise.

Response: We thank the reviewer for the insightful comment. It is true that gradient computation from quantum measurements can introduce additional complexity in the training process due to realistic hardware noise. As addressed in our response to w1, the revised version includes an investigation into the effect of noise on QPA using noise models derived from real quantum computers. Our findings indicate that QPA exhibits minimal sensitivity to noise and, in some cases, even demonstrates improved performance under noisy conditions.

W4. The paper mentions ... effectiveness of QPA.

Response: We sincerely thank the reviewer for raising this important point. While it is true that the theoretical analysis of convergence behavior, trainability, and learnability characteristics of QPA is deferred to future work, the current study primarily focuses on demonstrating the feasibility and potential of QPA for PEFT in large-scale machine learning tasks, particularly for billion-parameter LLMs.

To address this gap, we have included several empirical analyses to provide insights into the method’s effectiveness. For instance, in the newly added Appendix H, we investigate gradient variance and demonstrate that QPA does not exhibit the exponential vanishing gradient phenomenon, often referred to as barren plateaus, typically associated with QNNs as the number of qubits increases. Furthermore, the robustness of QPA to noise, as analyzed in Appendix G, underscores its potential applicability to real-world quantum systems, even under non-ideal conditions.

While a comprehensive theoretical framework is indeed an important direction for future work, the empirical results provided in this study lay a strong foundation, demonstrating the practical viability of QPA and its ability to scale to large models. This serves as an essential step toward understanding and optimizing QPA for long-term effectiveness.

Dear Reviewer,

With the rebuttal phase nearing its conclusion, we would like to gently remind you of our responses.

We have thoughtfully addressed all your comments and questions, and we hope our revisions meet your expectations.

If there are any unresolved concerns, we are more than willing to discuss them further. Otherwise, if you find our updates satisfactory, we kindly invite you to consider raising your score.

Thank you again for your time and valuable feedback!

Q3. The current parameterized ... performance improvements?

Response: We sincerely appreciate the reviewer’s thoughtful question, which has helped us improve the context of our work. The rationale for using $R_Y$ and $CNOT$ gates is rooted in the ultimate goal of generating parameters for the PEFT methods, which are typically real numbers. This reasoning guided our choice of the $R_Y$ circuit in the main content.

While it is reasonable to explore alternative circuit constructions to identify potential performance improvements, we have added Appendix F, “Effects of Different Circuit Ansatz,” in the revised version to investigate this topic. In addition to $R_Y$ and CNOT combinations, we examined $R_X$ + CNOT , $R_Y R_Z$ + CNOT , and $R_X R_Z$ + CNOT combinations for generating LoRA parameters in the QPA task (Fig. 7). The results indicate no significant performance differences between these circuit constructions. However, when considering a finite number of measurement shots in a noiseless environment (Fig. 9), we observed that the $R_Y$ + CNOT combination outperforms the $R_X$ + CNOT combination.

[1]. Jarrod R McClean, et al. Barren plateaus in quantum neural network training landscapes. Nature communications, 9(1):4812, 2018.

[2]. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249–256. JMLR Workshop and Conference Proceedings, 2010.

[3] Scott Aaronson, Shadow Tomography of Quantum States, arXiv preprint arXiv:1711.01053, 2017.

We sincerely thank Reviewer tsMN for the valuable suggestions, which have significantly enhanced the quality of our work. We deeply appreciate the reviewer’s time and effort in providing thoughtful feedback that has helped us refine and improve our paper.

W1. From my understanding, the ... more practical alternative.

Response: We appreciate the reviewer’s constructive comments. In the implementation, QPA only requires the measurement probabilities from the quantum circuit (i.e., phase information and complex amplitudes are not needed). While this is computationally intensive, advancements in fast measurement processes are both necessary and anticipated to mitigate this challenge. Furthermore, the precision of the measurement probabilities is determined by the number of measurement shots. When the learning process does not demand highly precise results (similar to the effects of quantization), it is possible to reduce the number of measurement shots required, thereby mitigating the computational burden.

Although a generative model is considered one possible approach to address this issue, its role is more complementary than a full replacement. The quantum signal remains essential as the initial raw data, with the generative model assisting in enhancing or extending the data rather than entirely substituting it.

From the perspective of quantum-inspired algorithms, it is feasible to leverage fast quantum simulation or approximation methods, such as tensor networks, to represent the state of a quantum circuit. However, most of these methods require circuits with limited entanglement or shallow depths to ensure that the quantum state remains simple enough to approximate. In contrast, the current proposal for the quantum circuit is designed to be more general, allowing for the construction of complex and deep quantum neural networks.

Q1. Would the algorithm perform ... potentially improve performance.

Response: We express gratitude to the reviewer for asking the insightful question. We would like to clarify that the role of the quantum circuit is not only to produce an intermediate hidden state, but to produce $2^n$ distinct numbers (output measurement probabilities). For a classical neural network to achieve this, the width of the output layer should be at least $2^n$ to produce this amount of distinct outputs, for example, the shape of the final layer should be $(m, 2^n)$, where $m$ is a positive integer. This kind of substitution will obviously have more parameters than the target model or target PEFT method.

For the challenge of barren plateaus raised by the reviewer, we have added a new appendix section, “On Gradient Variance of Quantum Circuit Parameters,” to address this issue. In the original study [1], barren plateaus arise from loss functions defined as the expectation value of Hermitian operators. In contrast, QPA evaluates the loss of a target classical model (an LLM in this study), with updates generated using PEFT methods. The QNN output provides measurement probabilities, not operator expectations.

Our analysis of QNN gradient variance $\partial \theta$ in QPA-LoRA results for GPT-2 (Fig. 10) shows no significant downward trend with increasing qubits, likely due to backward propagation through the mapping model. A slight downward trend appears with increasing QNN repetitions $L$, resembling patterns in deep feedforward networks and barren plateaus [1,2], but to a much lesser extent. This suggests QPA’s gradient dynamics share traits of both quantum and classical paradigms without severe barren plateau effects.

Q2. Is there potential to ... reducing computational costs?

Response: We appreciate the reviewer’s constructive question. While it is possible to develop a method aimed at reducing the number of quantum state measurements, however, our planned approach focuses on leveraging a pre-trained generative model to generate measurement results from a limited set of real quantum measurements.

Regarding the shadow tomography method mentioned by the reviewer, its primary focus, as noted in [3], is to “recover not the full density matrix of $\rho$, but only the “shadow” that $\rho$ casts on the measurements $E_1, \ldots, E_M$.” Since this technique is designed to recover specific measurement outcomes rather than infer other measurement probabilities from known results, we currently find it less applicable to our specific challenges. However, we remain open to exploring potential methods that may prove helpful in addressing these issues as our work progresses.

Dear Reviewer,

With the rebuttal phase nearing its conclusion, we would like to gently remind you of our responses.

We have thoughtfully addressed all your comments and questions, and we hope our revisions meet your expectations.

If there are any unresolved concerns, we are more than willing to discuss them further. Otherwise, if you find our updates satisfactory, we kindly invite you to consider raising your score.

Thank you again for your time and valuable feedback!

Thank you for your detailed responses, which have addressed most of my concerns.

One more follow-up question: to produce the intermediate quantum state with $2^n$ distinct amplitudes, is it necessary to perform an exponential number of measurements, or would a polynomial number suffice?

I look forward to seeing your follow-up work on leveraging generative models to reduce the measurement burden.

Q2. Could the authors elaborate on the ... number of qubits?

Response: We appreciate the reviewer’s thoughtful suggestions, which have helped us enhance the comprehensiveness of our work. Regarding the concern about the barren plateau phenomenon, we have added Appendix H, “On Gradient Variance of Quantum Circuit Parameters,” to investigate the behavior of QPA in this context.

In the original barren plateau study [2], the exponential vanishing gradient behavior was derived from a learning task where the loss function is defined as the expectation value of a Hermitian operator. In contrast, our QPA approach differs. The objective function in QPA is the loss evaluated from a target classical model (an LLM in this study). The updates to this model are generated using PEFT methods, with the PEFT parameters produced by the quantum circuit and MLP mapping model. Consequently, the output of our QNN is not the expectation value of an operator but rather the measurement probabilities of basis states.

To explore whether barren plateaus are present in QPA, we analyzed the variance of the QNN gradients, $\partial \theta$, in the QPA-LoRA results for GPT-2, as shown in Fig. 10. Notably, no significant downward trend in gradient variance was observed with increasing qubit counts. This behavior may be attributed to the backward propagation of QNN gradients through the subsequent mapping model, which prevents them from exhibiting the exponential vanishing trends commonly seen in expectation value-based objectives in traditional QML.

Interestingly, however, a slight downward trend in gradient variance is observed as the QNN repetition $L$ increases. This behavior shows some similarity to both classical deep feedforward neural networks and barren plateaus in QNNs, as described in [2,3], albeit to a much lesser extent. This suggests that, while QPA does not exhibit barren plateaus in the same manner as traditional QML (particularly concerning qubit counts), its gradient dynamics may still reflect certain characteristics of both quantum and classical learning paradigms.

Furthermore, while our largest model studied is Gemma-2, with 2 billion parameters, QPA generates only the PEFT parameters (e.g., LoRA parameters) instead of directly generating the full model parameters. This significantly reduces the required number of qubits compared to directly handling the full model. As a result, the maximum number of qubits used in this study is limited to 11.

[1] Larocca, Martin, et al. "Theory of overparametrization in quantum neural networks." Nature Computational Science 3.6 (2023): 542-551.

[2] Jarrod R McClean, et al. Barren plateaus in quantum neural network training landscapes. Nature communications, 9(1):4812, 2018.

[3] Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249–256. JMLR Workshop and Conference Proceedings, 2010.

We sincerely appreciate the valuable suggestions provided by Reviewer jv7T. These insights have significantly contributed to enhancing the quality of our work. We are deeply thankful for the reviewer’s time and effort in helping us refine and improve our paper.

W1. The authors assert that ... classical fine-tuning methods difficult.

Response: Thank you for the insightful comment. Reducing the number of training parameters in QPA does indeed require additional computations for quantum operations and the mapping model. As quantum computing is still in its early stages, we anticipate improvements in operation speed in the future. To compare the computational cost of QPA with classical approaches at the current stage of simulation, we have included execution time data in Appendix I, “On Computational Time.” In the LoRA results from Fig. 2, the average execution time of QPA is approximately three times longer than that of classical LoRA. This implies that while QPA completes 1 epoch, LoRA can complete 3 epochs. To assess the performance of both methods under comparable computational time constraints, we present results where QPA with 1 epoch is compared to LoRA with 3 epochs for GPT-2, and QPA with 1 and 2 epochs is compared to LoRA with 5 epochs for Gemma-2 (Fig. 11). The findings indicate that, while QPA with a similar total execution time does not outperform LoRA to the same degree as when both methods are trained for the same number of epochs, it still demonstrates better performance in regions with smaller parameter sizes.

Regarding the concern about the barren plateau phenomenon, we have added Appendix H, “On Gradient Variance of Quantum Circuit Parameters” to address this topic in detail. Further elaboration on this matter will be provided in the response to Q2.

W2. Additionally, as I understand it from ... classical fine-tuning methods.

Response: Thank you for your constructive comments. First, regarding the reference to $4^n$ in the work mentioned by the reviewer [1], this dimension arises from the unitary compilation task, where the goal is to decompose a target unitary $V$ into a quantum circuit $U(\theta)$. In this case, the Dynamical Lie Algebra (DLA) defined in that paper has a dimension of $4^n$ . Since the task involves a quantum object as the target, such a high-dimensional requirement is understandable. However, the dimension of the DLA can vary depending on the specific task. One of the key findings of the referenced paper is that there can exist quantum neural networks (QNNs) that are overparameterized with a polynomial number of parameters.

In our case, the dimension of the low-rank weight matrix is $r(d+k)$ , which approximates or represents the “update” of the weight matrix with a dimension of $dk$ during fine-tuning. If the LoRA method is effective, this implies that the weight updates can be represented in a low-rank form. However, it is important to note that the “lower-dimensional” representation does not necessarily need to have a dimension of $r(d+k)$. The QPA method aims to generate PEFT parameters using fewer parameters via a quantum circuit, effectively exploring a more flexible (or potentially even lower-dimensional) representation of the weight updates during fine-tuning of large language models (LLMs).

Therefore, while the statement “Thus, to represent a parameterized quantum state for an arbitrary vector of dimension $r(d+k)$ —the dimension of the low-rank weight matrix $\Delta W = BA$ —the quantum circuit would also need to have parameters on the order of $r(d+k)$ ” may generally be accurate, it does not fully apply to our case. In our context, the target vector (or matrix) to be represented corresponds to the weight updates during fine-tuning. Empirical results demonstrate that these updates exhibit a lower degree of freedom, allowing similar performance to be achieved with fewer training parameters.

Q1. Could the authors provide ... computational complexity?

Response: Thanks to the reviewer for giving us helpful suggestions. As we have replied in w1, we have added an appendix section of the result for comparison of QPA and LoRA with similar total execution time, and the result shows that QPA could still be beneficial in the small size parameter region.

Dear Reviewer,

With the rebuttal phase nearing its conclusion, we would like to gently remind you of our responses.

We have thoughtfully addressed all your comments and questions, and we hope our revisions meet your expectations.

If there are any unresolved concerns, we are more than willing to discuss them further. Otherwise, if you find our updates satisfactory, we kindly invite you to consider raising your score.

Thank you again for your time and valuable feedback!

Q4. It is unclear to me ... MiniGrid environments (Liu et al., 2024c).”

Response: We thank the reviewer for this important question, as it provides an opportunity to highlight the contributions and significance of our work. This study pushes the state-of-the-art (SOTA) boundaries in several ways:

1. Model Size: Previous works (as mentioned in the question) in this domain primarily focused on toy-sized models, such as a CNN model with approximately 280k parameters. While these studies are valuable, they do not adequately reflect the practical scalability or real-world applicability of the proposed methods. In contrast, our work investigates parameter-efficient learning techniques for significantly larger models, tackling up to 2 billion parameters (Gemma-2). This represents a substantial leap, moving from 280k to 2 billion parameters, and demonstrates the scalability of quantum-assisted methods when applied to LLMs.

2. Novel Perspective on Parameter Efficiency: Instead of directly generating model parameters—a challenging and resource-intensive task—we focus on the parameters of the parameter-efficient “update” using PEFT techniques. By fine-tuning only specific layers of the model, we effectively utilize quantum resources for training while ensuring that the model remains operational entirely on classical hardware during inference. This unique perspective not only reduces the quantum hardware requirements but also makes the method more practical and applicable to large-scale systems.

3. First QML Application to Billion-Parameter LLMs: Our work is the first in quantum machine learning (QML) to address the fine-tuning of classical LLMs in the billion-parameter range (Gemma-2), with comprehensive simulation results in both ideal and noisy situations. This achievement highlights the potential of QML to handle state-of-the-art machine learning tasks, bridging the gap between quantum and classical paradigms in a meaningful way. In contrast, most QML studies to date have focused on significantly smaller-scale tasks.

In summary, this work advances the field by demonstrating the scalability and applicability of QML to real-world, large-scale models, showcasing its potential to fine-tune billion-parameter LLMs while leveraging quantum resources effectively. We believe these contributions represent a meaningful step forward in the field and hope this perspective resonates with the broader audience.

Q5. How expensive is it to update the gradient on real hardware?

Response: Similar to our response in Q3, based on the current rates, if the gradient estimation requires several circuit executions within approximately 4 seconds, the estimated cost would be around $3.2 USD per gradient update.

Q6. What compute was used ... classical methods used?

Response: We sincerely appreciate the reviewer’s thoughtful question. We assume the reviewer is inquiring about the hardware used for our experiments. As detailed in Appendix C, all experiments were conducted on NVIDIA V100S and NVIDIA H100 GPUs.

Regarding the total wall clock time, in the revised version, we have included execution time data in Appendix I, “On Computational Time.” Based on the LoRA results from Fig. 2, the average execution time of QPA is approximately three times longer than that of classical LoRA. This means that while QPA completes one epoch, LoRA can complete three epochs.

To evaluate the performance of both methods under comparable computational time constraints, we present results where QPA with 1 epoch is compared to LoRA with 3 epochs for GPT-2, and QPA with 1 and 2 epochs is compared to LoRA with 5 epochs for Gemma-2 (Fig. 11). The findings suggest that, although QPA does not outperform LoRA to the same extent when both methods are trained for the same number of epochs, it still exhibits better performance in regions with smaller parameter sizes under similar total execution times.

[1] Yidong Liao, Chris Ferrie, “GPT on a Quantum Computer”, arXiv preprint arXiv:2403.09418 (2024).

[2] Laia Domingo, G Carlo, and F Borondo. Taking advantage of noise in quantum reservoir computing. Scientific Reports, 13(1):8790, 2023.

[3] Chuhan Wu, et. al. Noisytune: A little noise can help you finetune pretrained language models better. arXiv preprint arXiv:2202.12024, 2022.

[4] https://www.ibm.com/quantum/pricing

[5] https://docs.quantum.ibm.com/guides/estimate-job-run-time

Q1. Does this technique ... with finite measurements.

Response: We appreciate the reviewer’s thoughtful comment and the opportunity to clarify the utility of our technique. While it is true that strong simulation of quantum circuits is required in our current implementation and situation, this does not limit the broader applicability of the proposed approach for several reasons:

1. Addressing Robustness to Noise: In the revised version, we have included an appendix (e.g., “Effects of Finite Measurement Shots and Noise”) where we investigate the performance of our technique under realistic noise models derived from IBM quantum hardware. Interestingly, the results show that even with noise, our method performs comparably in many cases and, in some instances, benefits from quantum noise (similar to findings in other quantum computing studies). This suggests that the technique has potential utility in noisy quantum environments, contrary to concerns about its robustness.

2. Finite Measurements: We have also analyzed the performance of our technique with finite measurement shots. As shown in the newly added results, increasing the number of measurement shots improves performance, eventually converging toward exact simulation results. While practical implementation would require careful consideration of the trade-off between measurement counts and performance, the method remains effective even with reasonable measurement budgets. This demonstrates its potential utility beyond idealized strong simulation.

3. Future Practical Implementation: As quantum hardware continues to improve, issues such as noise and measurement overhead are expected to become less significant. The technique’s reliance on strong simulation in its current form is primarily due to the limitations of available hardware. However, the conceptual framework and potential for generating parameter-efficient representations of PEFT methods remain valid and promising as quantum technology advances.

4. Hybrid Model Synergy: Lastly, even when strong simulation is used, our method shows promise as a hybrid quantum-classical approach. By leveraging quantum circuits to generate low-dimensional parameter representations, the technique provides insights into integrating quantum resources with classical machine learning systems, which may inspire future noise-resilient adaptations.

Q2. Have you tried ... If not, why not?

Response: At this stage, our method has not been implemented on actual quantum hardware for several reasons. Quantum hardware at this stage is primarily shared among a large number of users, resulting in long waiting queues for access. Tasks such as fine-tuning or machine learning, which involve a large number of circuit executions, require significant computational resources and repeated runs. Obtaining results for such tasks within a reasonable timeframe becomes infeasible on current quantum hardware under shared access constraints.

Given these limitations, we focused on simulations using realistic noise models derived from quantum hardware (e.g., IBM’s ibm_torino and ibm_fez). These simulations allow us to evaluate the method’s performance in noisy environments without being constrained by hardware availability or capabilities. The results, as presented in the newly added appendix section, demonstrate the robustness of our method, validating its potential for real-world quantum devices.

Q3. How expensive ... fault-tolerant manner?

Response: We thank the reviewer for the valuable comments. Based on the pricing information provided by a major quantum computing provider like IBM Quantum [4], it is possible to estimate the cost of fine-tuning tasks. For instance, if the premium plan at 48 USD/minute is selected, this translates to approximately 0.8 USD/second. Assuming that a single training step involving a quantum job (including job submission, compilation, execution, and result retrieval) takes around 4 seconds [5], and considering the batch size used in our study with the WikiText-2 dataset, one epoch would require approximately 36,000 steps. The cost of executing three epochs can then be estimated as follows: $0.8 \times 4 \times 36,000 \times 3 = 345,600 \, \text{USD}.$ While this cost is indeed high, as mentioned earlier, it is expected to decrease as quantum hardware improves over time. Despite these current limitations, the conceptual framework and potential for generating parameter-efficient representations remain valid and promising as quantum technology continues to advance.

We sincerely thank Reviewer s1Tx for the valuable suggestions, which have greatly contributed to improving the quality of our work. We deeply appreciate the reviewer’s time and effort in helping us refine and enhance our paper.

W1. Misleading claims about ... which are extremely costly!

Response: We sincerely appreciate the reviewer's thoughtful feedback and apologize for any confusion caused. The corresponding footnote has been updated to: “IBM Quantum announced the achievement of discovering a quantum error-correcting code capable of preserving up to 12 logical qubits using 288 physical qubits through error correction methods,” incorporating the reviewer’s suggestion. While real-world implementation may be challenging, we believe it is still worthwhile to explore the potential of using quantum computing techniques to fine-tune classical LLMs. This approach, which enables inference entirely on classical hardware, differs significantly from methods that aim to reconstruct transformer architectures [1] on quantum circuits. Notably, it reduces reliance on quantum computers during the inference stage.

W2. No serious discussion of ... noiseless setting on certain problems.

Response: We deeply value and appreciate the reviewer’s insightful feedback. In response to the suggestion, we have added an appendix section titled “Effects of Finite Measurement Shots and Noise” in the revised version. In this section, we apply the noise model provided by IBM, derived from real hardware data from ibm_torino and ibm_fez, to investigate the potential behavior of QPA in real-device environments.

Interestingly, the introduction of noise from ibm_torino and ibm_fez results in mixed outcomes: while performance slightly decreases in some cases, it improves in most instances. A similar observation has also been reported in previous quantum computing studies, where quantum noise has been found to enhance performance in specific paradigms [2]. Similarly, in classical machine learning, studies have shown that introducing small amounts of noise can benefit the fine-tuning of LLMs [3]. Since the quantum circuit in QPA generates measurement probabilities that are fed into a classical MLP mapping model to produce PEFT parameters, it is plausible that quantum noise plays a role analogous to the “small noise” effect observed in previous studies, helping to explore more parameter spaces and aiding in the fine-tuning of LLMs.

W3. No theoretical performance ... results, just simulations).

Response: We sincerely thank the reviewer for their valuable feedback and suggestions. While QPA is a newly proposed approach, theoretical analysis is planned as a future direction for further investigation. Although this study primarily focuses on simulation results, in the revised version, we have included results that incorporate a real quantum computer noise model. This addition provides insights into both ideal and more realistic scenarios.

W4. I am not sold ... expensive to train too!

Response: We thank the reviewer for their valuable insights and feedback. While generative models can be challenging to train, an ideal approach to mitigate the issue of finite measurement shots could involve using a pre-trained model. This model would be trained on a general measurement result generation task and then fine-tuned for specific tasks. Nevertheless, as discussed in the appendix, we acknowledge the potential challenges associated with QPA and propose possible approaches to address them. While these suggestions may not represent definitive solutions, they aim to provide valuable insights into problem-solving paradigms for tackling such issues.

W5. It is an open ...or at least soften it. & W6. “This hybrid approach ... this claim.

Response: We appreciate the reviewer’s suggestion regarding the description in the introduction section. In response, the corresponding sentences have been revised and softened to: “Classical systems are well-suited for tasks like data processing, while quantum computing shows potential in optimization and exploring large state spaces,” and “This hybrid approach holds promise in quantum machine learning (QML), with the potential to support the training and fine-tuning of large models.”

Dear Reviewer,

With the rebuttal phase nearing its conclusion, we would like to gently remind you of our responses.

We have thoughtfully addressed all your comments and questions, and we hope our revisions meet your expectations.

If there are any unresolved concerns, we are more than willing to discuss them further. Otherwise, if you find our updates satisfactory, we kindly invite you to consider raising your score.

Thank you again for your time and valuable feedback!