Quantum-PEFT: Ultra parameter-efficient fine-tuning

We thank the reviewer for the valuable feedback and address the concerns below.

Q1. The paper describe the compute ansatz in (2). However, the backward-pass, especially computing the gradient update of the parameters is not covered

Thank you for your comment. The trainable parameter $\theta_{k,i}$ in Eq. (2) is associated with individual RY gate, defined in (1). RY gate is based on trigonometric function wrt $\theta$, and thus closed-form expression of the gradient is known: $\frac{\partial \mathsf{RY}(\theta)}{\partial \theta} = \frac{1}{2} \begin{bmatrix} -\sin(\theta/2) & -\cos(\theta/2) \\\\ \cos(\theta/2) & -\sin(\theta/2) \\\\ \end{bmatrix}$. Eq. (2) is based on Kronecker product of RY gates: $\otimes_k RY(\theta_{k,i})$. Its derivative is straightforward. Specifically, we have $\frac{\partial}{\partial \theta_{m,i}} \bigotimes_{k=1}^q RY(\theta_{k,i})= RY(\theta_{1,i}) \otimes RY(\theta_{2,i}) \otimes \cdots \otimes \frac{\partial RY(\theta_{m,i})}{\partial \theta_{m,i}} \otimes RY(\theta_{m+1,i}) \otimes \cdots$. Correspondingly, the gradient update of quantum parameters is computationally feasible. In experiments, we simply used PyTorch's autograd. We leave further engineering optimization as a future work. Nevertheless, our analysis in Fig. 6 shows that the gradient update of the Pauli parameterization is still competitive even without custom kernels. In the revised manuscript, we added a discussion on how we compute the gradient update.

Q2. A gradient descend on the Factors of the proposed parametrization

Indeed, the convergence of quantum tensor networks and quantum machine learning is not fully understood yet. Nevertheless, there are many empirical literature showing sufficient trends of loss decreasing in many practical cases, e.g., [5]. Remarkably, [4] provides a theoretical proof of sublinear convergence under a reasonable assumption. In our case, we empirically observed no issues in loss convergence and added discussion on the employed gradient method in Sec. 4.1.

Q3. How does your factorized update scheme relates to [1,2,3]?

[2] optimizes the neural network weight matrices with gradient flow along the low rank weight manifold. [3] considers for manifold-constrained optimization in the federated learning setting. [1] proposes rank-adaptivity in CNNs via low-rank Tucker. Quantum-PEFT parameterizes $U, V$ factors of low-rank delta weights through trainable intrinsic parameters instead of directly optimizing the low-rank factors of network weights. This approach differs from [1, 2, 3], which employ manifold optimization to maintain feasibility of the factors. In Quantum-PEFT, the $U,V$ matrices are not directly trainable parameters, but rather computed via efficient Pauli unitary parameterization from underlying smaller learnable parameters. Hence, our optimization happens in the Euclidean space of the parameters of the parameterization. We included discussions wrt [1,2,3] in Sec. 2.

Q4. Is there an option to make the method rank-adaptive?

In our current method, we introduced an intrinsic rank $K'$ as a mechanism to further control the effective number of trainable parameters compared to the specified rank $K$ by keeping the top $K'$ columns of Lie parameters, while the other parameters are frozen or null-out. By choosing $K' < K$, we control the intrinsic dimensionality of the learned subspace, offering a finer-grained control and enabling higher parameter reduction. As demonstrated in our sensitivity analysis for $K'$ (Table 8), reducing $K'$ can significantly diminish the number of parameters while incurring minimal performance degradation. For example, even with $K'$ reduced to 1, the performance drop is only 0.49%, while still maintaining a competitive advantage over LoRA.

While the current framework allows for manual adjustment of $K'$ as shown in the sensitivity analysis, one could devise a fully rank-adaptive method that dynamically adjusts $K'$ during training. A crucial aspect of such an adaptive strategy would be developing a principled approach for ranking the importance of the Lie algebra parameters. In fact, unlike AdaLoRA, where the singular values directly correspond to the importance of each rank component, the relationship is not as straightforward in Quantum-PEFT due to our Pauli parameterization. In future work, it would be interesting to rigorously analyze this relationship and incorporate it into a $K'$-adaptive algorithm.

[1] Rank-adaptive spectral pruning of convolutional layers during training. Neurips 2024. [2] Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations. Neurips 2022. [3] Federated dynamical low-rank training with global loss convergence guarantees, 2024. [4] Analyzing convergence in quantum neural networks: Deviations from neural tangent kernels. ICML 2023. [5] Improving gradient methods via coordinate transformations: Applications to quantum machine learning. Physical Review Research.

Dear Reviewer,

The authors have provided their rebuttal to your comments/questions. Given that we are not far from the end of author-reviewer discussions, it will be very helpful if you can take a look at their rebuttal and provide any further comments. Even if you do not have further comments, please also confirm that you have read the rebuttal. Thanks!

Best wishes, AC

I thank the authors for their clarification. I decide to keep my score and raise my confidence score.

We thank the reviewer for the valuable feedback.

Q1. Delineation of learnable parameters

We have carefully revised Section 4.1 to more clearly delineate the trainable parameters. In Quantum-PEFT, we use parameter-efficient representation of unitary transformations, i.e., the trainable parameters are Lie algebra for unitary parameterization. Specifically, in Eq. (2), the trainable parameters are the rotation angles $\theta_{k,l}$ associated with the employed RY gates. In fact, in (2) the Lie algebra, i.e., $\theta_{k,l}$, for each RY gates is trainable. The RY gate has a single parameter, as in (1), and thus the total number of trainable parameters for (2) is $(2L+1)\log_2(N)-2L$. This logarithmic scaling is a key advantage of our approach.

As an example, for 5-qubit case with $N=2^5$, we illustrated it in Fig. 2(a): the left-most 5 RY gates in Fig. 2(a) correspond to the right-most RY term in (2); the middle 4 RY gates in Fig. 2 correspond to the middle term in (2); the last 4 RY gates in Fig. 2 correspond to the first RY term in (2). For more general cases of $N'>2$ in Fig. 2(b), the lie algebra, i.e., strictly lower triangular matrix $B$, has $K'N'-K'(K'+1)/2$ parameters per generalized RY gate as in Fig. 3(a). Besides orthogonal nodes, diagonal node has additional trainable parameters of size $K$ as in Fig. 3(b).

Q2. Computational complexity analysis

For the Pauli parameterization $Q_\mathsf{P}$, we obtain better-than-LoRA parameter scaling with runtime $O[(L+1) N\log_2(N)]$, using the efficient Kronecker Shuffle algorithm (Plateau, 1985), where in the experiments we used $L=1$. LoRA has runtime $O(2NK)$. Detailed analysis of computational complexities is given in Computational complexity paragraph in Sec. 4.2, where we have emphasized that our parameterizations remain within the same complexity as existing methods. In summary, while the constant factors may slightly differ, this demonstrates that the overall complexity remains highly competitive with LoRA even with the mapping.

Wall-clock time is measured in Table 4 and reported below for reference, where we compare Quantum-PEFT against baselines when fine-tuning GPT2.

Table og4k-1: Efficiency comparison on GPT2.

We find that all methods have very similar runtime; notably, we observe a 4.03x reduction in number of parameters with Quantum-PEFT and overall better performance than LoRA with 4x fewer parameters (see Table 3 in manuscript). Our implementation is based on Kronecker product operation and optimizations with custom CUDA kernels are possible. We made a rigorous runtime analysis for different unitary parameterizations in Fig. 6, which shows Pauli can be faster than other parameterizations at large sizes.

Q3. Fig. 5 placement

We agree that Fig. 5 gives an intuitive illustration of the core concepts in Quantum-PEFT to benefit a broader audience. It illustrates the structure of the quantum circuit from its fundamental building quantum gates as unitary matrix transformations from small unitary matrices. Therefore, we have moved it to the main body as Fig. 2 for clearer understanding of Eq. (2) for all readers. We have also revised Introduction and Preliminaries sections with more detailed and clearer self-contained explanation of the necessary background to improve accessibility to researchers from different backgrounds.

Q4. A dedicated notation section would significantly improve readability and accessibility

We have substantially expanded the notation paragraph at the beginning of Sec. 3 based on your feedback. We have also added a table defining all symbols used in the paper (see Appendix A.7), serving as a quick reference for the reader. We believe these enhancements significantly improve the accessibility and readability of the presentation.

Q5. What are the practical limitations and deployment considerations?

Quantum-PEFT offers significant advantages in parameter efficiency for fine-tuning, achieving significant parameter reduction compared to LoRA. In implementation, one needs to consider: i) redundant memory to generate full-rank unitary matrix; ii) algorithmic performance for unitary mapping, and iii) numerical error at low-precision. In our work, we provided solutions for i) by using tensor contraction ordering (Pfeifer et al.), ensuring memory efficiency and avoiding redundant storage of full unitary matrices, and ii) by Kronecker Shuffle algorithm (Plateau), performing a detailed experiment for the required complexity and unitarity accuracy in Fig. 6, showing that the Pauli parameterization offers competitive speed at little unitarity error compared to other unitary parametrizations. Regarding iii) w.r.t. the impact with low-precision quantization, we discussed quantization-aware training in Appendix A.5 and Table 7, showing promising results even with 1-bit quantization.

Dear Reviewer,

The authors have provided their rebuttal to your comments/questions. Given that we are not far from the end of author-reviewer discussions, it will be very helpful if you can take a look at their rebuttal and provide any further comments. Even if you do not have further comments, please also confirm that you have read the rebuttal. Thanks!

Best wishes, AC

We thank the reviewer for the insightful feedback and we address the remaining concerns below.

Q1. Analysis of how L layers affect entanglement capacity.

Thank you for the comment. There is existing literature on the relationship between circuit depth and entanglement, e.g., [7] provided a rigorous analysis of entangling capacity with different quantum circuts. It was shown that generally quantum circuits exhibit increased capacity with more layers. In our experiments, a single entanglement layer ($L=1$) in the simplified two-design ansatz showed good performance for PEFT tasks, while increased layers can moderately improve the performance at the cost of increased parameters and computational overhead. We have included the discussion of the entangling layers in the revised manuscript (highlighted in blue) accordingly in Appendix A.4.

Furthermore, we conducted a sensitivity analysis w.r.t. the number of entanglement layers $L$ in Table XPHL-1, showing the accuracy of the ViT CIFAR10 transfer learning task of Section 5.4 with a 2-bit quantized base ViT model, across different values of $L$. We observe a saturation effect, where the performance gains plateau beyond $L=3$. It is reasonable that the optimal $L$ is task-dependent, with more complex tasks potentially benefiting from a deeper circuit. We have included these findings in the revised manuscript in Table 9.

Table XPHL-1: Accuracy over the number of entanglement layers $L$ in ViT CIFAR10 task. Base ViT is 2-bit quantized.

Q2. Benchmarks on larger and newer models.

Following the reviewer’s suggestion, we have conducted additional experiments by fine-tuning Mistral-7B [5] on the standard GLUE benchmark. Mistral 7B is a recent LLM that has been shown to outperform Llama 2-13B on many benchmarks [5]. We provide empirical comparisons below, where we used the same rank and alpha as in Section 5.1 and employed AdamW with 5 epochs and 4-bit quantization, and fine-tune query/key/value/gate projection matrices, following the setups of [6], for all methods. The results are presented in Table XPHL-2 with LoRA and AdaLoRA, which are the strongest baselines on GLUE in Section 5.1. Quantum-PEFT maintains its significant advantage in parameter efficiency even at 7B scale while achieving competitive performance with LoRA and AdaLoRA. We observed a reduction in trainable parameters by a factor of 4.67x, while even exceeding the performance of LoRA and AdaLoRA in almost all datasets. Quantum-PEFT offers a highly parameter-efficient alternative to LoRA-based approaches, which is crucial when scaling to billion-scale large language models. We have included these new experiments in Section 5.3 of the revised manuscript.

Table XPHL-2: Results with Mistral-7B on GLUE Benchmark.

Q4. How does Eq. 4 work?

Eq. 4 demonstrates recursive cosine-sine decomposition (CSD) to construct a unitary matrix of arbitrary size by using multiple unitary matrices of smaller size. Tensor decomposition used in, e.g., QuanTA [4], is not readily compatible to arbitrary size of tensor. CSD overcomes this restriction. For example with $N=257=2^8+2^0$, CSD gives a way to use unitary representation with $N_1=256=2^8$ (i.e., 8-qubit) and $N_2=2^0$ (i.e., 0-qubit). 8-qubit matrices $U_1, V_2\in\mathbb{R}^{256\times 256}$ are easily parameterized by tensor decomposition, while 0-qubit special unitary is $U_2=V_1=1$. Eq. 4 becomes

Q3. Comparative analysis with recent works [1,2,3,4].

We first provide discussions and then present empirical comparisons.

• Quantum-PEFT is a new technique for highly efficient reparameterization-based fine-tuning . [1] is orthogonal to LoRA-based techniques: it finetunes some important sampled layers while freezing the rest for some certain iterations. Their results show that [1] shares similar memory requirements as LoRA, while our method achieves significantly higher parameter efficiency than LoRA.
• [2] intervenes on a low-rank subspace of the intermediate model embeddings rather than model weights. This creates additional hyperparameters, i.e., prefix and suffix positions to intervene on and which layers to intervene on, creating a combinatorial growth of hyperparameters. Therefore, their claimed higher efficiency comes at the cost of higher computational burden of hyperparameter tuning for each task. On the other hand, Quantum-PEFT mainly considers two hyperparameters, i.e., the intrinsic rank $K'$ and the number of entanglement layers $L$. We conduct a sensitivity analysis for $K'$ in Table 8, where it is shown that reducing $K'$ incurs in minimal performance degradation. For example, even with $K'$ reduced to 1, the performance drop is only 0.49%, while still maintaining a competitive advantage over LoRA. Additionally, unlike methods like Quantum-PEFT learning delta weights directly, their updates cannot be merged with the model weights, so their method incurs in computational overhead at inference time.
• [3] employs a learnable square matrix $M \in \mathbb{R}^{\hat{K} \times \hat{K}}$ for full-rank fine-tuning and compatibility mappings to ensure that the dimensions match with the weight $W \in \mathbb{R}^{N \times M}$. They set $\hat{K}=\lfloor \sqrt{(N+M)K} \rfloor$ to achieve the highest rank with square matrix at the same total number of trainable parameters of LoRA with rank $K$. [3] is therefore not able to have a lower-than-LoRA scaling, which is instead achieved by Quantum-PEFT.
• [4] is a tensor-network-inspired parameterization without consideration of unitary gain, so their parameterization leads to parameter redundancy and potential instability. Our Pauli parameterization under Lie algebra can strictly maintain unitary constraint without parameter redundancy. [4] is based on tensor folding, requiring in principle that the matrix size is factorizable as $d=d_1\times d_2 \times \cdots \times d_N$. Therefore, [4] is not readily compatible for arbitrary size. For example, if the matrix size is $d=257$, it is difficult to fold into multiple axis. We clearly provided the way to solve this issue by quantum Shannon decomposition, which enables to decompose into sum of powers-of-two: i.e., $N_1=256$ and $N_2=1$. Furthermore, our paper provides more general applicability and insight to any arbitrary tensor network to reduce the parameter number as shown in Figs. 5, 7, and provides more flexibility with adjustable entangling layer size, which is not explored in [4].

We have included the above works in the Related Work section in main body and added detailed comparative discussions in Appendix A.6.

To address the reviewer's concern, we have conducted additional experimental comparisons with the very recent MORA [3] and QuanTA [4, to appear in NeurIPS 2024], which are comparable to ours learning in the low-rank subspace of model weights. We follow the setups of Table 2 in the paper and use rank 3 for MORA as for the other methods. W.r.t. QuanTA [4], using their optimal setting $d=768=2^8*3$ to give the lowest number of parameters, QuanTA [4] still has 0.093M trainable parameters compared to 0.013M parameters for our method. We report the results in Table XPHL-3. We observe that Quantum-PEFT outperforms both MORA and QuanTA while using 37.87x and 7.15x less parameters, respectively.

Table XPHL-3: Comparisons on GLUE with DeBERTaV3 including the very recent related works [3,4].

Thanks for the detailed rebuttal. The authors answered most of my questions. I'm still curious about the authors' choice of benchmarking on Mistral and DeBERTa but not LLaMA against the new references. Only [1] included benchmarks on Mistral but all [1] to [4] included benchmarks on LLaMA. None of the references included or made any claims about DeBERTa. It can be difficult to compare different works based on different benchmarks, and it is unclear from the provided benchmarks how Quantum-PEFT compares to these new methods on NLG tasks. Understandably, benchmarking fine-tuning methods is difficult and time-consuming. I'd be happy to raise the score if the authors could (qualitatively) explain how the benchmarks on Mistral and DeBERTa could translate to the benchmarks on LLaMA as shown in the references in order to better gauge the results, or if the authors could provide more theoretical reasoning on Quantum-PEFT's advantage over these methods.

Dear Reviewer,

The authors have provided their rebuttal to your comments/questions. Given that we are not far from the end of author-reviewer discussions, it will be very helpful if you can take a look at their rebuttal and provide any further comments. Even if you do not have further comments, please also confirm that you have read the rebuttal. Thanks!

Best wishes, AC

We thank the reviewer for the valuable feedback and address all remaining concerns below.

Q1. It would be great if authors could add some self-contained introduction to the necessary quantum concepts used in the paper.

While Quantum-PEFT draws inspiration from quantum computation principles, the proposed parameterization can also be purely understood within the context of classical linear algebra. In fact, quantum circuits represent unitary matrices as a product of small unitary matrices (i.e., quantum gates) with a total number of parameters growing logarithmically with the dimension. To further enhance clarity for readers less familiar with quantum computing concepts, in the updated manuscript (highlighted in blue), we have revised the Introduction and the Preliminaries sections with more detailed and clearer self-contained explanation of the necessary background. This includes an explanation of Pauli operators as matrices and the concept of quantum circuits as matrix transformations. We believe these explanations provide a more accessible entry point for a broader audience, including those unfamiliar with quantum machine learning. Specifically, the Preliminaries section includes a concise explanation of Pauli matrices, quantum gates (RY and CZ), and quantum circuits as transformations with unitary matrices. Furthermore, we have added Fig. 2 to the main text (previously in the appendix) to provide a visual aid for understanding the quantum-inspired PEFT modules. This figure illustrates the structure of the quantum circuit from its fundamental building gates, i.e., small unitary matrices. We have also improved the notations by improving the paragraph summarizing all notation used in the paper (beginning of Section 3) and added a table of symbols in Appendix A.7.

Dear Reviewer,

The authors have provided their rebuttal to your comments/questions. Given that we are not far from the end of author-reviewer discussions, it will be very helpful if you can take a look at their rebuttal and provide any further comments. Even if you do not have further comments, please also confirm that you have read the rebuttal. Thanks!

Best wishes, AC