Thank your for your valuable feedback. Following are the clarifications on your comments:

Mathematical notation

Thank you for your advice on improving the mathematical notation. We have revised the notation to make it more consistent and clear, which is reflected in equations (5) and (6) in the revised manuscript.

• However, we retain the use of $\phi$ and $\psi$ to denote the learnable parameters, as they are a part of the parameters of the weight-generator network. This lets us distinguish between standard fine-tuning methods and GIFT. Our GIFT generates the fine-tuned weights from the pre-trained weights, rather than learning them as model parameters.
• This also keeps the notation consistent across equations (5) and (6).

Content redundancy

We have rephrased the Introduction (Section 1) and Approach (Section 2) to be more precise and concise. Please help skim those to see if we address your concerns.

Clarification on statements

Thank you for your carefully reading through our submission. Those statements highlighted by you are indeed at a high level, indended to convey intuition behind GIFT. We have carefully rewritten some statements in the revised manuscript to be more precise and clear.

Line 051: but the learnable weight-residuals do not have direct information exchange with the pre-trained weights

Line 135: one of the simplest updates that minimally distorts and maximally preserves the pre-trained knowledge is defined by Eqn.1 and Eqn.2, thanks to the low-rank factorized linear projection in the parameter space.

Line 181: Additionally, adding fixed random matrices with learnable scales in PEFT makes the relationship between fine-tuned models and frozen pretrained models less intuitive

We have replaced these statements to convey the message more clearly in Section 2.3 in the revised manuscript. We reiterate the modified statements from Section 2.3 here for convenience:

"Pretrained Transformer backbones encode diverse knowledge from large-scale pretraining datasets within their weights. Fine-tuning them for a downstream task aims to incorporate new information from the task-specific training data and utilize the information present in the pretrained weights to the fullest extent. To achieve this, the fine-tuned weights can be directly conditioned on the pretrained weights, such that the new information is learned conditionally from the information in the pretrained weights. While LoRA and it's variants use a residual structure to address this, the residual weights are not directly conditioned on the pretrained weights, but rather learned via back-propagation (chain rule) updates. One of the simplest functions that can achieve this explicit conditioning is a linear transformation of the pretrained weights, as leveraged in Eqn. 7. Hence, the fine-tuned weights can also be expressed in the space of the pretrained weights $W_{d_{out}\times d_{in}}$ via $W_{d_{out}\times d_{in}}\cdot \Theta_{d_{in}\times d_{in}}$."

Line 194: "Furthermore, token-level interventions lack a holistic understanding of the relationship between ReFTed models and frozen pretrained models."

The idea behind the statement is as follows: Although ReFT learns precise interventions for representations at token level, it is applied to fixed token position (prefix and suffix tokens), without regard to the actual token sequence. With GIFT's representation tuning perspective, the interventions can be more flexible since the tuning is applied to all the tokens. We have removed this statement, and instead rephrased the connection between GIFT and ReFT in Section 2.2 of the revised manuscript to convey the message more clearly.

Question 1

Section 2.3 in the original manuscript describes the gradient of the learnable parameters in GIFT. The accumulation refers to the accumulation of the gradient across layers which occurs due to layerwise sharing of the learnable parameters. As this section is not critical to the understanding of the method, we have moved it to the appendix (Section B) in the revised version.

Question 2

We have modified the description (Section 3.5) to be be more clear. We would like to point out that the segmentation maps are formed by projecting the output of the final projection layer in the MHSA block using the learned GIFT (equations 10 and 11 in the revised manuscript), and not the attention matrix.

Dear Reviewer 53Qz,

Hope all is well with you.

We would like to request your feedback on our rebuttal, as well as the revised manuscript at your convenience.

We look forward to it.

Thank you very much.

Dear Reviewer 53Qz,

Thank you for pointing out this great work (MEND, Model Editor Networks with gradient Decomposition). We will discuss it in our revision.

We would like to share our understanding with you in terms of the difference between the MEND method and our GIFT.

Summary of MEND and Its Notations

To edit layer $l$ with pretrained weights $W_l$, layer activation $u_l$ and output gradients $\delta_{l+1}$ are concatenate, $z_l=\text{Concat}(u_l, \delta_{l+1})$, as input, the editor network $g$ that is parameterized as an MLP with low-rank weight matrices, $(U_1, V_1, U_2, V_2)$, residual connections and a single hidden layer with activation function $\sigma()$, and can be shared across layers with layer-specific scale $s_l$ and offset $o_l$,

$g(z_l) = h_l + \sigma(s^2_l \odot U_2 V_2 h_l + o^2_l)$ (Eqn.3b in MEND)

$h_l = z_l + \sigma(s^1_l \odot (U_1 V_1 z_l +b) + o^1_l)$ (Eqn.3a in MEND)

The output of $g(z_l)$ is split into pseudoactivations $\tilde{u}_l$ and

pseudodeltas $\tilde{\delta}_{l+1}$, the model edit for weight matrix $W_l$ is defined by,

$\tilde{\nabla}_{W_l} =$

$\sum_{i=1}^B {\tilde{\delta}^i_{l+1}} {\tilde{u}^i_l}^{\top}$ (We split the equations in two lines due to Markdown issues)

The final edited weights are

$\tilde{W}_l =$

$W_l - \alpha \tilde{\nabla}_{W_l}$

Notations in our GIFT (e.g. Eqns. 4 and 5)

$\hat{W}^l_{d_{out}\times d_{in}} = W^l_{d_{out}\times d_{in}} + \mathcal{G}(W^l_{d_{out}\times d_{in}}; \Theta)$

$\qquad \qquad =W^l_{d_{out}\times d_{in}} +$ $W^l_{d_{out}\times d_{in}} \cdot \phi_{d_{in}\times r}\cdot \psi_{r\times d_{in}}$

We use $W^l_{d_{out}\times d_{in}}$ to denote the pretrained weights, rather than $W_l$ used in MEND, since we want to explicitly show the dimensions of the weight matrix in the subscript.

We use $\hat{W}$ to represent the final weights, rather than $\tilde{W}$ used in MEND.

We use $\phi_{d_{in}\times r}$ and $\psi_{r\times d_{in}}$ to denote the parameters of our GIFT weight-generator, rather than $U, V$ used in MEND.

Overall, we think the notations used in our GIFT is self-contained and clear. We would like to hear your specific suggestions in terms of notation changes.

How is the comparison result of GIFT among some newly advanced models like gpt4o or others

We would like to clarify that we aim for fair comparisons in experiments following protocols used in the state-of-the-art PEFT/ReFT methods. Table 1 is meant to compare with other PEFT/ReFT methods that use the same model (Llama 2 7B) and finetuning dataset (Ultrafeedback). A fair and valid evaluation requires using the same models and datasets as prior methods. As prior methods have demonstrated performance using Llama 2 7B, we have chosen to compare using the same model, and hence we believe that our comparisons are valid and meaningful. We also emphasize that using the same model and training data, none of the prior methods outperform GPT 3.5 Turbo.

Thank you for your valuable feedback. Below, we provide clarifications on the points raised:

Lack of some comparison settings on full finetuning

We acknowledge the value of such comparisons; however, due to resource limitations, we were unable to conduct experiments involving full fine-tuning of models. Please note that none of the prior works we compare against perform full fine-tuning. We have included the full fine-tuning baseline for instruction-following tasks where prior works have made these results available.

Potential Scalability Concerns

• We have added experiments with LLaMA-1 13B for Commonsense Reasoning and Arithmetic Reasoning tasks. Table 2 and Table 3 in the revised manuscript show that GIFT performs better/on par with prior PEFT and ReFT methods while being significantly more parameter efficient. This shows that GIFT can be scaled to larger models on the scale of 13B, and can potentially be scaled to even larger models. We have summarized the results here for convenience.

Commonsense Reasoning (LLaMA-1 13B)

Arithmetic Reasoning (LLaMA-1 13B)

• While scalability is an important consideration, we are unable to run experiments on models larger than 13B due to resource limitations as an academic lab. We note that we have submitted the code in the supplementary material to promote follow up work on larger scale experiments.

Limited Ablation on Layer Selection and Configuration

• In all our experiments, we perform a fair comparison with PEFT methods by fine-tuning the same layers with different methods. We follow [1] (cited as (Hu et al., 2023) in the manuscript) in choosing the layers to finetune in the Commonsense Reasoning and Arithmetic Reasoning tasks (QKVUD), and [2] (cited as (Wu et al. 2024a) in the manuscript) in choosing the layers to finetune for the Instruction Following task. We believe that this is a fair comparison as we are using the same layers as prior works.
• For a fair comparison with LoReFT and DiReFT, we evaluate $\text{GIFT}_{\overline{O},\overline{D}}$ formulation of GIFT. We beleive these experiments are sufficient to demonstrate the effectiveness of GIFT.
• We further evaluate a block-wise sharing scheme $^B\text{GIFT}^{16}_{\underline{QKV},\overline{O},\underline{UG},\overline{D}}$, which is unique to GIFT to demonstrate the flexibility of the sharing scheme enabled by GIFT.

References

[1] Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, Roy Ka-Wei Lee: LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models. EMNLP 2023: 5254-5276

[2] Muling Wu, Wenhao Liu, Xiaohua Wang, Tianlong Li, Changze Lv, Zixuan Ling, Jianhao Zhu, Cenyuan Zhang, Xiaoqing Zheng, Xuanjing Huang: Advancing Parameter Efficiency in Fine-tuning via Representation Editing. ACL (1) 2024: 13445-13464

Dear Reviewer k2rS,

Hope all is well with you.

We would like to request your feedback on our rebuttal, as well as the revised manuscript at your convenience.

We look forward to it.

Thank you very much.

Thank you for your valuable feedback. We address your comments as follows:

Sharing half of the lora weights across layers

• With all due respect, we would like to clarify that there is a key distinction between our proposed approach and your interpretation that our GIFT learns layer specific matrices $A$ and shared matrix $B^\prime$ as $\hat{\omega} = \omega + \omega \cdot A\cdot B^\prime$.
• This is incorrect: GIFT shares all the learnable parameters across layers. We have revised the writing to make the formulation more clear.
• GIFT is formulated as $\hat{\omega}^l = \omega^l + \omega^l \cdot \phi \cdot \psi$, where both $\phi$ and $\psi$ are shared across layers (e.g., all query layers selected in fine-tuning).

Involving the original weight as an extra term into the weight delta

The introduction of the pretrained weights in the delta term is not arbitrary, but a consequence of the generative approach of GIFT in generating the fine-tuned weights from the pretraiend weights. We have revised Section 2 to better formulate this. To briefly summarize, GIFT aims to learn the fine-tuned weights as $\hat{W}^l_{d_{out}\times d_{in}} = W^l_{d_{out}\times d_{in}} + \mathcal{G}(W^l_{d_{out}\times d_{in}}; \Theta)$. $\mathcal{G}$ generates the residuals as

$\mathcal{G}(\Omega_{L\times d_{out}\times d_{in}};\Theta) = \text{Linear}\Biggl( g\biggl(\text{Linear}\bigl(\Omega_{L\times d_{out}\times d_{in}};\phi\bigr);\theta\biggr);\psi \Biggr)$

where $\Omega_{L\times d_{out}\times d_{in}} = \{W^l_{d_{out} \times d_{in}}\}_{\ \ \forall l \in L}$. As per our hypothesis, $g(\omega^l; \theta)$ can be a simple identity function (verified in Section 4.2 of the revised manuscript). Hence, the formulation of GIFT becomes

$\hat{\omega}^l = \omega^l_{d_{out}\times d_{in}} + \omega^l_{d_{out}\times d_{in}} \cdot \phi_{d_{in}\times r}\cdot \psi_{r\times d_{in}}$

Comparison with Tied LoRA

We have added an ablation study to verify that the generative approach in GIFT with shared learnable parameters is more effective than simply sharing the LoRA weight residuals (i.e., $\Delta W^l = B \cdot A\ \ \forall l \in L$, denoted by Shared LoRA). Table 5 in the revised manuscript shows that GIFT performs much better than Shared LoRA on Commonsense Reasoning using the Llama-3 (8B). This suggests that Tied LoRA, which further ties the residuals for Query, Key and Value together, may not be effective, as it shares the residuals directly across layers (we found it is non-trivial to reproduce Tied LoRA in the HuggingFace's PEFT framework that we use in our experiments, and thus did not directly compare with it in ablation due to time limit). We have reproduced the results of our ablations here for convenience.

Content organization can be better

We have organized the content in the revised manuscript to be more precise and concise. We have rephrased the introduction and organized the content to be more straightforward. We have also modified the notation to make it more consistent and clear. Please refer to the revised manuscript and the global comment for the changes made.

Benefit of the proposed changes

The proposed changes have several advantages:

• The layer-wise parameter sharing in GIFT leads to a 14x decrease in the number of trainable parameters compared to LoRA, while outperforming prior PEFT and ReFT methods, as shown through multiple experiments on Commonsense Reasoning, Arithmetic Reasoning, Instruction Following, and Visual Recognition in Section 3.
• Although VeRA can significantly reduce the number of learnable parameters, the rank of the random matrices needs to be sufficiently high for achieving good performance, which leads to a significant increase in memory consumption and training time in practice (as observed in our experiments). In contrast, GIFT maintains the computational efficiency of LoRA while achieving better performance. For e.g., VeRA takes about 1.5 days in training, while our GIFT takes about 4 hours on the arithmetic reasoning benchmark. The wall time required to train VeRA and GIFT is included in Sections 3.3 and 3.4.
• The linear formulation of GIFT bridges the gap between PEFT and ReFT (please refer to Section 2.1).
• The generative approach proposed in GIFT is more holistic in terms of parameter sharing across layers. As opposed to simply sharing the LoRA weight residuals, GIFT generates the residuals in a layer specific way while sharing the trainable parameters across layers. In Section 4.1 of the revised manuscript, we include an ablation study that verifies that the generative approach in GIFT is more effective than simply sharing the LoRA weight residuals.

Backbone model for visual recognition experiments

We use the ViT-B/16 architecture pretrained on Imagenet21k for the visual experiments. This information is provided in the Models section of the Visual Recognition experiments.

Dear Reviewer Rg2D,

Hope all is well with you.

We would like to request your feedback on our rebuttal, as well as the revised manuscript at your convenience.

We look forward to it.

Thank you very much.

Dear Reviewer Rg2D,

We appreciate your valuable time and efforts.

We hope we have addressed your concerns in the initial review and your follow-up questions on the inductive biases. We also respect your decision.

We would like address your latest concerns as follows.

The modeling process of GIFT appears to be as follows (hopefully I have no more misunderstandings):

Every submission may have an exploration journey under the hood. We would like to share some of our efforts to clarify things. When we developed our GIFT, we start with computer vision tasks, and with more sophisticated realization of the $g(\cdot; \Theta)$ in Eqn.6. We show the ablations studies in Sec.4.2 and Table 6. We obtained very promising performance, but then challenged ourselves to seek simpler formulation, resulting in the simple GIFT in this submission. After we observed that the simple GIFT works well, we start to explore its relationships to other PEFT and later on ReFT.

It is our strong believe that finding a simple formulation that works well in practice worths revealing to the community. Of course, we all have future work to do to make our methods better and stronger.

We would like to point out a typo in your suggestion: In LoRA, we have \Delta=BA, not \Delta =AB; Following your specification of \Delta=AB to \Delta = ABW, which does not hold, since $B\in \mathbb{R}^{r\times d_{in}}$, $A\in \mathbb{R}^{d_{out}\times r}$, and $W\in \mathbb{R}^{d_{out}\times d_{in}}$.

If we understand your intent correctly, we have applied our GIFT to the $d_{dout}$ dimension too (Eqn.8), and propose a block-wise sharing configuration: $^{B}$ GIFT $^{r}_{\underline{QKV},\overline{O},\underline{UG},\overline{D}}$ ,

which consists of GIFTs applied to both $d_{in}$ (underline) and $d_{out}$ (upperline) dimensions, and shows stronger consistency of achieving better results across tasks (Commonsense Reasoning and Arithmetic Reasoning), as we highlighted in the global response.