$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning

We would like to thank the reviewer for valuable feedback and comments on our paper. We appreciate the opportunity to address your concerns and clarify any misunderstandings. Below, we provide detailed responses to each of your comments.

The application scope of the proposed method appears limited due to the added constraints on training PEFT components: it necessitates training these components with an additional discriminator. This requirement is uncommon and incurs extra costs.

Thank you for this suggestion, we are happy to elaborate on the discriminator overhead and would include this detail in the final version of the paper. Discriminator training typically takes just 1 epoch to reach over 90% accuracy. LoRAs were trained for 20 epochs in our experiments which was empirically observed to produce best performance for the source LoRA models. Given that both used equal amounts of samples per epoch (half real half synthetic for discriminator training), the cost of training discriminator is only around 1/20 of training cost of LoRA modules. Synthetic data generation for training discriminators took less than 5 minutes for most tasks on a single V100 (for all base models). These costs are almost negligible compared against training the original LoRA.

Further discussion on scalability is needed. As the number of PEFT components grows, updating all components seems time-consuming. By contrast, the approach in [1] suggests keeping all PEFT components static while only updating the base model in a specific way.

We appreciate your suggestion. We are glad to cite Taca [1] in the related work, it is similar to our work in terms of the usage of distillation. However, Taca is in fact targetting one (very general) type of downstream task when being trained - the task of Vision-and-Language modeling - making image encoder tokens “readable” by an LLM decoder. If Taca is to be applied in our setting, it also needs to be trained separately for each downstream task. In this sense, our work is as scalable as Taca. Addressing the reviewer's concern on scalability, although each PEFT module needs to be transferred separately, these transfers can be executed independently in parallel to greatly speed up the process.

Additional properties of the synthetic data should be considered. While the proposed method focuses on in-distribution generation, the diversity of the synthetic data is also crucial.

Thank you for pointing this out. As shown in an example T-SNE plot for the us_foreign_policy task (part of MMLU), the distributions of synthetic data and real data are just as diverse. They have similar coverages and characteristics as is apparent from the T-SNE plot. We will include this plot in our final paper.

More baselines of data-free knowledge distillation are needed, such as [2].

Thank you for providing this reference! In fact, the baseline of data-free knowledge distillation is equivalent to the "unfiltered synthetic data" column in our ablation (Table 5), where samples are generated by simple prompting. We fixed our synthesis prompt to a simple prompt throughout all of our experiments for consistency and simplicity. Our primary contribution is the methodology allowing the use of synthetic data for lossless LoRA transfer. We believe that our approach’s ability to obtain consistent positive transfer results in all experiments using just the simple prompts underline its effectiveness. We also believe that other synthetic generation methods such as [2] are orthogonal to our approach. Any good data synthesis method can be applied together with our proposed Trans-LoRA approach. Combining these methods is a promising research direction for future work. We are happy to cite this work and include this discussion in the final paper.

Given that the proposed method requires a small set of descriptive data, it may be more accurately described as "data-efficient" rather than "data-free."

We thank the reviewer for noting this, and we fully agree. We actually used the term “nearly data-free” in all 8 descriptions in our main paper except only 1 place in line 46. We will fix the reference on line 46 in our final version. Thanks!

Dear Reviewer 7oKi,

The authors have provided a rebuttal. Can you please provide your feedback after reading the rebuttal? The deadline is approaching fast.

Thanks, AC

We would like to thank the reviewer for valuable feedback and comments on our paper. We appreciate the opportunity to address your concerns and clarify any misunderstandings. Below, we provide detailed responses to each of your comments.

While the paper demonstrates the effectiveness of Trans-LoRA on LLaMA and Gemma models, evaluations on a wider range of models and tasks would strengthen its generalizability.

We thank the reviewer for this suggestion. To better illustrate the generality of our Trans-LoRA approach we include the following experiment on the popular T5 model with Coqa, Newsroom, and Squadv2 datasets. The results included in the table below illustrate that our approach can be also effectively applied to more tasks and models. We will include this result in the final manuscript.

The reliance on a discriminator introduces an overhead, and the paper does not fully address who will bear this cost in a cloud scenario.

Thank you for this suggestion, we are happy to elaborate on the discriminator training overhead and would include this detail in the final version of the paper. Discriminator training typically takes just 1 epoch to reach over 90% accuracy. LoRAs were trained for 20 epochs in our experiments which was empirically observed to produce best performance for the source LoRA models. Given that both used equal amounts of samples per epoch (half real half synthetic for discriminator training), the cost of training discriminator is only around 1/20 of training cost of LoRA modules. Synthetic data generation for training discriminators took less than 5 minutes for most tasks on a single V100 (for all base models). These costs are almost negligible compared to training the original LoRAs.

The paper relies on synthetic data generation, which can introduce biases and limitations. A more detailed discussion on these aspects would strengthen the paper.

Thanks for this suggestion! However, we have already provided a detailed MMD analysis of the synthetic data distribution in section 5.1, also ablating how the discriminator-based filtering improves on the MMD with the real data distribution. Moreover, we also highlighted some potential failures of synthetic data generation and even provided a potential mitigation addressing such failures in Section 5.2. We will further emphasize the importance of these aspects refering to these sections from the method section.

Exploring the potential integration of other synthetic data generation methods like ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback and GOLD (Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation) could enhance the data quality and effectiveness.

Our primary contribution is proposing a methodology (Trans-LoRA) that demonstrates the possibility of using synthetic data for lossless LoRA transfer. Currently our experiments used a simple and straightforward prompting approach, while still demonstrating lossless and even positive transfer in all cases, further highlighting the effectivity of our Trans-LoRA approach. We believe that other synthetic generation methods, such as the great methods referenced by the reviewer, are orthogonal to our approach. Any good generation methods can be applied together with our Trans-LoRA, which is a promising direction left for future work to explore. We will add this to the discussion section of the final version of the paper, also citing the works mentioned by the reviewer (that are certainly applicable for this orthogonal future work extension).

Although the synthetic data generation process and discriminator filtering are designed to minimize risk, the potential for synthetic data to inadvertently reflect proprietary information remains a concern. Further safeguards and monitoring practices should be discussed to ensure data privacy.

We performed further analysis on the us_foreign_policy task under MMLU. We find the closest pair of questions from the real data and our synthesized data under embedding space of a pretrained MPNet. This closest pair has a Euclidean distance of 0.604 (so there is absolutely no overlap between synthetic samples and real samples) and consists of: “What were the implications of the Cold War for American exceptionalism?” (real) and “What was the significance of the Cold War to the development of American foreign policy?” (synthesized). Note that these questions are asking for completely different aspects of the subject. We also exhibit the T-SNE plot on the embeddings in the shared response PDF. Although the distributions of synthetic data and real data are similar, they do not share any identical points. We will add this analysis and discussion to the final version of the paper, thanks for suggesting!

We thank the reviewer for their continued support and for raising the score! We will make sure to incorporate all the points raised in the above discussion into the final version of the paper.

We would like to thank the reviewer for valuable feedback and comments on our paper. We appreciate the opportunity to address your concerns and clarify any misunderstandings. Below, we provide detailed responses to each of your comments.

I understand that the current trend is to apply PEFT on decoder-only LLMs like LLaMa, but I'm a bit worried that the method might not be generic enough. I think the method might be suitable to alternative model architectures, for example - Encoder-only LMs like DeBERTa or RoBERTa. Since Trans-LoRA requires generation from the source model, it could be challenging to implement. One could work around this by using another LM with a decoder for generation/discrimination, but this approach seems suboptimal. - Similarly, ViT from the vision community.

We thank the reviewer for this suggestion. To better illustrate the generality of our Trans-LoRA approach we include the following experiment on the popular T5 model with Coqa, Newsroom, and Squadv2 datasets. The results included in the table below illustrate that our approach can be also effectively applied to generative models beyond “decoder only” (encoder-decoder in this case), as well as to weaker models than LLaMA and Gemma. We will include this result in the final manuscript. The paper is initially intended for decoder-only architectures, but exploring encoder-only models could be an interesting future direction.

Also, the method seems to rely heavily on the generative ability of the source model/discriminator mdoel. I wonder if it's feasible to use weaker LMs like T5 or GPT2. The results in Table 5 also show that if we use random wikipedia (similarly to use a weaker LM), the performance drops significantly.

Thanks for the suggestion! In the above table, we tested with the T5 series model and demonstrated that our approach can be applied to these weaker language models as well on 3 separate tasks. We will include this result in the final manuscript. The experiment with transferring using random wikipedia data in Table 5 is only intended to show that data for the transfer is very important, it is not mimicking data generated from weaker models (as demonstrated by results on T5).

One of the nice property of the method is one only needs 5 real samples to conduct PEFT. But computation-wise, one still needs a relatively large LM to generate samples/discriminate. Is that actually favorable in practice?

Thank you for this suggestion, we are happy to elaborate on the discriminator overhead and would include this detail in the final version of the paper. Discriminator training typically takes just 1 epoch to reach over 90% accuracy. LoRAs were trained for 20 epochs in our experiments which was empirically observed to produce best performance for the source LoRA models. Given that both used equal amounts of samples per epoch (half real half synthetic for discriminator training), the cost of training discriminator is only around 1/20 of training cost of LoRA modules. Synthetic data generation for training discriminators took less than 5 minutes for most tasks on a single V100 for all base models. These costs are almost negligible compared to training the original LoRA.

In terms of synthesis during LoRA transfer, the generation process with the discriminator is quite fast (typically taking less than an hour for an entire dataset on V100). Moreover, our approach supports:

1. re-use of previously synthesized dataset (see Table 7)

2. parallelization of synthesis by using multiple GPUs.

This implies that our Trans-LoRA approach is effective even when we only synthesize the dataset ONCE for all future transfers, and this one time synthesis can be done fast. Additionally, we also showed above that it is possible to use a smaller LM (T5) for our approach, which would greatly speed up generation as well.

• minor: line 198: is it A100 40GB?

Thank you for noting this! It means a V100 with 40GB of machine memory, rather than 40GB of GPU memory. We will make sure to clarify this in our final paper.

Thanks for your continued support and for raising the score! We will make sure to incorporate all the points raised in the above discussion into the final version of the paper. We agree that our approach obtains promising results and opens many interesting future work directions for us and others to explore further.