The Quest for Winning Tickets in Low-Rank Adapters

Thank you for your thoughtful evaluation and feedback. We appreciate your recognition of the novelty and potential utility of Partial-LoRA, as well as your detailed suggestions for improving the scope and rigor of our work.

Due to space constraints, we'll address the weaknesses section here:

• W1 - Adaptive Masking Mechanism: Our method employs Algorithm 1 in the paper to every layer separately. For attention layers, we apply our method to query, key, and value weights separately as well. Therefore, the sparsity ratio is already adaptive to the layer at hand and is not set for the whole model or for multiple layers at once. This means we do not require adaptive mechanism since our method is naturally adaptive.
• W2 - Theoretical Justification: Our theoretical results backed by the empirical analysis on both vision and language tasks show that partial-LoRAs can approximate LoRAs. Thereby, our work is focused on the approximation of LoRAs and masking low-rank adapters. Investigating generalization guarantees would be out of the scope of this paper.
• W3, Q6 - Robustness and Noise Testing: We consider testing under adversarial conditions or noisy data to be out of the scope of this work. As mentioned, our work is concerned with approximation of LoRAs and showing that pruning LoRAs can allow for significant parameter reductions without performance drops.
• W4, Q7 - Reproducibility Information: We provide implementation details and the testing scenarios in Section 5. Additionally, as mentioned, our approach is based on pruning adapters such as LoRA and AdaLoRA. For each, we use the same settings used in the respective works for a fair comparison. We will make this more clear in the text.
• W5 - Cross-Domain Transferability: We're not sure what is exactly being asked here. Additionally, no works are cited for this question. We provide results in different modalities on datasets of different granularity with different sizes.

Summary of Our Rebuttal

We provide a very brief summary of our rebuttal as follows:

Adaptive Masking: We clarified that our method is already naturally adaptive as a clear result of Algorithm 1.

Reproducibility: We provide implementation details and the testing scenarios in Section 5. Our work sparsifies LoRAs and its variants. Therefore, we use the same hyperparameters as the work that our method is sparsifying and we have provided the details for the remaining hyperparameters in Section 5. We will add this to the implementation details in Section 5 and revise accordingly.

Layer-Wise Performance Variability: We have provided extensive visualization and deep insights into the variations of the sparsity ratios across the layers (Figure 5, Figures 7 to 10, Figure 14, and Figure 17.).

Comparison to Suggested Works: We are grateful for these suggestions and we provided our results on LLAMA2-7b on 3 datasets focusing on varying tasks and compared our results to the suggested work that was directly comparable to our work (Section 5.4 and Table 1).

Generalization Guarantees, Noise Testing, Adversarial Conditions, and Cross-Domain Transferability: Our theoretical results backed by the empirical analysis on both vision and language tasks show that Partial-LoRAs can approximate LoRAs. Therefore, our work is focused on the approximation of LoRAs and masking low-rank adapters. Investigating generalization guarantees, adversarial conditions or cross-domain analysis would be out of the scope of this paper.

We would be happy to elaborate further or clarify any of the points.

Thank you for your detailed review and for highlighting the clarity and presentation quality of our work, as well as the potential impact of our findings on the community.

• W1, Q1 - Evaluation Tasks: LLM instruction tuning and task alike for autoregressive models still focus on next word prediction which is inherently a classification task. We don't see any additional benefits of such evaluation changing anything for any other tasks. What we are trying to show is not a new method for adaptation but showing that the outputs of a pruned LoRA using our approach can approximate the outputs of a fully-paramterized LoRA. Nevertheless, we are running experiments on LLAMA2-7b as a larger model and will report the results when completed.

• W2 - Choice of Tasks: We chose 2 smaller and 2 larger datasets from GLUE which is the commonly used approach to evaluation on GLUE in recent works [a]. We can supplement these results with other datasets in GLUE for the camera-ready submission. As for vision experiments, our target was not the VTAB dataset since our work is not focused on improving image classifcation. We experiment on a wide array of datasets with narrow-focused datasets up to general image classification datasets. This is to evaluate the performance of our approach on datasets of different label granularity and observe the impact of LoRA sparsification. This includes Pets, CIFAR-10, CIFAR-100, Flowers, FER2013, GTSRB, FGVC-Aircraft and ImageNet-1k. If the reviewer believes another dataset would derliver unexpected results or new information, we'd be thankful for them to point out the dataset. We will add the corresponding results to the camera-ready submission.

• W3 - GLUE Development Set: Due to the rate limits on the website provided for benchmarking the test set of GLUE, the recent work [b] have chosen to evaluate on the development set and slice out a part of the training set for validation. We have followed the same approach for train/val/test split since using the GLUE benchmark website would not allow for direct comparison to the SoTA methods.

[a] Ilharco, G., Ribeiro, M., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., & Farhadi, A. (2022). Editing Models with Task Arithmetic. ArXiv, abs/2212.04089. [b] Zhang, Q., Chen, M., Bukharin, A., He, P., Cheng, Y., Chen, W., & Zhao, T. (2023). Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. ArXiv, abs/2303.10512.

Additional LLM Results on LLama2: We provide the results on LLAMA2-7b here and have added a new section in the revised version of the main text. The percentages inside parentheses show the fraction of the number of trainable parameters compared to that of full finetuning.

Here we compare the results of our work against full fine-tuning, low-rank adaptation, and Balanced Masking techniques [e]. We have added this new work as a comparison to other masking techniques for PEFT approaches recommended by reviewer vQUE. The related work section of the main text has also been revised accordingly. As evident in the table above, for LLAMA2-7b on SST-2, the sparsification of LoRA is also possible with larger models. We will be adding more datasets (starting with free-generation datasets) to this table in the coming few days. Compared to Balanced Masking, we obtain similar performance without the requirement of an expensive grid search over the sparsity ratio since we use the sparsity ratio derivation technique proposed in Section 4.2 of our paper.

We briefly bring a comparison to [e] here that can also be found in our related work section. Our approach employs an efficient masking technique that is non-ad-hoc with respect to how the residuals are masked. We theoretically justify our method using the literature on LTH where [e] does not. Perhaps more importantly, we employ a forward-only mechanism adaptive to the required capacity by the task and derive the required sparsity used for sampling our random masks. [e] does not provide any approach to deriving the probability of the Bernoulli distribution for the random masks and for experimentation, performs a grid search on the sparsity ratio as a hyperparameter.

We have also updated the submission with a revised version reflecting the discussed changes.

[e] Xu, Jing, and Jingzhao Zhang. "Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning." arXiv preprint arXiv:2405.02596 (2024).

I want to thank the authors for their continued efforts in providing additional clarifications and experiments.

Open-ended or label-based, every benchmark requires a form of quantitative evaluation and that metric could simply replace the accuracy used in Algorithm 1.

Thank you for the clarification. I think this aspect should be stressed in the paper, perhaps alongside a discussion on the factor 0.9 as threshold. This factor appears quite intuitive in classification settings, but is the same true for all relevant metrics (e.g. CIDEr, Rouge, BLEU, F1, ...)? Another case to consider is when the base model (before fine-tuning) achieves 0 accuracy on the task at hand or cannot perform it at all for different reasons. This case is relevant, as LoRA is not dependent on any prior performance.

• We provide a list of partially and completely open-ended benchmarks alongside their evaluation metric
• If the reviewer is aware of any benchmarks where quantitative evaluation using the abovementioned metrics is impossible, we would appreciate it if they could share it with us for further examination.

Clearly, many common benchmarks use accuracy and similar classification metrics for evaluation, and the proposed method is applicable there. One type of evaluation not considered seems to be LLM-as-a-judge such as applied to MT-Bench [1] LLaVA [2] or DPO [3], where performance is relative to another model or ground truth text answers. However, it seems possible to handle this in the same way as suggested for other quantitative metrics as suggested in the previous response.

A general aspect to consider is that fine-tuning is a very general framework that can be applied to both supervised and unsupervised tasks in greatly varying settings. The main application of PEFT methods are not benchmarks, even if we evaluate the methods there and expect that the observed performance generalizes. If a method with the same generality as LoRA is proposed, care must be taken not to restrict the scope of application, or clarify what is the intended (narrower) scope of application.

The vision experiments are done through a narrow to general image classification settings spanning 8 datasets. The language experiments are done on 2 smaller and 2 larger datasets from the GLUE benchmark.

Compared to DoRA [4] and VeRA [5], the number of different tasks is still relatively small, even if the number of datasets is comparable. For example, DoRA features 4 tasks (treating video and image understanding as separate tasks) and VeRA features 4 tasks as well. I understand that the main comparison target of this work is LoRA, but the evaluation presented there is not replicated, and then it becomes difficult to understand why these particular tasks are chosen. Without common grounds, my worry is that tasks can be cherry-picked from a relatively large pool of possible applications.

We provide the results on LLAMA2-7b here and have added a new section in the revised version of the main text

I appreciate the effort in providing these additional experiments, and I also appreciate that a revised pdf has been made available. It would have been great to see results on an additional task (instead of a new model on a GLUE task which had already been evaluated before). Finally, why does full fine-tuning only tune a fraction of 1e-6 parameters of the model?

In summary, I would like to ask the authors to add a discussion on the metric in Algorithm 1 where they explain how the proposed method can be used for arbitrary tasks and metrics. If also additional tasks can be added to the evaluation, I think the paper will be much stronger and I will be happy to further raise my score.

[1] Zheng et al.: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In NeurIPS, 2023
[2] Liu et al.: Visual Instruction Tuning. In NeurIPS, 2023
[3] Rafailov et al.: Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In NeurIPS, 2023
[4] Liu et al.: DoRA: Weight-Decomposed Low-Rank Adaptation. In ICML, 2024
[5] Kopiczko et al.: VeRA: Vector-Based Random Matrix Adaptation. In ICLR, 2024

Thank you for your insightful review and for appreciating the novelty and theoretical foundation of our work.

Due to space constraints, we'll address all the points in the weaknesses section here:

• W1 - Table with All Results:Thanks for your suggestion. We'll add a table in the Appendix quantitatively detailing the parameter count alongside the accuracy for each experiment.
• W1, Q1 - Reducing the parameters on the full adapter: The only way to reduce the number of parameters of the fully-parameterized adapters would be to reduce their rank which would be unfair since then they would have fewer subspaces accessible during finetuning. Nevertheless, Figure 6 in Section 5.5 allows for comparison of different ranks for LoRA to that of Partial-LoRA where the fewer ranks of LoRA cannot achieve the same accuracy as Partial-LoRA with a higher rank. We will add a secondary x-axis to Figure 6 showing the number of parameters, allowing for parameter and rank comparison simultaneously.
• W2 - Larger Models: We are running experiments on LLAMA2-7b as a larger model and will report the results when completed.
• W3 - Targeted Sparsification: We provide results for such a method we named "Targeted-LoRAs" in Table 1 of the main text where instead of using the top extracted parameters only for sparsity ratio, we target those parameters specifically. We show in Table 1 that Targeted-LoRAs perform worse than Partial-LoRAs on average and this slightly lower performance from Targeted-LoRA remains consistent across different number of shots as shown in Table 2 in Appendix B. Additionally, we provide an inbetween method where we use the extracted top parameters but treat them stochastically and provide those results in Figure 11 in Appendix D to show the potential of introducing randomness to the targeting process.
• W4 - Masking Model vs. Masking LoRAs: There is a threshold for sparsity after which the number of parameters required for training when the model itself is masked is less than the same number when using LoRAs: A LoRA has $m \times d + d \times n$ parameters, where $m$ and $n$ are the dimensions of the model's weight matrix, and $d$ is the LoRA's rank. To determine the threshold where training the full model is better than using a masked LoRA, we compare the number of parameters where $m \times n \leq m \times d + d \times n$. Simplify to get $\frac{m \times n}{m + n} \leq d$. Therefore, this threshold is dependent on the rank of the LoRA. For example, considering ViT-B-16, we have matrices of size $3072 \times 768$, in a simplified case of balanced pruning on rows and columns, the mask would have to sparsify the rows and columns to only 25 rows and 5 columns for masking the model itself to be more efficient than a LoRA with a rank of 4. Therefore, the number of parameters for such as scenario would be very small unless the weight matrix itself is small to begin with. In such a case where the model is small, a LoRA would already not be appropriate for fine-tuning.
• W5 - Necessity for a Subset Dataset for Identifying Sparsity Ratios: For training LoRAs we will always need a dataset. We have shown that a few-shot dataset is enough to extract masks (lines 312-314). The same dataset is used for training the subnetwork. Note that for open-world prediction models such as CLIP, a separate training phase is not necessary since we can extract the subnetwork by initializing the linear classifier using the text encoder as discussed in Section 5 under Datasets and Models.
• W6 - Revision of Theorem to Make it More Rigorous: We will revise the theorem to make it more rigorous.

Thank you for your thoughtful review and for highlighting both the strengths and areas for improvement in our work.

• Q1, Q2 - Clarification on the Mask and Theorem 4.1: We explicitly state that there exists a sparse mask $U$ in line 222 where after the application of this mask to the residuals of a low-rank adapter, the outputs of this adapter stay within a bound compared to the outputs of an unmasked adapter. This point is more intuitively discussed in lines 222 to 226. Therefore, our proof is sufficient in the context of our work and as an extention of [a]. We will revise and clarify further in the main therorem text. We also note that we have clearly cited and shown the link between our work and Theorem 2.5 from [a]: there is no proof for the bound for adapters in this prior work. Moreover, $U$ is only required to be sparse and the approach we took to decompose it has no implications in the theorem. However, our approach leads to much more efficient implementation. In other words, there is no computation necessary for the decomposition of the mask as explained in lines 289 to 294.

• Q3 - Evaluation on recent models: We use Deberta for our language experiments and ViT-B-16 for our vision experiments. The reason for choosing these models was to directly compare with state-of-the-art [b]. The current literature commonly uses Deberta alongside the GLUE benchmark due to how streamlined the comparison can be to the recent works. We're not sure what the reviewer thinks new experiments on newer models would reveal that isn't already addressed by the current experiments, given that the structure of these models is identical to DeBERTa. The recency of the recent models does not introduce an aspect of finetuning that is different to that of less recent models. Nevertheless, we are running experiments on LLAMA2-7b as a larger model and will report the results when completed.

• Q4 - Significance of the Parameter Reduction We first would like to highlight that the significance of our work is in drawing connection between the low-rank adapter models and lottery ticket hypothesis that has never been investigated before. Based on this theoretical insight, we designed an algorithm that shows the parameter size could be in fact even lower in the adapters that will even lead to higher performance. We hope the merits of our work is seen in this light. Second, a practical impact of our lower paramter count is in the case of models that do not fit on consumer hardware. For example, in cases such as LLAMA-7b, without memory reduction techniques, finetuning is impossible on consumer hardware. Given issues arising with finetuning with quantization [c], our approach enables finetuning without relying on such hardware oriented techniques, allowing for democratization of fine-tuning.

[a] Advait Gadhikar, Sohom Mukherjee, and Rebekka Burkholz. Why Random Pruning Is All We Need to Start Sparse, May 2023. URL http://arxiv.org/abs/2210.02412. arXiv:2210.02412 [cs].

[b] Zhang, Qingru et al. “Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning.” ArXiv abs/2303.10512 (2023): n. pag.

[c] Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2021). A survey of quantization methods for efficient neural network inference. CoRR, abs/2103.13630. https://arxiv.org/abs/2103.13630

I extend my gratitude to the authors for their comprehensive response, and here are some of my follow-up questions:

• I appreciate the authors' commitment to refining the statement of Theorem 4.1. A statement following the style similar to theorem 2.5 in (Gadhikar et al., 2023), which explicitly states "Then with probability ..., there exists a mask so that ...", would greatly enhance the clarity and understanding of the theorem.

• I remain concerned about the connection between the proposed method and lottery ticket hypothesis as claimed in the paper. Theorem 4.1 is proving the existence of winning ticket, represented by a sparse mask $U$. Specifically, Theorem 4.1 does not address the existence of a winning ticket that can be written in the format of Equation (6). It is unclear to me how the sparse LoRA with $u_{row}$ and $u_{col}$ by Equation (6) can be guaranteed to be a winning ticket. Could the authors provide more details and explanation on this?

• I appreciate the authors' experimental results with LLAMA-7b. In addition to the accuracy results, could the authors also provide data on memory reduction? This would help demonstrate the effectiveness of the proposed method in enabling fine-tuning without relying on hardware-specific techniques, as claimed.

This is done by zeroing out the rows of the $B$ matrix and columns of the $A$ matrix in a low-rank formulation (i.e. in LORAs) that results in $\Delta W = (u_{row}B)(u_{col}A)$. While it is true that matrix multiplication is non-commutative, this formulation is the exact same in terms of the final result compared to $\Delta W \cdot U$.

I appreciate the authors' clarification on this point. However, from my perspective, the above statement implies the following proposition holds:

(Existence of Decomposable Winning Tickets) Given $\Delta W = BA$, there exists a mask $U$ such that

1. error caused by $U$ is constrained by the bound specified in Theorem 4.1,
2. there exist $u_{row}\sim\mathcal{B}(p_{row})$ and $u_{col}\sim\mathcal{B}(p_{col})$ such that $(u_{row}B)(u_{col}A) = \Delta W \cdot U$.

Unless the above proposition holds, it is difficult to explain why the direct implementation can be exact same as a decomposed implementation, or why there always exists a decomposed implementation that is a winning ticket. Maybe I misunderstood the authors' point, and I would appreciate it if the authors could provide more insights on this.

I delve deeply into this point because it is the critical link between the proposed method (decomposed formulation) and the proof of the lottery ticket hypothesis (Theorem 4.1, direct formulation). Missing this link would significantly weaken the paper's persuasiveness in drawing a connection between the low-rank adapter models and the lottery ticket hypothesis.

Memory Reduction for LLAMA2-7B: We greatly appreciate the reviewer’s detailed breakdown of LoRA’s memory footprint. We have revised the manuscript to report the exact trainable parameters across different methods. Below is this updated table:

We acknowledge the reviewer’s calculation, which accurately estimates that the memory overhead of a single LoRA adapter (e.g., 4.2M parameters with AdamW optimizer states) constitutes a small fraction of the total memory footprint for LLAMA2-7B. We bring here two scenarios where LoRA pruning for LLAMA2-7b is crucial to the application:

High-Rank Finetuning: We consider task vectors [a] where the model is required to be fine-tuned on a small dataset with the strict requirement of a large rank for the adapter. Moreover, the application of the adapter is not only done on query (Q) and value (V) matrices but on every linear layer. In the case of LLAMA2-7b using LoRA with a rank of 256 on every linear layer, an unpruned LoRA would require 639 million parameters which would constitute just over 10GBs of memory. In such a case a pruned LoRA would be the only possible solution to obtain a task vector on that dataset.

Multi-LoRA: Our initial claim was motivated by scenarios where multiple LoRAs are combined within a single model—a setting explored in prior works such as [b], [c], and [d]. As a future work, we hope to extend our masks to have potentially non-overlapping masks for different LoRAs. In [b], the approach involves training thousands of LoRA adapters and combining them with varying coefficients in a continual learning paradigm. In such cases, reducing the size of each LoRA adapter becomes critical. By sparsifying LoRA and reducing the trainable parameter count from 4.2M to just 68K, the model can accommodate a significantly larger number of adapters, enhancing scalability and flexibility in multi-task or continual learning applications.

We appreciate the opportunity to clarify this point and will incorporate these insights into the revised manuscript. We hope this addresses the reviewer’s concerns and provides a more nuanced perspective on the benefits of sparsifying LoRA in practical use cases where the use case extends beyond a conventional experiment on a curated dataset. We hope that the reviewer considers raising their score for our paper and supporting its acceptance.

[a] Ilharco, G., Ribeiro, M., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., & Farhadi, A. (2022). Editing Models with Task Arithmetic. ArXiv, abs/2212.04089.

[b] Sheng, Y., Cao, S., Li, D., Hooper, C., Lee, N., Yang, S., Chou, C., Zhu, B., Zheng, L., Keutzer, K., Gonzalez, J.E., & Stoica, I. (2023). S-LoRA: Serving Thousands of Concurrent LoRA Adapters. ArXiv, abs/2311.03285.

[c] Huang, C., Liu, Q., Lin, B., Pang, T., Du, C., & Lin, M. (2023). LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition. ArXiv, abs/2307.13269.

[d] Zhao, J., Wang, T., Abid, W., Angus, G., Garg, A., Kinnison, J., Sherstinsky, A., Molino, P., Addair, T., & Rishi, D. (2024). LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report. ArXiv, abs/2405.00732.

Thank you for your thorough and constructive review of the paper and for recognizing our work as well-written and interesting.

• W1 - Dataset and model choices: We use ViT-B-16 for vision and Deberta-v3-base for language models which are the commonly used benchmarking models for works on LoRAs [a, b, c] to be able to directly compare our work to the literature. Interestingly, smaller models can be harder to prune [d]. We are running experiments on LLAMA2-7b as a larger model and will report the results when completed.

• W1 - Case where LORA underperforms full-finetuning: Our method highlights the lottery ticket hypothesis in the low-rank approximations based on which we propose Partial-LORA. Therefore, we don't expect Partial-LoRAs outperforming the full finetuning in the case where LoRAs underperform. Nevertheless, we report here the results of full finetuning on GTSRB and CIFAR10 datasets where full finetuning performs better than parameter efficient low-rank counterparts alongside each method's paramter count in parentheses. In both cases our method exhibits marginal difference compared to LORA while utilizing fraction of parameters. We will add the full finetuning performance results to the paper.

Table I: Comparison to full finetuning on CIFAR-10 and GTSRB datasets.

• W2 - Few-shot and Sparsity Ratio As shown in Figure 13 in Appendix E, over different number of shots, there is a large overlap between the extracted subnetwork from both SVD-based and SNIP (gradient-based) methods. Therefore, considering that one of the main messages of our work is about the importance of sparsity ratios, the insignificant changes in the top indices across different shots shows that the sparsity ratio is not dependent on the variability of the shots. We'll add a discussion on this topic to the main text of the paper. On the matter of time it takes for the computation of the sparsity ratio, if gradient-based methods such as SNIP or IMP are used (Section 4.2), larger datasets may need more time and memory to extract the ratios. However, using our proposed SVD-based approach, due to the forward-only aspect of our method, the timing does not change significantly when extracting subnetworks for larger datasets. As an example, for the case of larger models such as LLAMA2-7b, the subnetwork extraction process takes less than an hour on a 4090 GPU.

• Q1 - Training Threshold: That is an interesting experiment considering that this threshold directly determines the resulting sparsity ratio. The choice of 90% accuracy threshold is adopted from the intrinsic dimensionality literature [e] which themselves take that threshold from [f] where the 90% accuracy margin is deemed a satisfactory condition. We will complement our experiments with 70% and 80% margins.

• Q2 - Different Method of Memory Usage Reduction: Great observation! We suggest using higher-rank because our results show that only a small number of parameters are required to finetune the model, i.e., Sparse residuals can make the necessary adjustments to access critical subspaces. By increasing the rank of the residuals, we can more precisely modify the top subspaces of the model while still using fewer parameters than a fully-parameterized LoRA. Meanwhile, simply increasing the number of parameters may not lead to better performance since these critical subspaces are already accessible. That said, there is a limit: reducing the number of parameters too much can prevent access to these important subspaces, hurting performance. We'll expand on this idea in the discussion section.

[a] Hayou, S., Ghosh, N., & Yu, B. (2024). LoRA+: Efficient Low Rank Adaptation of Large Models. ArXiv, abs/2402.12354.

[b] Zhang, Qingru et al. “Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning.” ArXiv abs/2303.10512 (2023): n. pag.

[c] Hu, J.E., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., & Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. ArXiv, abs/2106.09685.

[d] Xu, Jing, and Jingzhao Zhang. "Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning." arXiv preprint arXiv:2405.02596 (2024).

[e] Aghajanyan, Armen, Luke Zettlemoyer, and Sonal Gupta. "Intrinsic dimensionality explains the effectiveness of language model fine-tuning." arXiv preprint arXiv:2012.13255 (2020).

[f] Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. arXiv preprint arXiv:1804.08838, 2018.