EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction

The paper lacks explicit analysis of time costs (e.g., wall-clock or FLOP) of EMLoC for the fine-tuning and correction stages versus baselines, which would clarify real-world training costs. (W1, W2, Q3)

We thank the reviewers for raising such practical concerns. We have provided detailed measurements of memory consumption, throughput, and FLOPs during fine-tuning in Table 10 of the Supplementary Material, and the overall overhead of emulator construction in Table 4. For a more comprehensive analysis, we now include additional wall-clock runtime comparisons covering the full EMLoC pipeline — including emulator construction, fine-tuning, and LoRA correction — against both LoRA and QLoRA fine-tuning.

From these results, we observe that:

• EMLoC achieves lower overall runtime compared to the baselines, even when accounting for the overhead of emulator construction and correction.
• This efficiency stems from the activation-aware SVD construction, which is fast and avoids expensive continual pretraining, and from the lightweight emulator, which increases throughput during fine-tuning.
• Furthermore, our correction step is computationally negligible compared to other stages while providing additional performance improvement.

These findings reinforce the practical impact of EMLoC: it not only reduces memory consumption but also remains competitive or superior in wall-clock efficiency, making it well-suited for real-world, resource-constrained deployment scenarios.

Lack of comparisons with baselines on distillation or a foundation model of smaller size that is capable of being finetuned within the memory budget. (W3, Q2)

Thanks for the insightful suggestion. In fact, we have already included such comparisons in Table 1, where we report results for InternVL2B and InternVL4B, both of which have comparable training costs with our methods.

Regarding the suggestion to include distilled model baselines, we appreciate this valuable point. Unfortunately, we were unable to find any open-source, well-distilled versions of the InternVL2.5 series for a fair and direct comparison. Additionally, due to resource constraints, we are not in a position to conduct large-scale distillation ourselves. Nevertheless, to provide further insight, we conducted an additional experiment using InternVL3-2B (the latest version of the InternVL2.5 models used in our paper) as a stand-in for a compact model baseline. The results below show that EMLoC applied to InternVL2.5-8B with a 25% compression ratio still outperforms direct fine-tuning of InternVL3-2B. This suggests that EMLoC can retain the performance benefits of larger models even under tight memory constraints, offering a compelling alternative to using smaller or distilled models.

Formatting: In table 5, the last row reads "Ours" while others read "EMLoC." Perhaps this is a typo. (Q1)

We thank the reviewer for catching this. Yes, the last row in Table 5 should read "EMLoC", and we will correct it in the final version.

Novelty lies in using SVD-LLM for the emulator and a specific correction, but builds on prior concepts of adapter transfer. (W1)

We’re happy to clarify the novelty of our EMLoC and explain how it differs from prior SVD-based methods. Our EMLoC introduces several distinct innovations:

1. Flexibility in parameters to update: Unlike prior methods such as LoRAM (row pruning) and Offsite-Tuning (layer dropping), which impose hard constraints on which parameters can be updated, EMLoC adopts SVD to preserve the flexibility of standard fine-tuning, which allows all weights to be tunable.
2. No additional data / continual pretraining for emulator construction: By leveraging downstream data for activation-aware SVD, EMLoC avoids costly continual pretraining required by earlier approaches such as LoRAM, significantly reducing the overhead of emulator construction.
3. LoRA correction mechanism: We propose a LoRA correction mechanism that explicitly addresses the misalignment between the emulator used during training and the original model used during inference—a challenge that prior work has overlooked.

As demonstrated in Table 4, these innovations enable EMLoC to achieve better performance (up to 2.6% in accuracy) with significantly lower overhead (0.3 vs. 214 GPU-hours) compared to prior methods like LoRAM.

How sensitive are the results to the size or the distribution of the calibration data? (W2, Q1)

As shown in Figure 5 of the Supplementary Materials, performance plateaus after 32–64 calibration samples. We therefore use 64 examples as a practical default, indicating that our method is not highly sensitive to calibration set size, and a very small number of downstream samples is sufficient for effective emulator construction.

Regarding data distribution, Table 4 highlights the importance of using downstream data for calibration. Specifically, following LoRAM, we applied compression and continual pretraining using a general-domain dataset. However, this setup yielded worse results—up to 5% lower accuracy—compared to EMLoC where a small calibration set from the downstream-task dataset is used. This supports our design choice of using a small but downstream-aware sample set.

LoRA correction involves hyperparameters (𝜆) and a basis choice, but its effect is only briefly analyzed. (W3)

The impact of the 𝜆 is analyzed in Figure 4. We observe that increasing 𝜆 up to a moderate value (e.g., 5 in Figure 4) improves accuracy by 0.5%, likely due to less distortion in the correction term. However, overly large 𝜆 values degrade performance—even falling below that of uncorrected LoRA. We hypothesize that an overly large correction term may dominate the original LoRA, ultimately degrading performance.

Regarding the basis choice in the correction step, we provide a detailed discussion in Section C.2 of the Supplementary Material. We show that the resulting corrected LoRA weights remain the same, regardless of the choice of basis—demonstrating that the correction is mathematically invariant to this choice.

EMLoC focuses on model-weight memory only, not reducing optimizer state or activation memory. It remains complementary to approaches like gradient checkpointing or 4-bit quantization. (W4, L1)

It is correct that EMLoC focuses on reducing memory related to model weights. However, our method can be served as complementary to existing techniques. In fact, EMLoC is designed to be integrated seamlessly with methods like quantization, and activation checkpointing. As shown in Table 1, we combine EMLoC with QLoRA, and gradient checkpointing, and consistently observe better performance than baselines such as row pruning proposed by LoRAM and layer dropping proposed by Offsite-Tuning. These results demonstrate that EMLoC effectively reduces the memory footprint of model weights while remaining fully compatible with other techniques, jointly advancing the frontier of memory-efficient fine-tuning.

Did the authors compare against the recent LoRAM method or evaluate quantization-based fine-tuning? (W5, Q2)

We have included comparisons and discussion with QLoRA, using InternVL2.5 26B and 38B models in Table 11 of the Supplementary Material, where we observe that even with 4-bit quantization, QLoRA still requires 43GB of memory—while EMLoC can fine-tune the 38B model using only 20GB and achieves up to a 9% accuracy gain. Compared to QLoRA, EMLoC offers two main advantages:

• Greater flexibility in controlling the compression ratio, rather than being constrained by fixed bit-widths in quantization-based method;
• No degradation in inference performance, since quantization affects the model even during inference, whereas EMLoC retains the original model parameters for deployment

Regarding LoRAM, we clarify that our experiments do include a direct comparison. As shown in Table 4, the first row corresponds to LoRAM, while the last row corresponds to EMLoC. We observe that EMLoC achieves comparable or better performance, while also incurring significantly lower overhead for constructing the emulator.

Could the authors combine EMLoC with model quantization to push memory even lower? (Q5)

Yes, we have explored combining EMLoC with quantization, and the results are presented in Table 1, specifically in the third block of rows. The results show that EMLoC still outperforms prior baselines—row pruning corresponding to LoRAM, and layer dropping corresponding to Offsite-Tuning—by up to 2.9% in accuracy, suggesting that EMLoC is complementary to quantization for further memory reduction.

What is the runtime and memory overhead of building the emulator and performing LoRA correction? (W6, Q4)

We thank the reviewers for raising such practical concerns. We have provided detailed measurements of memory consumption, throughput, and FLOPs during fine-tuning in Table 10 of the Supplementary Material, and the overall overhead of emulator construction in Table 4. For a more comprehensive analysis, we now include additional wall-clock runtime comparisons covering the full EMLoC pipeline — including emulator construction, fine-tuning, and LoRA correction — against both LoRA and QLoRA fine-tuning.

The table below details EMLoC's memory usage across different steps. For the 8B model, each step of EMLoC uses less than 12GB, while the 38B model stays under 24GB. This efficiency makes EMLoC accessible to individual researchers, as it avoids the substantial computational and memory overhead typically required by previous methods that rely on continual pretraining.

Most experiments are on VQA and math QA benchmarks. How does EMLoC perform on large-scale open-domain NLP tasks or on very low-data regimes? (L3)

Thanks for the suggestion. While recent works such as LoRAM (ICLR 2025) have primarily focused on NLP tasks like QA and code generation, our evaluation goes beyond standard text-generation settings by including vision-language tasks (VQA) and diffusion model customization, demonstrating the broad applicability of EMLoC. Due to time constraints during the rebuttal period, we were unable to include additional experiments in other domains, but we have conducted more extensive ablations and added new baselines to address the review comments and to strengthen our analysis.

Can EMLoC extend to other adapter forms, considering that the correction algorithm is defined for LoRA? (Q3)

Yes. EMLoC’s first two stages are fully compatible with other adapters, still enabling memory-efficient training. To demonstrate this, we conducted an additional experiment using Houlsby Adapters in place of LoRA. As shown in the table below, EMLoC still performs better baseline methods in Table 1, highlighting its flexibility beyond LoRA. However, the current LoRA correction algorithm is indeed tailored to the linearity of LoRA modules, and we believe there is no straightforward extension to other PEFT methods.

Which layers receive correction, and is chosen per-layer or globally? (Q3)

The correction algorithm is applied to each LoRA independently, and we will clarify this in the revised paper.

The LoRA correction assumes linear mismatch between the emulator and the original model on a subspace. What if the mismatch is nonlinear? (L2)

We agree with the reviewer that our correction method assumes a linear correction. However, this limitation is shared by all approaches that apply weight-based corrections to LoRA modules without modifying the model architecture. As noted in (W1), EMLoC advances beyond prior work by explicitly addressing the mismatch between the training and inference models—an issue previously overlooked. While nonlinear correction remains an open direction for all LoRA-based methods, our results show that even linear correction consistently improves performance.

Diffusion results show somewhat lower scores than full training, suggesting possible quality trade-offs not deeply examined. (L4)

We totally agree with this observation. All efficient fine-tuning methods, including ours, involve trade-offs between efficiency and performance. While EMLoC achieves significant memory savings, there may be slight drops in performance compared to our upper-bound, full training.

What is the difference and relation between EMLoC and previous works like LoRA and E3VA? (W1, W7)

To clarify, as explained in Lines 32–40, the total memory cost of fine-tuning includes three main components:

• Optimizer states — auxiliary information such as momentum and variance in the Adam optimizer for each trainable parameter;
• Intermediate activations — values retained during the forward pass for use in backpropagation;
• Model parameters — the full base model itself for forward and backward passes.

LoRA reduces memory usage by lowering the number of trainable parameters, thereby decreasing the optimizer states. E3VA reduces activation memory by avoiding backpropagation through the backbone via architectural changes to module placement. In contrast, EMLoC addresses the often-overlooked bottleneck—the memory required to load and store the overall model parameters—by constructing a lightweight emulator to replace the original model during fine-tuning. Thus, EMLoC targets a fundamentally different source of memory consumption compared to LoRA and E3VA. However, as shown in Table 1, EMLoC combined with QLoRA and gradient checkpointing still consistently outperforms baselines such as row pruning proposed by LoRAM and layer dropping proposed by Offsite-Tuning. These results demonstrate EMLoC’s compatibility with other techniques and effectiveness in reducing model weight memory

What is the role of the "emulator" in EMLoC? How does the emulator-based framework reduce training memory usage? What is the difference between emulator-based framework and the standard LoRA framework? (W1, W2, W3)

To reduce the memory usage of model parameters, EMLoC introduces a lightweight surrogate model (i.e., the emulator), to replace the original model during standard LoRA fine-tuning, as described in Section 3.1. This emulator is constructed via activation-aware SVD using a small calibration set with the following three criteria (discussed in Section 3.2):

1. Fewer parameters: This enables memory-efficient fine-tuning since EMLoC no longer needs to load full original model parameters during fine-tuning
2. Flexible placement of LoRA modules: This makes EMLoC fully compatible with standard LoRA workflows and serves as a plug-and-play replacement.
3. Retention of downstream task knowledge: This makes the emulator behave closer to the original model during fine-tuning, alleviating the model gap between the emulator and the full model.

This design differs from standard LoRA, which only reduces the number of trainable parameters but still requires loading the full model into memory. In contrast, EMLoC reduces the memory footprint of the entire model during training.

The results in Table 4 are remarkable, but it is not clear why the proposed method significantly reduces training costs. (W6)

The significant reduction in emulator construction overhead shown in Table 4 comes from our use of activation-aware SVD on a small set of downstream data. Compared to prior methods like LoRAM which relies on continual pretraining over large general corpora and causes substantial overhead and potential downstream-task misalignment, our EMLoC is a lightweight and efficient approach. As a result, our method not only avoids this costly stage, but also produces an emulator more tailored to the target task, leading to both lower construction cost (0.3 vs. 214 GPU-hours) and better downstream performance (up to 2.6% difference in accuracy).

Missing discussion on a SVD-based design of LoRand. (W4)

Unfortunately, we were unable to locate a method called “LoRand,” but we believe the reviewer may be referring to RandLoRA [1]. While RandLoRA also utilizes SVD, its goal is fundamentally different from ours. RandLoRA draws inspiration from SVD to reformulate the LoRA module as a summation of multiple bases in order to improve expressiveness. In contrast, our use of SVD is applied to the base model itself, with the objective of compressing model parameters to enable memory-efficient fine-tuning. We will clarify this distinction and cite RandLoRA in the related work section.

[1] Albert, Paul, et al. "RandLoRA: Full-rank parameter-efficient fine-tuning of large models." arXiv preprint arXiv:2502.00987 (2025).

Table 6 seems to suggest that the impact of LoRA modifications is marginal. (W5)

We thank the reviewer for this observation. The marginal improvement is due to our aggressive compression setting, which may lead to relatively large misalignment between the emulator and the original model.

To better understand the results in Table 6, we conducted an additional ablation experiment using a less aggressive 50% compression ratio during emulator construction. The results support our hypothesis that the effectiveness of LoRA correction is related to the degree of misalignment between the emulator and the original model. When the emulator is constructed without activation-aware SVD, it fails to preserve key downstream behavior of the original model, possibly resulting in greater misalignment during fine-tuning and, consequently, weaker correction effects. Conversely, with less severe compression, the misalignment is reduced, and we observe that the impact of LoRA correction becomes more pronounced.

These findings suggest that while our current use of SVD-LLM provides a strong baseline for emulator construction, there is room for future exploration of more adaptive or task-aware compression strategies to further enhance alignment and correction effectiveness.

Provide a direct comparison between EMLoC and QLoRA in terms of accuracy and memory consumption, better demonstrating the advantages of EMLoC. (W1)

Thanks for the suggestion. In fact, we have included comparisons and discussion with QLoRA, using InternVL2.5 26B and 38B models in Table 11 of the Supplementary Material, where we observe that even with 4-bit quantization, QLoRA still requires 43GB of memory—while EMLoC can fine-tune the 38B model using only 20GB and achieves up to a 9% accuracy gain. Compared to QLoRA, EMLoC offers two main advantages:

• Greater flexibility in controlling the compression ratio, rather than being constrained by fixed bit-widths in quantization-based method;
• No degradation in inference performance, since quantization affects the model even during inference, whereas EMLoC retains the original model parameters for deployment

EMLoC shares some conceptual similarities with module arithmetic. It may be beneficial to include discussions of this connection in related work or appendix. (W2)

We thank the reviewer for pointing out the potential connection to module arithmetic. We agree that, at a high level, both approaches involve modifying or adjusting model weights. While there is a conceptual similarity, our LoRA correction method serves a different purpose. Module arithmetic techniques typically aim to merge multiple modules to combine different capabilities. In contrast, our method focuses on addressing the discrepancy between the compressed emulator used during training and the full model used during inference, by compensating for the misalignment introduced by this difference to further improve the performance. We will include the above discussions into the related work session.

Thank you for your response. I will keep my score.