We sincerely thank Reviewer pPup for the thoughtful and constructive feedback. Below, we provide detailed responses to each of the key questions raised.

1. Response to the scalability.

(1) Per-class forget vector limitation. We respectfully argue that this design choice does not significantly hinder scalability, for several reasons:

• Forgetting vectors are small in size (e.g., 0.15M parameters), especially compared to full model weights (e.g., ViT-Base: 85M, ViT-L: 303M). Even when preparing vectors for a large number of classes, the total storage remains negligible.
• Reusability across settings: Once learned, class-wise forget vectors can be reused across different forgetting requests (e.g., class subsets or random data unlearning).
• Training Independence & Parallelism: Forgetting vectors for each class can be learned independently and in parallel with intact model, making the process highly scalable in practice.
• Based on Table A2 in Appendix F, using ImageNet-10 and ViT-Base as an example. While retraining requires 301.98 minutes, our method learns a class-wise forget vector in only 4.80 minutes, highlighting the practical advantage of our approach in terms of computation time.

(2)Scalability to larger datasets. We evaluate the scalability of our method on ImageNet-100, a more diverse and challenging dataset with 100 classes, using the ViT-Base backbone. As shown in Table R11, our method achieves strong unlearning performance (UA, MIA-Efficacy) while preserving utility (RA, TA) and maintaining a low Avg. Gap. These results demonstrate that our approach remains effective in large-class settings, without requiring model retraining—an advantage in scenarios where retraining is costly or infeasible.

Table R11: Performance overview of various MU methods for image classification in class-wise forgetting scenario on ImageNet-100 using ViT-Base.

(3) Fine-grained cases. We thank the reviewer for raising this important point. ImageNet-100 does include several semantically similar or visually confusing classes, such as "Tiger Cat" and "Egyptian Cat". These results in Table R12 highlight the fine-grained generalizability of our forgetting vectors, effectively forgetting one class while retaining accurate recognition of a semantically similar class.

Table R12: Accuracy under forgetting of semantically similar classes ("Tiger Cat" vs. "Egyptian Cat")

2. Response to the connection between Sec. 4 and other chapters.

(1) Chapter connection. We thank the reviewer for pointing this out. Section 4 is intended as an empirical motivation for the design of Forget Vectors in Section 5. Specifically, we examine whether model-based unlearning persists under input perturbations (e.g., pixel-level noise), and find that while forgetting effects remain, model utility degrades significantly.

This observation motivates our core hypothesis: if data perturbations can induce forgetting, can we learn such perturbations that are minimal yet effective? Section 5 addresses this by introducing input-level forget vectors that suppress forgotten data while preserving performance on retained data. We clarify that Section 4 motivates the formulation of the FV objective, which explicitly balances forgetting and utility. We will revise Section 4's introduction to make this connection clearer.

(2) Line graph in Figure 2. Thank you for your comment. We will replace it with a bar chart and improve the figure presentation in the next revision.

3. Response to insufficient experiments.

(1) Scale with the size of data in larger dataset. Table R13 evaluates different forgetting ratios (10% and 20%) on larger dataset ImageNet 100, demonstrating that compositional unlearning scales effectively with the size of the data.

Table R13: Compositional unlearning on ImageNet100 for random data forgetting with 10% and 20% forgetting ratios.

(2) Fine-grained cases. Please refer to the Response 1 and Table R12.

4. Response to numerical instability.

We appreciate the reviewer’s careful observation. However,the observed variation in UA gap across different combinations of $w_1$ and $w_2$ in Figure 3a is not a symptom of numerical instability, but rather a natural and expected result of semantic directionality in our forget vector arithmetic.

• Our method operates in input space, and vector combinations (e.g., between "automobile" and "bird") represent semantic, not numerical, interactions. The effect of adding or subtracting a forget vector is nonlinear and asymmetric, depending on how the perturbation aligns with the model's learned representations. This explains why UA does not change linearly with weight.
• As shown in Figure 3c, our method identifies a robust operating point (green star) achieving low UA and RA gaps, confirming the method’s controllability and reliability despite nonlinearity.

In short, our approach does not involve numerically sensitive optimization procedures, which enables direction-aware forgetting behavior.

5. Response to more detailed discussion is needed regarding the novelty of the proposed method.

(1) Difference with provided paper. We thank the reviewer for highlighting this work. While Shibata et al. introduce mnemonic codes for class-level forgetting in lifelong learning, our approach differs significantly:

• Problem formulation: Shibata’s method predefines mnemonic codes for all classes and removes them at test time to forget, requiring prior code assignment. In contrast, our forget vector is learned only for classes to be forgotten and applied directly to inputs ("add-to-forget" vs. "remove-to-forget").
• Inference-time label dependency: Shibata’s method requires access to class labels at inference to discard the correct code. Our method is label-agnostic—once learned, the forget vector can be applied to any input without needing label information, enabling black-box or privacy-sensitive deployment.
• Granularity of forgetting: Shibata’s method is designed for class-level forgetting only. The mnemonic codes are tightly coupled with each class. In contrast, our forget vector can be learned for any subset of samples, enabling flexible unlearning, including class-wise and random data forgetting.
• Post-hoc applicability: Mnemonic codes must be integrated during model training. Our method is post-hoc and model-agnostic, requiring no retraining or architecture changes.
• Compositional unlearning: Our framework supports compositional forgetting across multiple requests, a capability absent in Shibata et al.

(2) Application advantage. Our method intentionally avoids modifying model parameters, enabling post-hoc, on-demand unlearning in frozen or resource-constrained settings. While residual information in the weights is a valid concern, our approach focuses on behaviorally suppressing the model’s ability to recognize forgotten data through input-level perturbations, rather than physically altering the parameters.

This design offers key practical benefits:

• Reusability & utility preservation: The model remains unchanged, ensuring consistent performance on future downstream tasks.
• Dynamic unlearning: Forget requests can be handled on-the-fly without retraining or versioning. Forgetting is reversible—simply remove the applied vector to restore recognition.
• Lightweight storage: Only compact forget vectors are stored per request, avoiding multiple large model checkpoints. Therefore, we argue that while we do not remove forgotten data from model weights, our model-agnostic, input-level strategy is well-suited for practical deployment scenarios, especially those requiring flexibility, efficiency, and test-time control.

6. Response to the more detailed discussion.

Q1: Scalability of the proposed method.

• Larger classes: Please refer to above Response 3 and Table R13.
• Fine-grained cases: Please refer to above Response 1 and Tabel R12.

Q2: Numerical instability. Please refer to above Response 4.

Q3: Novelty of the proposed method and the significance of not modifying existing model parameters. (1) On Novelty. As quoted by Reviewer eoHp, our work explores novel unlearning paradigms and expands the applicability of unlearning. Meanwhile, Reviewer Wzpy explicitly recognize that our proposed methodology is novel. We introduce a new input-based approach to machine unlearning via forget vectors—universal, input-agnostic perturbations that remove the influence of target data without modifying model weights.

Additionally, our work is the first to propose compositional unlearning, allowing flexible, scalable unlearning through arithmetic combinations of class-wise forget vectors. From a practical standpoint, our method is inherently suited to black-box settings, requiring only input-output access, and thus supports unlearning even on proprietary or closed-source models.

(2) On significance of not modifying existing model parameters. Please refer to above Response 5 (2).

Q4: More comprehensive experiments regarding class combinations are insufficient. Please refer to above Response 3 and Table R13.

Dear Reviewer pPup,

Thank you very much for reviewing our paper and carefully reading our responses and comments from other reviewers. We sincerely appreciate your recognition of our responses as clear and thoughtful, and we are truly grateful for your more positive reassessment of our paper. Your constructive feedback throughout the review process has been instrumental in helping us improve the clarity and overall quality of our work. We will surely improve our manuscript to reflect the responses to your comments. If you have any further questions or suggestions, please feel free to reach out.

Best regards,

Authors

We greatly appreciate Reviewer BKJF’s thoughtful and constructive feedback. Below, we provide detailed responses to each of the key points raised.

1. Response to broader transformations and contrast to feature-level perturbations.

(1) Vulnerable to broader transformations. While our method indeed operates in the pixel space, we would like to emphasize that our forget vector exhibits substantial transferability to unlearn the unseen forget set, as shown in Table 2 and the accompanying analysis (page 8). In these evaluations, our method significantly outperforms existing baselines in unlearning accuracy across all three test scenarios, demonstrating robustness even when applied to out-of-distribution forget data.

(2) Contrast to feature-level perturbations. We agree that VPT offers a promising direction in parameter-efficient adaptation and may yield enhanced robustness in certain scenarios. However, our approach fundamentally differs in deployment assumptions and design philosophy. VPT operates by injecting learnable prompts into the activation space of the transformer layer, which requires full access to the model at the inference phase to activate or deactivate the prompting effect. In contrast, our forget vector method is designed to operate entirely in the input space, requiring no access to model weights or activations. This enables it to be applied to a given model without any model internal weights/activation intervention.

2. Response to scalability with the number of classes or the size of data on combining class-wise forget vectors.

Thank you for pointing this out. We have conducted additional experiments to examine the scalability of our method with different settings on combining class-wise forget vectors:

(1)Scale with the number of classes overall. We conduct experiments on ImageNet-100 using the ViT-Base backbone, with random forgetting ratios of 10% and 20%. As shown in Table R8, we compare results with individually trained forget vectors (FV) and observe that compositional unlearning via vector arithmetic (CU-FV) remains effective. This demonstrates that our method scales well to larger datasets with more classes.

(2) Scale with the size of data.

Table R8 evaluates different forgetting ratios (10% and 20%), demonstrating that compositional unlearning scales effectively with the size of the data.

Table R8: Compositional unlearning on ImageNet100 for random data forgetting with 10% and 20% forgetting ratios.

3. Response to the formal deletion guarantees.

We thank the reviewer for raising this important point. It is true that our method does not offer formal deletion guarantees in the same theoretical sense as retraining-based approaches. However, our primary goal is to develop a practical, lightweight, and deployable method that enables unlearning without requiring access to or modification of model weights, which is an increasingly relevant need in real-world scenarios such as on-device learning and frozen-model APIs. From a broader perspective, unlearning techniques can be categorized into three stages:

• Pre-processing: Manipulate the training data before model training.
• In-processing: Modify the training process or model weights to remove memorization.
• Post-processing (ours): Keep the model fixed and perform unlearning at inference time.

In fact, our approach uniquely belongs to the post-processing paradigm of unlearning methods, where the model remains intact, and forgetting is achieved solely through input-space perturbation at test time.

4. Response to scale to larger datasets.

We empirically validated the scalability of our method on ImageNet-100, a more challenging and diverse dataset with 100 classes, using the ViT-Base backbone. We demonstrate the results under the class-wise forgetting setting. As shown in Table R9, our method maintains competitive unlearning effectiveness (UA, MIA-Efficacy) while preserving model utility (RA, TA), and keeps the Avg. Gap low. This indicates that our approach scales well to large-class settings without modifying the model weights, which is particularly appealing when model retraining is impractical.

Table R9: Performance overview of various MU methods for image classification in class-wise forgetting scenario on ImageNet-100 using ViT-Base.

5. Response to forgetting at the level of individual data points.

We appreciate the reviewer for raising this subtle but important question. We would like to clarify that forgetting individual datapoints is fundamentally challenging, especially in vision classification tasks. In classification models, a single datapoint is typically embedded within a dense neighborhood of semantically similar examples belonging to the same class. Due to this data redundancy and inter-class generalization, the model can often interpolate from nearby data points to retain its predictive capability, even if one individual image is removed or masked. As highlighted in recent work such as "Challenge Forgets: Can LLMs Forget Specific Training Data?" (Liu et al., 2024), forgetting individual data is often shown to be infeasible or unstable, even in large-scale generative models. In vision tasks, this difficulty is further amplified by the high structural similarity among instances within the same class. For instance, trying to forget a single cat image among hundreds of similar cats rarely affects the model’s recognition capability, due to its semantic generalization ability. Our goal is to remove the model’s capability to recognize a class or subset, not necessarily to ensure perfect deletion of any specific training instance. In line with this, most existing unlearning literature also evaluates at the class- or subset-level, rather than per-instance, due to the practical and theoretical limitations described above.

6. Response to scalability to the number of classes or the amount of data to be forgotten.

(1) Regarding the scalability to the number of classes.

To evaluate how our approach scales with the number of classes in the forget set, we conduct experiments on ImageNet-10 using the ViT-Base backbone. Specifically, we measure performance as the model is tasked to forget 1, 2, and 3 classes simultaneously under the class-wise forgetting setting. The results can be found in Table R10.

Table R10. Performance overview of various MU methods for image classification in class-wise forgetting scenario on ImageNet-10 using ViT-Base with different number of forgetting classes.

(2) Regarding the scalability of the amount of data to be forgotten.

Please refer to Response 2 and Table R8.

These results indicate that our method remains effective and stable across a range of forget ratios and larger dataset, achieving consistent suppression of forgotten data while maintaining high accuracy on retained data. This demonstrates the scalability and robustness of our approach under varying unlearning demands.

Dear Reviewer BKJF,

Thank you for your more positive evaluation of our paper. We sincerely appreciate your thoughtful and constructive feedback. We’re very glad to hear that our clarifications and additional analyses helped address your concerns and highlighted the strengths of our work. We will make sure to incorporate the discussed points into the final version to further enhance the clarity and impact of the paper. If you have any further questions or suggestions, please feel free to reach out.

Best regards,

Authors

We greatly appreciate Reviewer Wzpy’s insightful and constructive comments. Below, we provide detailed responses to address each of the points raised.

1. Response to W1 about "Right to be forgotten" legislation

We thank the reviewer for raising this insightful question. Our primary goal is to enable the removal of data influence as manifested in the model’s classification behavior. While our approach, like most approximate unlearning methods, does not guarantee complete erasure of the original data points, it effectively eliminates their impact on the model’s predictive capability. This is why we cited "right to be forgotten" legislation as a motivation. To illustrate this perspective, consider the following analogy: if one wishes to "forget a person," it may be difficult to erase all associated memories. However, if the person's face is always blurred when they appear (analogous to applying a data perturbation), the perceiver–in this case, the model–can no longer recognize them. From the standpoint of unlearning outcomes, both our data-based approach (using a forget vector) and model-based methods ultimately aim to eliminate the influence of the targeted data, albeit through different mechanisms. In the revision, we will clarify our motivation more precisely and explicitly distinguish our data-based unlearning approach from traditional model-based methods, as well as its relation to the objectives of the "right to be forgotten" legislation.

2. Response to W2 about practical application

While input perturbation may intuitively result in utility degradation on retain set, we carefully designed our method to ensure that such perturbations are minimally invasive to utility, while still effectively disrupting the model’s ability to remember the forget data. In particular, as shown in Equation (4) in the main paper, during the optimization of the forget vector, our loss function jointly considers both forgetting (on the forget set) and retention (on the retain set). This allows us to achieve a practical trade-off between effective forgetting and minimal utility loss. Such a trade-off is further investigated through our experiments (see Table 1 in main paper). The Avg. Gap with Retrain shows that the forget vector improves unlearning effectiveness, ranking among the top two unlearning methods, which is significant compared to the certain loss in utility. As with model-based unlearning approaches, we acknowledge that our method involves a trade-off between unlearning effectiveness and utility preservation. However, part of this trade-off arises from our conservative evaluation protocol. Specifically, the reported testing accuracy (e.g., in Table 1) reflects a worst-case scenario in which the forget vector is mistakenly applied to the retain data. This setup ensures that our results represent a lower bound on utility performance under potential deployment errors. That said, a key practical advantage of the forget vector lies in its input-based nature: since it does not alter the model’s weights, the same model can be used in its original utility mode, without any loss in performance, simply by omitting the forget vector at inference time. In this sense, full utility is preserved unless the forget vector is explicitly applied.

3. Response to Q1 about the explanation of sentence from Line 222.

In the context of classification models, higher prediction accuracy on the forget set implies that the model has retained knowledge about that data. Conversely, when the prediction performance drops significantly on forget set, it indicates that the model may no longer reliably recognize or recall the underlying patterns from forget data, which is the goal of unlearning. Therefore, a drop in classification accuracy on the forget set is used as a proxy signal for unlearning effectiveness: the worse the model performs on the forget set, the more evidence that the model has "forgotten" that data.

4. Response to Q2 about the scalability with the number of classes overall and the number of classes in the forget set

(1)Scale with the number of classes overall. We empirically validated the scalability of our method on ImageNet-100, a more challenging and diverse dataset with 100 classes, using the ViT-Base backbone. We demonstrate the results under the class-wise forgetting setting. As shown in Table R6, our method maintains competitive unlearning effectiveness (UA, MIA-Efficacy) while preserving model utility (RA, TA), and keeps the Avg. Gap low. This indicates that our approach scales well to large-class settings without modifying the model weights, which is particularly appealing when model retraining is impractical.

Table R6: Performance overview of various MU methods for image classification in class-wise forgetting scenario on ImageNet-100 using ViT-Base.

(2) Scale with the number of classes in the forget set.

To evaluate how our approach scales with the number of classes in the forget set, we conduct experiments on ImageNet-10 using the ViT-Base backbone. Specifically, we measure performance as the model is tasked to forget 1, 2, and 3 classes simultaneously under the class-wise forgetting setting. The results can be found in Table R7.

Table R7: Performance overview of various MU methods for image classification in class-wise forgetting scenario on ImageNet-10 using ViT-Base with different number of forgetting classes.

5. Response to paper formatting.

Thank you! We will correct these formatting errors.

Dear Reviewer Wzpy,

Thank you for taking the time to review our paper. We sincerely appreciate your thoughtful and constructive comments. As the discussion deadline approaches, we would like to kindly check whether our response has sufficiently addressed your concerns. If you have any additional feedback or questions, we would be glad to further discuss and provide clarification.

Best regards,

Authors

We sincerely thank Reviewer eoHp for the thoughtful and constructive feedback. Below, we provide detailed responses to each of the key questions raised.

1. Response to W1 about the insufficient motivation for data-based unlearning

We thank the reviewer for raising this important concern. While we acknowledge the vulnerability of data-based methods under white-box adversaries (Appendix H), such limitations are not unique to our approach. Model-based unlearning methods have also been shown to be ineffective under relearning or adversarial attacks. Unlike model-based unlearning that require modifying model parameters, our data-based method operates purely at the input level. This design offers several practical advantages:

(1) Preserved utility: By not altering model weights, our method maintains the model’s general-purpose utility and downstream performance of the original model, which is valuable in dynamic environments where the model may be reused for tasks unrelated to the forgotten data.

(2) Memory efficiency: Our input-agnostic forget vector avoids updating high-dimensional model weights. This also avoids the need to store multiple model checkpoints, which is often required by model-based methods. Additional details regarding efficiency analysis are provided in our next response.

(3) Compositional unlearning: New forget requests can be fulfilled via simple arithmetic over class-wise forget vectors, reducing the optimization burden to just a few scalar coefficients, enabling plug-and-play unlearning. Such flexibility is not available in model-based approaches.

(4) Black-box model compatibility: Since we only perturb the input, our method naturally supports black-box settings where model parameters are inaccessible.

2. Response to W2 about the missing discussion of computational savings

Thank you for raising this point. We would like to clarify that supporting measurements for the forget vector (FV) are provided in Appendix F (Table A2). Moreover, the computation cost is further reduced via compositional unlearning, where only scalar coefficients need to be optimized. To highlight this, we include additional efficiency analysis for Compositional Unlearning via Class-wise Forget Vectors (CU-FV) in Table R1. Besides, we conducted additional experiments on a larger backbone ViT-Large in the scenario of random data forgetting with a specified forget ratio of 10% on ImageNet10. As shown in the Table R1, CU-FV is up to 19× faster than FV.

Table R1: Performance and efficiency overview of various MU methods under random data forgetting scenario on ImageNet-10 using ViT-Large, across the metrics: UA, RA, TA, MIA-Efficacy, Avg. Gap vs. Retrain, run-time efficiency (RTE) and parameter number (Param.#), where RTE is measured in minutes and M refers to Million.

3. Response to W3 about the empirical support for compositional unlearning

Thank you for your thoughtful feedback. We acknowledge that in some cases, directly trained forget vectors may outperform the compositional unlearning in preserving model utility. This reflects a trade-off between precision and efficiency. However, CU-FV offers unique practical benefits that are not captured by standard FV as follow:

(1) CU-FV significantly reduces computation time by optimizing only a small set of scalar weights (e.g., 10 parameters for random data forgetting on ImageNet10), rather than full input-level perturbations (which scale with image size), as shown in the Table R1 in Response 2.

(2) CU-FV enables efficient editing of unlearning requests in real-time systems, where a single set of class-wise FVs can serve multiple "random data forgetting" requests.

A clear computational benefit to compositional unlearning can be found in the Table R1, where we observe that CU-FV achieves up to 19× faster efficiency compare to FV. Such finding highlights CU-FV’s utility as a lightweight and flexible solution in practical deployment where multiple or large-scale unlearning requests may arise dynamically. In addition, to further demonstrate the lightweight nature and flexibility of compositional unlearning, we also report the RTE and Param.# on ImageNet10 using ViT-Large with different forgetting ratios (10%, 20%, 30%) in Table R2.

Table R2: Comparison of runtime efficiency (RTE) and parameter number (Param.#) between Forget Vector (FV) and Compositional Unlearning (CU-FV) under different forgetting ratios on ImageNet10 using ViT-Large.

4. Response to W4 about the limited experimental scope

(1) On focusing solely on image classification (not generative models).

We agree that MU for generative models is important and emerging, but it falls outside the scope of our work. Our method is designed for classification tasks, applying input-level perturbations without altering model weights. Extending this to generative models (e.g., diffusion or language models) involves fundamentally different challenges. For instance:

(a) In text-to-image diffusion models, the conditioning is based on text prompt, not images, making our forget vector framework inapplicable.

(b) In large language models, both the input and task (e.g., text generation) differ fundamentally from classification.

Thus, unlearning for generative models is a distinct, complex challenge requiring a different formulation. While important, it is beyond the scope of this work, which focuses on efficient, data-based unlearning for classifiers.

(2) On robustness to counter-unlearning attacks.

(a) Relearning: While recent studies (e.g.,[A]) shows model-based unlearning can be vulnerable to relearning attacks due to residual memory, our input-level unlearning approach fundamentally differs. By applying perturbations directly to inputs, we decouple unlearning from model parameters. We validate this by fine-tuning a ViT-Base model (trained on ImageNet10) using 10% of test data (including forget and retain classes) and re-evaluating the forget vectors. As shown in Table R3, the model maintains its ability to forget the targeted forget data and recognize the retained data, with only minor metric shifts, demonstrating the robustness of our method even under post-unlearning retraining.

Table R3: Performance comparison of the original and relearned ViT-Base models on ImageNet10 to assess the persistence and robustness of the forget vector.

(b) Quantized model. We applied post-training 8-bit quantization to the original model, ensuring classification accuracy on clean inputs remained comparable. To test robustness, we reused the original forget vectors on the quantized ViT-Base model trained on ImageNet10. As shown in Table R4, the quantized model still effectively forgets target classes and preserves recognition of retained data, indicating that our input-level unlearning approach remains robust and effective under quantization.

Table R4: Evaluation of forget vector effectiveness on quantized ViT-Base Models in the class-wise forgetting scenario.

(3) On scalability across different unlearning ratios.

We further evaluated our method under different forget ratios (10%, 20%, 30%) by performing random data forgetting on ImageNet-10 with a ViT-Base backbone. As shown in Table R5, our approach consistently suppresses forgotten data while preserving high accuracy on retained data, demonstrating robustness and scalability across varying unlearning demands.

Table R5: Scalability of various unlearning methods across varying forgetting ratios on ImageNet10 using ViT-Base.

5. Response to Q1 about interpretable patterns.

Thank you for the question. We did examine the perturbation patterns of the FVs in pixel space for each unlearned class. However, we found that the raw pixel-level perturbations do not exhibit consistent or interpretable visual patterns across different classes. To illustrate the spatial influence patterns induced by the FVs, we thus investigated this from an input saliency perspective (Grad-CAM); see Figure 4 of the main paper and Figures A4 and A5 in Appendix G.

6. Response to typos.

Thank you for pointing out the typos and phrasing inconsistencies, and we will correct them.

Dear Reviewer eoHp,

We sincerely appreciate your valuable time and constructive comments on our submission. As the discussion deadline approaches, we would like to kindly check whether our response has sufficiently addressed your concerns. If you have any further feedback or questions, we would be happy to engage in additional discussion and provide clarification.

Best regards,

Authors