No Training Data, No Cry: Model Editing without Training Data or Fine-tuning

I sincerely appreciate your thoughtful rebuttal together with additional experiments that are related to my concerns. The overall presentation was remarkably improved (although there is still room for improvement in terms of figures and tables)! Besides, the authors addressed concerns about the validation scope and reliance on external data.

Therefore, I raised my rating from 3 to 5.

• I think the biggest contribution of this work is a novel theory, and I wanted to grasp the applicability of this kind of theoretical analysis to the broader setting where the model does not use batch normalization (non-BatchNorm architectures are very common for modern neural networks).
• Although the authors provide an experiment with Swin-Transformer and I appreciate it, there is no theoretical implication for non-BN architecture.
• Also, I still wouldn't be so sure about the significance of improvement. Although the authors claim this, the trade-offs between accuracy and efficiency (Tables 1 and 2) are not so significantly improved compared with the baseline to my eyes.

Anyway, I am open to further discussion upon the extended deadline.

Thank you,

Reviewer d7ey

We thank the reviewer for their time and finding value in our theoretical analysis of BatchNorm at inference.

Based on their comments, we have made changes to the manuscript to enhance clarity and have provided additional results on unlearning for Vision Transformers.

We first wish to clarify a statement in the summary of the reviewer:

"The authors hypothesize that High Fidelity (HiFi) components..."

We wish to clarify that we are the first work to define and introduce the notion of HiFi components in Lines 228-230 in Section 4.1. We hypothesize that such channels are few in well trained models in Hypothesis 1 in lines 285-288.

We now address the individual concerns raised by reviewer d7ey.

W1: Limited applicability of the proposed method

In this paper, there are several proposed methods for model editing: RowSum Heuristic described in Lines 277-280 for identifying HiFi components; Algorithm 2 (BNFix) in Section 5 and Theorem 2(Weight compensation) in Section 6 as an alternatives to retraining in our setting; A novel analysis, Theorem 1, of BatchNorm at inference; CoBRA algorithms, Algorithm 3, for editing models with complex interconnections without training data or the loss function. As noted by reviewer ZvuL, the proposed method can be applied across areas such as continual learning and explainability but is beyond the scope of this work and remains open for future explorations.

Although ResNet models are still popular in some cases, given that Vision Transformer (ViT) ... compared to previous work on model editing [1]

The correlation based techniques and the CoBRA algorithms developed in Section 6 are readily applicable to transformer based models. [1] requires access to the loss function to perform component attribution and would not be applicable to the setting in this work.

Could the insights provided in this work have some implications for the transformer-style models?

As per your suggestion, we have added Appendix B.8 to describe how to edit QKV and MLP layers in transformer based vision models in the updated manuscript along with experiments on classwise unlearning on Swin-Transformer models.

Limited validation scope

We provide emperical validation across multiple tasks, pruning and unlearning on CIFAR10, CIFAR100, and ImageNet in Section 7 in Tables 1 and 2. The datasets we use in our experiments match those of our state of the art baselines [5, 6], which investigates machine unlearning (specifically classwise unlearning) and structured pruning for image classification. The methods proposed in [2] require multiple pre-trained models (or the ability to train new models on desired data subsets) and would not be applicable in our setting. Technique [3] would require access to training data and the loss function, making it unsuitable for comparison in this setting.

Insufficient empirical advantage

We would like to highlight the empirical advantage of our work on both pruning and classwise unlearning.

Pruning On CIFAR10 and CIFAR100 datasets, our method achieves 2.8x improvement in sparsity over the nearest baseline for ResNet50 without finetuning, and outperforms L2-based pruning with finetuning on synthetic data, as we show in Table 1 in Section 7.

Unlearning On ResNet50 trained on CIFAR10, we achieve superior Forget Class accuracy than the baselines proposed in [5] without any fine-tuning, whereas the baselines require multiple epochs of fine-tuning; we refer the reviewer to Table 2 in Section 7 of the manuscript for further details.

Reliance on external data (through distributional access)

Our method is inspired by works such as Dreaming to Distill [4], which motivates the use of samples drawn from a similar distribution to overcome concerns of privacy and security. The methods proposed in [2] require multiple pre-trained models (or the ability to train new models on desired data subsets) and would not be applicable in our setting.

Could the authors provide an ablation study for the size of the external dataset used for proposals?

We refer the reviewer to Appendix B.1 for details of the external datasets and Figures 3 and 4. And Appendix B.3 for the study on the size of the synthetic dataset. We experiment with varying number of samples to conclude that 1500 synthetic samples are sufficient for BNFix and use the same number of samples for computing RowSum.

Bad presentation quality

We thank the reviewer for these corrections. We have made updates to the manuscript to reflect them.

References

1. Decomposing and Editing Predictions by Modeling Model Computation, Shah et al. 2024
2. Editing Models with Task Arithmetic, Ilharco et al. 2024
3. Decoupling the Class Label and the Target Concept in Machine Unlearning, Zhu et al. 2024
4. Dreaming to Distil, Yin et al, 2021.
5. Model sparsity can simplify unlearning, Jia et al, 2023
6. DFPC, Narshana et al., 2023

Thank you for the active discussion and further clarification.

I had some misunderstandings about the applicability of your theory and the details behind the experiment.

• Authors' additional responses addressed my remaining concerns in terms of the impact of theory and practical benefits.
• I would strongly recommend the authors 1) clearly denote the training cost comparison such as total runtime in the tables so that they highlight the training-free characteristics (which are distinctive from the baseline) and 2) insert the result for Swin Transformer within the 10-page main body of paper rather than appendix upon camera-ready revision.
  • Also, I believe the authors will further improve the presentation quality (increasing the font size in figures/tables, adjusting margin/space, and so on) upon camera-ready if this paper is accepted.
• Overall, although I still see some weaknesses, e.g., marginal improvement and limited scope of validation, other concerns that I had are all addressed by authors' sincere refute and I believe the paper has been significantly improved through the rebuttal period.

Based on this, I am raising my score from 5 to 6, and increased all the soundness/presentation/contribution items by one point.

The reviewer d7ey sincerely appreciates the authors' heartfelt discussion.

We thank the reviewer for their detailed feedback, and for appreciating the distributional approach we used, as well as the potentially broad applicability of our approach.

Based on their comments, we have made changes to the manuscript to enhance clarity.

We now address the individual concerns raised by reviewer ZvuL.

W1 A: The main weakness of this paper is the limited understanding of how keeping the HiFi part results in keeping the knowledge of learned models. Otherwise, how tuning the HiFi part results in forgetting the specific learned knowledge.

In this work, the 'knowledge' refers to the data distributions learned by the model - thus, by the definition of HiFi components (Section 4.1, lines 245-288), keeping HiFi channels in well-trained models ensures that the learned distributions are maintained. The reconstruction error based formulation proposed in Section 4 is principled and rigorously derived. Thus, by maintaining the HiFi components, it follows that the distributional information regarding the entire data distribution (for pruning), or the remain classes (for unlearning) is maintained.

W1 B: At the conceptual level of understanding, it is quite convincing that the components showing similar distributional behaviors with the layer outputs are probably the crucial parts of the knowledge. However, it is not guaranteed theoretically.

We point out that the knowledge learned by the model is the distribution generating the data, and as such, retaining HiFi components ensures that the model's knowledge is retained. Since retaining the HiFi components are rigorously derived from the minimization of the layer-wise output reconstruction error, our CoBRA-P and CoBRA-U algorithms (Section 6) are principled, theoretically motivated approach that ensures that knowledge is retained (CoBRA-P) or removed (CoBRA-U) as needed.

W2: Quality of presentation

We thank the reviewer for the feedback. We have addressed the typographical errors in the work, and have fixed the size of images in Figure 5. Additionally, we have made other improvements to the text, highlighted in blue.

W3 A: Although the authors have provided the 'Related Work' part in the Appendix ... in-depth analysis of the prior works investigating the importance of weights or sensitivity measures of weights should be considered.

The setting of our work, wherein we need to edit models (specifically for pruning and classwise unlearning), without access to the training data or loss function, is unique, with very few technically similar works available. We point out to the reviewer our significant literature survey into structured pruning methods, most of which analyze the importance of weights. For instance, in Appendix A.3, we discuss works that use gradients to measure weight importance for structured pruning [Molchanov et al, 2019], Weight Norms [Li et al, 2019], reconstruction error [Luo et al, 2017], feature map ranks [Lin et al, 2020], and a variety of other methods (see the survey [Hoefler et al, 2021]). In Appendix A.2, we note [Wang et al, 2022], which uses class-discriminative scores for unlearning in the federated setting. However, we have clarified Appendix A from Lines 894 to 906 to further highlight different measures of weight importance used in the literature.

W3 B: I think that HiFi is another viewpoint to measure the importance of weights so that it has the potential to show further impact on continual learning (also without data of the past tasks) and explainability.

We are glad that you point out that our notion of HiFi components is applicable other important model editing tasks. However, the scope of this work is limited to using HiFi components to combine structured pruning and classwise unlearning under the model editing umbrella.

Q1: In the "What is Model Editing" part on line 176, to my understanding, 'B' is the number of components (not an individual weight, but a group of weights) in the model. Therefore, the equation, |θ|−B, looks wrong because |θ| is commonly used for the number of weights, not components. Would you clarify the equations?

$|\theta|$ should indeed represent the number of components in the model. We have amended the main document by defining $C_{total}$, the total number of components in the network in place of $|\theta|$. The change is highlighted in blue.

The reviewer has noted the strengths of this work and raised valuable concerns which we have adressed in our rebuttal.

• We have made several corrections to enhance the clarity in presentation
• We have added additional context in the Related work (Appendix A) to incorporate the feedback of the reviewer
• Addressed the individual weaknesses in the General Rebuttal and the Respose to the reviewer

With the announcement of the extension of the review period, and several modifications to improve the draft, we hope to engage with the reviewer in a discussion to address their concerns in raising their score for this work.

Dear Reviewer ZvuL,

We would like to remind you that the discussion period has been extended and there are 3 days left in the author-reviewer discussion period. We are eager to engage with you to address your concerns. We have posted our General Response, and individual response with a summary.

Q1: In Section Introduction, How do photos from a personal device constitute samples of a large collection of photos having similar distributions?

Our method is inspired by works such as [2], which motivates the use of samples drawn from a similar distribution to overcome concerns of privacy and security. Photos from a personal device capture the same underlying distribution as a training set, which is a large collection of curated images. For example, a large repository of cat images and the photos of cats from a cat owner’s device both capture features relating to cats. This is one example of distributional access, in Section 7, we use 1500 synthetic samples from generative models as described in Appendix B.1.

Q2: In Figure 2, we show the relative reconstruction error after removing filters from a selection of layers of a ResNet50 trained on CIFAR10". Could you explain how to get Fig. 2 in detail, which is a key assumption in this paper?

In Figure 2, we compute the distributional dissimilarity (Equation 3) between each input contribution to a layer (post-activation features generated by filters in the previous layer), and the aggregate feature map (which is the sum of the input contributions). We have amended the manuscript to reflect this difference in Lines 260-268, and have clarified the plots to highlight which components are considered HiFi. We observe that only between 5 and 30% of filters are HiFi, which motivates our model editing strategy: identify HiFi components and keep them (pruning) or remove them (unlearning).

Q3 A: The introduction of HiFi components and the section of BNfix seem disjointed. Could you provide a clearer connection between these two concepts

The recipie for effective model editing described in Algorithm 1 requires editing and recovery which are independent aspects. Identifying HiFi channels addresses the challenge of finding components responsible for prediction in the setting of this work - editing models with complex interconnections without the training data or loss function. This setting also necessitates alternatives for retraining models that do not require the loss function or training data. BNFix is identified as a suitable replacement for retraining in this setting.

Q3 B: explain why only BN's statistics are fixed?

The BNFix algorithm can also be derived by analyzing the expected reconstruction error after a BatchNorm layer; we have added this derivation in Appendix E. In the main work, we present Theorem 1 as it quantifies the effect of adjusting the BatchNorm statistics on the loss function, as opposed to analyzing the local behaviour.

Q4: Is there anything additional information that needs to be stored during training time for the proposed methods to work?

No. The strength of our proposed technique is that our procedures are agnostic to the training procedure of the models. Our techniques can be applied to pretrained models and do not require storing any additional information at training time.

Q5: The empirical evidence is primarily ... evaluate the methods on more datasets and tasks.

Our technique is the first to tackle the setting of model editing for pruning and class unlearning. Our empirical evaluation follows the settings proposed in the state of the art techniques for pruning and unlearning [3,4], which focus on standard image classification datasets. More complicated tasks are the subject of our future investigations.

Q6: There are many citation errors ...

We thank the reviewer for noting these corrections, we have incorporated these changes into manuscript.

References

1. How does BatchNorm help Optimization Santurkar et al., 2018
2. Dreaming to Distil, Yin et al., 2021.
3. DFPC: Data flow driven pruning of coupled channels without data, Narshana et al., 2023
4. Model sparsity can simplify unlearning, Jia et al., 2023

We thank the reviewer for their time and noting the difficulty of the problem setting and our approach to identifying HiFi components novel.

Based on their comments, we have made changes to the manuscript to enhance clarity with changes highlighted in blue.

We now address the individual concerns raised by reviewer gwP1.

W1 A: While the ... technical novelty of the RowSum heuristic and BNFix algorithm appears limited.

RowSum: This work is the first to propose the notion of HiFi components in lines 228-230 to identify components that are responsible for making predicitions in a challenging setup- in models with complex interconnections without access to training data or a loss function. The RowSum heuristic is proposed as a feasible technique in this setting to identify HiFi channels. As also noted by reviewer ZvuL, this technique is broadly applicable beyond the problems of pruning and editing.

BNFix: As noted as a strength of this work by reviewer d7eY, this work is the first to rigorously analyze BatchNorm at inference utilizing advanced tools such as Fact 1 and state the update as an algorithm in Algorithm 2. As demonstrated by our experiments in pruning and unlearning presented in Tables 1, and 2, BNFix is an effective replacement for retraining in our setting. Additionally, our work is also the first to highlight the impact of rectifying BatchNorm statistics on Unlearning, as shown in Table 2. Additionally, we derive other algorithms (Algorithm 4) based on our anaysis in Appendix B.9.

W1 B: There are many papers proposing to update BN's parameters, a similar strategy to the one in this paper.

We wish to clarify that we do not update BatchNorm learned parameters $\gamma$ and $\beta$, only adjusting the means and variances $\mu$ and $\sigma$ as per Algorithm 2. We would like to emphasize the setting in which this update is proposed-editing without access to training data or loss functions. We kindly request the reviewer to mention any other papers that propose a similar strategy in this setting that we may have missed.

W2: The theoretical analysis focuses on ... might not be tight enough as a guarantee.

The focus of our theoretical analysis in Lemma 1 and Theorem 1 is not to obtain bounds that are strictly tight but to obtain viable procedures which can emperically validated. We would like to highlight that our bound is informative in explaining the improvement in the model’s accuracy after rectifying the BatchNorm statistics as shown in Appendix B.2 and B.5. In Appendix B.9, we discuss Algorithm 4, a variant based on our theoretical analysis in Theorem 1 and emperically verify the effectiveness of this variants, indicating that our analysis is informative. Moreover, the theoretical analysis using the largest eigenvalue of the Hessian follows standard practice, which assumes that the loss function is Lipschitz smooth, including other works analyzing BatchNorm, such as [1].

We address the questions in the subsequent comment.

References

1. How does BatchNorm help Optimization Santurkar et al., 2018

The reviewer has noted some of the strengths of this work and we have adressed their concerns in our rebuttal.

• We have made several corrections to enhance the clarity in presentation in the updated manuscript
• We have emphasized on the strengths missed by the reviewer and addressed weaknesses in the General Rebuttal and the Respose to the reviewer
• We have answered all of the questions raised by the reviewer in the Response to the reviewer

With the extension of the review period, and several modifications to improve the draft, we invite the reviewer to engage in a discussion to address their concerns in raising their score for this work.

Dear Reviewer gwP1,

We would like to remind you that the discussion period has been extended and there are 3 days left in the author-reviewer discussion period. We are eager to engage with you to address your concerns. We have posted our General Response, and individual responses(1, 2) with a summary.