A Fast Framework for Post-training Structured Pruning Without Retraining

Dear Reviewer 4NdE,

We thank you for your valuable feedback and that you find strength in our problem statement and methodology. Sorry for any misunderstanding caused by our presentation and layout. We respond to your comments below.

1. There are no experiments or citations to justify this claim (W1).

   • We extended Adaprune and gAP to directly apply layer-wise reconstruction on structured pruning, and experiments on ImageNet have shown that these methods cannot effectively alleviate the accuracy decrease caused by pruning (See Table 2). We will explicitly reference the results in the modified version.

2. The authors do not report the FLOPs and wall-clock time required (W2).

   • We previously thought that the report on actual running time was not very important, so it was included in appendix A.1. We will consider presenting this result in the main text in the revised version. In addition, we have reported the reduction in FLOPs, so no specific FLOPs were displayed.

3. The retraining-based pruning baselines selected are weak (W3).

   • Our paper focuses on pruning without retraining, so we only compare the $L_1$-norm based method to show the gap with retraining-based methods. We reproduced the pruning algorithm in [1] as $L_1$-norm pruned baseline and we retrained 100 epochs after pruning.
   • We have added the results of [2] and [3] in the revision. By the way, DFPC ([4]) is proposed as a data-free method retrained, it does not generalize to higher sparsity. Moreover, it retrained the model when reporting results on ImageNet. This method does not match our settings and therefore was not selected for the baseline.

4. More experimental details and sample sensitivity experiments (Q1& Q2).

   • We randomly sample indexes to obtain random samples. And the selection of samples only causes minor changes in the results. The results of ten calibrations after pruning ResNet-50 on ImageNet are {72.51, 72.57, 72.64, 72.56, 72.65, 72.70, 72.58, 72.60, 72.56, 72.54}
   • We use global pruning and do not set hierarchical pruning rates. And the third last line in Algorithm 1 minimizes the reconstruction loss.

5. Explanation about the state "the sparsity of different layers is inconsistent in a pruning group" (Q3).

   • It is our motivation for adopting sparse penalty. Channels with the same index in different layers in a group will be pruned together. However, these channels do not all have small weight norms. The combined importance of small-norm channels and large-norm channels may also be below the pruning threshold. And pruning them causes information loss. Therefore, we impose a sparsity penalty to make the pruned channels have consistent sparsity.

[1] Pruning filters for efficient convnets. Li et al. ICLR 2017.
[2] Neural Pruning via Growing Regularization. Wang et al. NeurIPS 2021.
[3] DepGraph: Towards Any Structural Pruning. Fang et al. CVPR 2023.
[4] DFPC: Data flow driven pruning of coupled channels without data. Narshana et al. ICLR 2023.

Dear Reviewer 2uwp,

We are grateful for your time to review our paper and that you find value in our work. We respond to your concerns below.

1. Explanation about imbalanced updates and over-optimization.

   • The pruned model lacks the activation values of some channels, and we can only reconstruct the activation values of the remaining channels, which introduces error. We consider holistic layer reconstruction loss to reduce error accumulation. However, holistic reconstruction loss causes gradients of different layers to be unbalanced and causes the model to update incorrectly. For example, the first layer is optimized by the reconstruction error of itself and all subsequent layers, but the last layer is optimized by the reconstruction error of itself only. So, we need to balance the updates of different layers.

2. Explanation about settings and results in Table2

   • The results of AdaPrune and gAP are produced by our extended version of structured pruning. We use the same framework and importance evaluation, prune the same structure and yield RP and RF factors. The difference lies in the calibration process, our method is more efficient in recovering the accuracy. The other two non-retraining methods are designed for structured pruning, so we do not need to modify. Neuron Merging merge parameters of neurons and does not require data for calibration. However, it yields the worst accuracy. Reborn Filters concentrates information on newly generated filters before pruning. This process requires calibration data. It can fine-tune or do nothing after pruning.

3. Explanation of the results for ResNet18 on CIFAR100 in Figure 5

   • CIFAR-100 is more difficult than CIFAR-10. ResNet-18 trained on CIFAR-100 contains less redundancy, and small amounts of pruning can result in significant accuracy reduction.

Besides, we have revised unclear expressions in an updated version.

Dear Reviewer aCSe,

We are grateful for your valuable feedback on our paper and that you find strength in our framework. We present our response to your comments below.

1. Contributions of our paper (W1).

   • Our main contribution is the post-training structured pruning framework and two-stage reconstruction method. We chose the commonly used weight criteria and group importance as the implementation, but it is not the focus of our paper. The key to achieving excellent results is our reconstruction method rather than group importance. As evidence, we extended AdaPrune and gAP using the same group importance but their performance was inferior to our method.

2. Motivation of our paper (W2).

   • Privacy or commercial concerns: While full business data is difficult to obtain, companies may agree to provide small samples. Moreover, our focus is on the amount of data, not whether it is labeled
   • Computational cost of retraining: The high cost of retraining is also one of the obstacles to the practical application of pruning technology. This is also the motivation for our retraining-free framework.

3. Could authors provide a more comprehensive experiment report (W3)?

   • Figure 3 provides intuition for why sparsity penalty works. It reduces the information loss caused by pruning. Moreover, Figure 5 provides ablation experiments to demonstrate the role of reconstruction and sparsity penalty. Besides, Figure 6 shows that balanced updates are key to holistic reconstruction loss success. We think we have provided a more comprehensive experiment. We sincerely hope to receive your specific experimental suggestions. By the way, we provided more experiments in the appendix and code in the supplementary material, which may help you understand our approach.

Dear Reviewer eiUM,

We are grateful for your valuable feedback on our paper and that you find value in our work. We present our response to your comments below.

1. About baseline selection

   • Our paper focuses on pruning without retraining, so the pruning methods with the same settings is the focus of our comparison. We have compared rich and representative baselines. As for red++ ([4]), since it does not provide code and we failed to reproduce their results, this method has not been selected as a baseline. In addition, in order to show the gap with the retraining method, we selected the simple $L_1$-norm based baseline in Table 1 and Table 2, which we retrained 100 epochs after pruning. We have added the results of [1] and [2] in the revision.

2. Contributions and significant performance improvement of our paper

   • Our main contribution is the post-training structured pruning framework and two-stage reconstruction method. Compared with methods that do not require retraining, our method has significant improvements, especially on the large-scale dataset ImageNet. Other reviewers have appreciated our performance improvements.

3. Explanation for "no retraining"

   • No retraining means avoiding costly training processes after pruning rather than no data. Our framework replaces tedious retraining with a simple and fast calibration process. Moreover, a small amount of calibration data is a realistic setting.

4. Dataset and model setup

   • Dataset: CIFAR-10/100 and ImageNet datasets are the common settings for pruning papers.
   • Dataset: We use the same settings as other papers for retraining-free pruning. And we use VGG-19 rather than VGG-16 for consistent settings. Besides, we provided results for more models in Appendix A.3.

5. Explanation about and over-optimization.

   • Holistic reconstruction loss causes gradients of different layers to be unbalanced. For example, the first layer is optimized by the reconstruction error of itself and all subsequent layers, but the last layer is optimized by the reconstruction error of itself only. So, we need to balance the updates of different layers.

6. About pruning ratio settings.

   • Our implementation using global pruning rate does not allow precise control of model parameters and FLOPs. We have used the same group-wise pruning wherever possible. We extended AdaPrune and gAP to use group-wise pruning. However, Neuron Merging and Reborn Filters are difficult to be compatible with group pruning, so we used the original method.

[1] Neural Pruning via Growing Regularization. Wang et al. NeurIPS 2021.
[2] DepGraph: Towards Any Structural Pruning. Fang et al. CVPR 2023.
[3] DFPC: Data flow driven pruning of coupled channels without data. Narshana et al. ICLR 2023.
[4] Red++: Data-free pruning of deep neural networks via input splitting and output merging. Yvinec et al. TPAMI 2022.