QP-SNN: Quantized and Pruned Spiking Neural Networks

Thank you very much for your recognition of our work and your insightful review. We will address each of your questions.

W1: The selection process for pruning rates appears arbitrary and lacks systematic justification.

Response to W1: We acknowledge that the setting of pruning rates is arbitrary, but no tedious manual design is required. The suitable pruning rates of each layer can be determined through the orthogonal NAS-based approach. However, we decide not to do this, aiming to maintain the simplicity and practicality of our method. More importantly, without requiring tedious manual design and parameter search, QP-SNN achieves state-of-the-art performance compared to advanced methods. This further demonstrates the effectiveness of our method. Of course, if the automated search mechanism is introduced into our method, the performance can be further improved. This will be an important focus of our future work.

Q1: How does the computational cost of calculating singular values in the SVS criterion compare to existing pruning criteria? Is there a significant overhead during training?

Response to Q1: Currently, the advanced methods in SNNs [1,2] perform pruning during the training process. In contrast, the proposed SVS criterion is applied only once before pruning, which does not involve the training process. More specifically, we first obtain a well-trained quantized model. Then, we use the SVS-based pruning criterion to assess the importance of each channel and prune redundant kernels based on the pruning rates. Finally, we fine-tune the pruned model. The SVD computation is not required during the fine-tuning stage, so the training efficiency is not affected.

[1] Shi, Xinyu, et al. Towards energy efficient spiking neural networks: An unstructured pruning framework. ICLR 2024.

[2] Li, Yaxin, et al. Towards efficient deep spiking neural networks construction with spiking activity based pruning. ICML 2024.

Q3: How sensitive is the method to the choice of ε in the SVS-based pruning criterion?

Response to Q3: ε is the threshold used to distinguish significant singular values, and its selection should be based on the specific task demands. A higher ε value zeros out small singular values, which helps reduce noise and prevent overfitting, making it suitable for scenarios with high data noise. A lower ε value preserves more fine-grained features, which helps prevent information loss, making it ideal for tasks that demand high accuracy. In summary, setting ε too high may overlook subtle linear relationships in the matrix, while setting it too low may introduce redundant information like noise. In this work, we set ε to a relatively small value to retain as much useful information as possible.

Q2: What is the impact of the proposed methods on the network's spike sparsity?

Response to Q2: To analyze the impact of our method on the spike sparsity of the neural network, we compare the firing rates of the original SNN and QP-SNN across each layer. The comparison is performed on the CIFAR-10 dataset using the VGG-16 architecture and the CIFAR-100 dataset using the ResNet20 architecture, and the results are summarized in the table below.

From the comparison, we observe that the firing rate of QP-SNN is marginally higher than that of the original model. Despite this slight increase, our method significantly reduces storage and computational costs, since it achieves extreme model compression by unifying both pruning and quantization. For example, on the CIFAR-10 dataset, the original model size is reduced by 98.74%, SOPs are decreased by 78.69%, and power consumption is lowered by 77.45%.

W2: This paper is only verified on the simple computer vision classification task.

Q4: How does the approach perform when applied to other types of neural networks beyond classification tasks, such as object detection?

Response to W2 &Q4: Thank you for pointing out this issue. Our method can be applied to more complex vision tasks like object detection. The reason we choose the simple classification tasks is to facilitate a comprehensive comparison with advanced compression methods in SNNs [1,2]. To address your concern more effectively, we have conducted additional experiments for the object detection task. We select two remote the sensing dataset SSDD [3]. The SSDD dataset specifically focuses on ship detection imagery acquired through synthetic aperture radar. In our experiments, we adopt the YOLO-v3 detection architecture with ResNet10 as the backbone. During training, we perform the pruning operation on the backbone and employ the SGD optimizer with a polynomial decay learning rate schedule, initializing the learning rate at 1e-2 and training for 300 epochs. Results are shown in the table below, which fully validates the effectiveness of QP-SNN for complex vision tasks.

[1] Shi, Xinyu, et al. Towards energy efficient spiking neural networks: An unstructured pruning framework. ICLR 2024.

[2] Li, Yaxin, et al. Towards efficient deep spiking neural networks construction with spiking activity based pruning. ICML 2024.

[3] Cheng, Gong, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE 105.10 (2017): 1865-1883.

Q: The authors have added experiments on the object detection task, but the selected SSDD dataset is relatively simple, focusing primarily on ship detection for a specific scenario. It is recommended that the authors consider validation on more challenging datasets, such as those containing multiple categories and complex scenes.

Thank you for your insightful feedback and the time you dedicated to evaluating our work. To better answer your question, we have conducted additional experiments on the NWPU VHR-10 dataset[1], which contains 10 object categories (airplane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge and vehicle) in complex scenes with various backgrounds.

Experimental details: We adopt the YOLO-v3 detection architecture with ResNet10 as the backbone. During training, we perform the pruning operation on the backbone and employ the SGD optimizer with a polynomial decay learning rate schedule, initializing the learning rate at 1e-2 and training for 300 epochs. All our experiments are conducted on four NVIDIA A100 GPUs, and the code has been included in the supplementary materials for reproducibility.

Experimental results: The results are summarized in the table below. Notably, on this more complex object detection dataset, QP-SNN still achieves a significant reduction in model size while maintaining satisfactory detection performance. This fully demonstrates the potential of our approach to extend to more challenging tasks. We have added the results and visualizations on this dataset in the revised manuscript.

[1] Cheng, Gong, Junwei Han, and Xiaoqiang Lu. "Remote sensing image scene classification: Benchmark and state of the art." Proceedings of the IEEE 105.10 (2017): 1865-1883.

W3: The results that the saving of synaptic operation is missing.

Q1: What is the effectiveness of this work in saving synaptic operation in SNN inference?

Response to W3 and Q1: Thank you for your valuable suggestions and guidance. We have added SOP comparisons of QP-SNN with full-precision uncompressed SNNs and related compressed studies.

• We first provide the comparison results between our method and full-precision uncompressed counterparts. As shown in the table below, QP-SNN demonstrates satisfactory performance under extreme compression ratios. For example, on the CIFAR-10 dataset, under the extreme connection ratio of 9.61%, QP-SNN reduces the model size by 98.74%, SOPs by 78.69%, and power consumption by 77.45%, while the accuracy decreases by only 2.44%. This trade-off between performance degradation and resource efficiency is highly advantageous in edge computing scenarios.

• We then add the SOP comparison results between our method with related compressed studies on the CIFAR-10 dataset. Experimental results are shown in the following table. It can be seen that QP-SNN exhibits competitive SOPs compared to compression work, while exhibiting extremely low model size due to quantization. Moreover, it is worth noting that the advanced works [1,2] focus on unstructured pruning, which typically achieves higher sparsity and performance but requires specialized hardware support. In contrast, our QP-SNN adopts uniform quantization and structured pruning, balancing the advantages of sparsity, performance, and hardware compatibility.

[1] Shi, Xinyu, et al. Towards energy efficient spiking neural networks: An unstructured pruning framework. ICLR 2024.

[2] Deng, Lei, et al. Comprehensive snn compression using admm optimization and activity regularization. TNNLS 2021.

[3] Li, Yaxin, et al. Towards efficient deep spiking neural networks construction with spiking activity based pruning. ICML 2024.

We sincerely appreciate the time you have dedicated to reviewing our manuscript and for providing such constructive feedback. We will address each of your concerns.

W1: The proposed technologies including the weight-rescaling strategy and the SVS-based pruning criterion do not fully utilize the unique characteristics of SNN. The technologies can also be used in ANN models.

Response to W1: As you pointed out, both proposed methods can also be used in ANNs. However, the proposed SVS pruning criterion is more suitable for SNN. Specifically, in SNNs, the SVS-based pruning criterion employs SVD on the spatio-temporal spike matrix, where the matrix values are constrained to a few discrete levels (for example, when $T=4$, the matrix values can only be {0, 1/4, 1/2, 3/4, 1}). Therefore, the advantage of SVS -based pruning in SNNs can be summarized as two aspects.

• First, SVS-based pruning criterion in SNNs offers significant computational efficiency advantages when considering the hardware deployment. SVD computations for continuous matrices typically require high-precision numerical processing, as matrix values are real numbers and can vary widely. In contrast, discrete matrices use a limited number of values, allowing for more compact data representation and reduced memory usage. This significantly improves computational efficiency during large-scale matrix decomposition.

• Second, SVS-based pruning criterion in SNNs demonstrates more stable kernel importance evaluation on discrete matrices. The SVD decomposition performed on continuous matrices is more sensitive to noise perturbations due to the high precision representation, which results in fluctuations in channel importance scores. In contrast, discrete matrices can more effectively suppress small noise changes due to their lower precision, leading to more stable importance evaluation.

W2: The comparison with SNNs that are not compressed is missing. Compared to these works, the accuracy degradation is still large.

Response to W2: Thank you for pointing out our issue. We have added a comprehensive comparison of QP-SNN with the corresponding uncompressed SNN.

• We acknowledge that our method exhibits accuracy loss compared to uncompressed SNNs. However, this performance degradation is a common challenge in the field of model compression. Therefore, QP-SNN that compresses the network by reducing both parameter bit-width and structures will inevitably bring the performance drop. Fortunately, QP-SNN demonstrates satisfactory performance under extreme compression ratios. For example, on the CIFAR-10 dataset, under the extreme connection ratio of 9.61%, QP-SNN reduces the model size by 98.74%, SOPs by 78.69%, and power consumption by 77.45%, while the accuracy decreases by only 2.44%. This trade-off between performance degradation and resource efficiency is highly advantageous in edge computing scenarios.

Despite the accuracy loss compared with uncompressed SNN, QP-SNN achieves state-of-the-art performance and efficiency among compressed SNNs. This can be observed from the following table. Therefore, QP-SNN underscores its potential for enhancing SNN deployment in edge intelligence computing.

[1] Chowdhury, Sayeed Shafayet, Isha Garg, and Kaushik Roy. Spatio-temporal pruning and quantization for low-latency spiking neural networks. IJCNN 2021.

[2] Deng, Lei, et al. Comprehensive snn compression using admm optimization and activity regularization. TNNLS 2021.

[3] Shi, Xinyu, et al. Towards energy efficient spiking neural networks: An unstructured pruning framework. ICLR 2024.

[4] Li, Yaxin, et al. Towards efficient deep spiking neural networks construction with spiking activity based pruning. ICML 2024.

Dear Reviewer Likb,

We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. We have provided detailed responses to your concerns, and we hope they have adequately addressed your issues. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you.

Best regards.

Thanks for the authors' reply. The supplementary experiments in rebuttal are still unconvinced. For example, why is the #SOP of VGG16 smaller than ResNet20? The input size of CIFAR10 and CIFAR100 is the same. The results are not correct. I will keep my score.

We are deeply grateful for your time in reviewing our manuscript. Your insights will be of great help to us in improving the quality of our paper.

W1: The ReScaW-based uniform quantization lacks innovation, as it is merely a combination of uniform quantization and weight scaling[1].

[1] Lin, Ji, et al. "AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration." Proceedings of Machine Learning and Systems 6 (2024): 87-100.

Response to W1: Yes, the ReScaW-based method is a combination of uniform quantization and weight scaling. However, we want to make a statement about its innovativeness and effectiveness. First is the innovation of the ReScaW-based method. Regarding the AWQ[1] you mentioned, although both AWQ and our work involve quantization and introduce scaling factors, there are significant differences between them, which can be summarized in two aspects.

• On the one hand, the problems they aim to solve are different. AWQ aims to identify and protect a small fraction (0.1%–1%) of salient weights, thereby reducing the quantization error of LLMs. In contrast, ReScaW focuses on improving the bit-width usage efficiency of vanilla uniform quantization, thereby addressing the reduced discrimination of quantized weights. The former emphasizes reducing quantization errors for a small fraction of salient weights, while the latter is dedicated to optimizing overall bit-width usage efficiency.

• On the other hand, their technical approaches differ markedly. AWQ employs an automatic search method during the training process to determine the scaling factor, with the search space defined based on the activation magnitude. In contrast, the ReScaW strategy utilizes weight distribution characteristics before quantization, such as maximum values, percentiles, and mean of the L1 norm. The scaling coefficient in our ReScaW does not involve optimization, automatic search, or activation values.

Second is the effectiveness of the ReScaW-based method. By combining uniform quantization with a scaling coefficient $\gamma$, it effectively addresses the performance degradation caused by inefficient bit-width usage in vanilla uniform quantization. To clearly illustrate the effectiveness of the ReScaW, we present the comparative results in the following table, where incorporating ReScaW into the baseline improves performance by 4.24%. More importantly, this scaling coefficient can be directly fused into the convolutional layer during inference. This simple method avoids the additional computation and storage overhead of complex strategies, which offers an efficient solution for resource-constrained edge deployments.

W2: The SVD-based pruning appears to be an application of SVD pruning in SNNs. There are many methods utilizing SVD decomposition or selecting important singular values for pruning. Please clarify the specific advantages of applying this in SNNs.

Response to W2: Yes, the SVD-based pruning method can also be applied to ANNs, but it has unique advantages in SNNs. More specifically, in SNNs, the SVS-based pruning criterion employs SVD on the spatio-temporal spike matrix, where the matrix values are constrained to a few discrete levels (for example, when $T=4$, the matrix values can only be {0, 1/4, 1/2, 3/4, 1}). If this pruning method is used in ANNs, it should be applied to full-precision feature maps accordingly. Therefore, the advantage of SVD-based pruning in SNNs can be summarized as two aspects.

• First, SVD-based pruning in SNNs offers significant computational efficiency advantages when considering the hardware deployment. SVD computations for continuous matrices typically require high-precision numerical processing, as matrix values are real numbers and can vary widely. In contrast, discrete matrices use a limited number of values, allowing for more compact data representation and reduced memory usage. This significantly improves computational efficiency during large-scale matrix decomposition.
• Second, SVD-based pruning in SNNs demonstrates more stable kernel importance evaluation on discrete matrices. The SVD decomposition performed on continuous matrices is more sensitive to noise perturbations due to the high precision representation, which results in fluctuations in channel importance scores. In contrast, discrete matrices can more effectively suppress small noise changes due to their lower precision, leading to more stable importance evaluation.

W3: The SVD decomposition method introduces additional time complexity. How much does it specifically affect the training time?

Response to W3: SVD on the spatiotemporal spike activity is applied only once, it does not impact the training process. Specifically, we first obtain a well-trained quantized model. Then, we use the SVS-based pruning criterion to evaluate the importance of each channel and prune redundant kernels based on the pruning rate. Finally, we fine-tune the pruned model; during this fine-tuning phase, SVD is not used again.

W4: The experiments are insufficient from several perspectives. Firstly, why use the 2021 SEW-ResNet and Spiking ResNet instead of the latest SNN models? Secondly, this method has only been tested on simple classification tasks; without testing on more complex downstream tasks. Thus, it is difficult to prove the potential applicability of this work.

Response to W4: Thank you for pointing out this issue. Our method can be extended to more complex architectures like the Transformer and applied to tasks such as object detection. The reason we choose the ResNet architecture and simple classification tasks is to facilitate a comprehensive comparison with advanced compression methods in SNNs [1,2]. To address your concern more effectively, we have conducted two additional experiments: (1) using the Spikingformer [3] architecture on the CIFAR-10 dataset, and (2) applying our method to an object detection task. The results of these two experiments are presented in the following.

• We first select the Spikingformer-4-384 structure [3] and validate it on the CIFAR-100 dataset. As shown in the table below, our method achieves a 87.93% reduction in model size, a 55.48% decrease in SOPs, and a 55.64% reduction in energy consumption, while maintaining performance of 76.94%. These results fully validate the effectiveness of QP-SNN for complex Spiking Transformer structures.

• We then conduct object detection experiments on the remote sensing dataset SSDD [4]. The SSDD dataset specifically focuses on ship detection imagery acquired through synthetic aperture radar. In our experiments, we adopt the YOLO-v3 detection architecture with ResNet10 as the backbone. During training, we perform the pruning operation on the backbone and employ the SGD optimizer with a polynomial decay learning rate schedule, initializing the learning rate at 1e-2 and training for 300 epochs. Results are shown in the table below, which demonstrate that QP-SNN achieves a reduction in model size by 88.85%, while increasing mAP@0.5 by 0.3% compared to the full-precision uncompressed model. This fully validates the effectiveness of QP-SNN for complex tasks.

In summary, through these two experiments, we demonstrate that our method remains effective on more complex architectures and tasks.

[1] Shi, Xinyu, et al. Towards energy efficient spiking neural networks: An unstructured pruning framework. ICLR 2024.

[2] Li, Yaxin, et al. Towards efficient deep spiking neural networks construction with spiking activity based pruning. ICML 2024.

[3] Zhou, Chenlin, et al. Spikingformer: Spike-driven residual learning for transformer-based spiking neural network. arXiv preprint arXiv:2304.11954 (2023).

[4] Wang, Yuanyuan, et al. A SAR dataset of ship detection for deep learning under complex backgrounds. remote sensing 11.7 (2019): 765.

W5: As an efficiency-focused study, why does the paper not present sufficiently robust efficiency metrics, such as OPs/SOPs, training time, inference speed, and energy consumption?

Response to W5: Thank you for pointing out this issue and for your valuable suggestions. Model compression aims to optimize efficiency during the inference phase, facilitating efficient deployment on resource-constrained devices. To this end, we have supplemented the key efficiency metrics of QP-SNN during inference, including model size, SOP, power consumption, the inference latency of the whole test dataset, and accuracy. Here, we present the results on the CIFAR-100 dataset with the ResNet20 structure. It can be observed that, compared with the uncompressed full-precision model, our QP-SNN shows a significant efficiency advantage while maintaining competitive performance.

W6: Why there is no presentation of pruning ratio statistics in the main text, which obscures the specific effectiveness of this aspect?

Response to W6: We sincerely apologize for the inconvenience caused to you in understanding our paper. As per your proposal, we have included the channel pruning rates for various network architectures and datasets in Appendix D. In the following, we select the VGG-16 network on the CIFAR-10 dataset as a representative case, providing detailed channel pruning rates along with the corresponding experimental results.

Under the above pruning rates, we present the performance of QP-SNN on the CIFAR-10 dataset and compare it with related studies. The experimental results are summarized in the table below. It can be observed that, under such a low pruning rate combined with quantization, QP-SNN achieves state-of-the-art performance and efficiency, underscoring its potential for enhancing SNN deployment in edge intelligence computing.

[1] Chowdhury, Sayeed Shafayet, Isha Garg, and Kaushik Roy. Spatio-temporal pruning and quantization for low-latency spiking neural networks. IJCNN 2021.

[2] Deng, Lei, et al. Comprehensive snn compression using admm optimization and activity regularization. TNNLS 2021.

[3] Shi, Xinyu, et al. Towards energy efficient spiking neural networks: An unstructured pruning framework. ICLR 2024.

[4] Li, Yaxin, et al. Towards efficient deep spiking neural networks construction with spiking activity based pruning. ICML 2024.

Dear Reviewer LMLx,

We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. As the author-reviewer discussion phase is drawing to a close, we would like to confirm whether our responses have effectively addressed your concerns. We have provided detailed responses to your concerns, and we hope they have adequately addressed your issues. If you require further clarification or have any additional concerns, please do not hesitate to contact us. We are more than willing to continue our communication with you.

Best regards.

W4: Secondly, this method has only been tested on simple classification tasks; without testing on more complex downstream tasks. Thus, it is difficult to prove the potential applicability of this work.

In addition to the SSDD dataset, we have also conducted additional experiments on the NWPU VHR-10 dataset[1], which contains 10 object categories (airplane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge and vehicle) in complex scenes with various backgrounds. We follow the same network architecture and experimental settings as conducted in the SSDD dataset. All our experiments are conducted on four NVIDIA A100 GPUs, and the code has been included in the supplementary materials for reproducibility.

The results are summarized in the table below. Notably, on this more complex object detection dataset, QP-SNN still achieves a significant reduction in model size while maintaining satisfactory detection performance. This fully demonstrates the potential of our approach to extend to more challenging tasks. We have added the results and visualizations on this dataset in the revised manuscript.

[1] Cheng, Gong, Junwei Han, and Xiaoqiang Lu. "Remote sensing image scene classification: Benchmark and state of the art." Proceedings of the IEEE 105.10 (2017): 1865-1883.

W1: Regarding the selection of the quantization scaling factor γ, this work employs three different sampling methods. Although experiments indicate that scaling with the 1-norm mean yields the best results, I believe there is still room for improvement, such as using a piecewise γ. I encourage the authors to provide further explanation.

Response to W1: Yes, your suggestion is correct. By using piecewise scaling factors, we can more precisely control the quantization levels of weights, thereby better capturing important information. For example, consider weights that follow a bell-shaped distribution. We can assign small scaling factors to weights concentrated around zero so that they are mapped to a larger range after quantization, thus using more allocated bitwidth. In contrast, we assign large scaling factors to the weights at both tails, so they are mapped to a small range and use less allocated bitwidth. However, this piecewise scaling factors method essentially results in non-uniform quantization, because employing different scaling factors in different intervals leads to uneven intervals of overall quantization levels.

This non-uniform quantization typically may achieve higher accuracy for a fixed bit-width, but it is difficult to deploy efficiently on general-purpose computing hardware like GPUs and CPUs. Therefore, we do not consider piecewise scaling factors in our method. In contrast, our proposed ReScaW strategy enhances bit-width utilization while maintaining the hardware-friendly characteristics of uniform quantization, offering a simple, effective, and practical solution.

W3: At the end of Algorithm. 1, fine-tuning is required after pruning. The specific details of this fine-tuning process should be provided.

Response to W3: After carefully reviewing the appendix, we find that the training details for the fine-tuning stage are missing, and some hyperparameters during training are incorrectly described. We apologize for these oversights and summarize the correct training and fine-tuning parameter setups as follows.

Q1: To my knowledge, the highest-performing architecture in the SNN field is the Spiking Transformer. Please discuss whether the proposed method can be effectively applied to the Spiking Transformer. If feasible, please provide some preliminary experimental results.

Response to Q1: Thank you for your valuable suggestion. To address your concerns, we present additional experiments using Spikingformer [1] as an example to demonstrate that our QP-SNN can be effectively applied to Spiking Transformer architectures.

Experimental details: We select the Spikingformer-4-384 structure and validate it on the CIFAR-100 dataset. The training process is as follows: we first train a 4-bit quantized Transformer with ReScaW strategy, and then we apply the SVS pruning criterion to identify and remove redundant convolutional kernels from the quantized Transformer, finally the pruned Transformer is fine-tuned to obtain the compressed version.

Experimental results: The results are summarized in the table below. As shown, the QP-SpikingTransformer achieves a 87.93% reduction in model size, a 55.48% decrease in SOPs, and a 55.64% reduction in power consumption, while maintaining the satisfactory performance of 76.94%. These results fully validate the effectiveness of QP-SNN for Spiking Transformers.

[1] Zhou, Chenlin, et al. Spikingformer: Spike-driven residual learning for transformer-based spiking neural network. arXiv preprint arXiv:2304.11954 (2023).

W4: In Table. 2 of the experimental section, the baselines for A-F are not consistent. I suggest the authors provide a unified ablation study. Additionally, regarding the compatibility issues of interest, I encourage the authors to conduct an ablation study on the compatibility of quantization and pruning.

Response to W4: The ablation experiments in Table 2 are divided into two groups with different baselines. Models in group A-B use only quantization, while those in group C-F use both quantization and pruning. The A-B comparison is designed to show that ReScaW improves the performance of quantized models, demonstrating its effectiveness even when used alone. The C-F comparison is designed to show the performance improvement of ReScaW and SVS for quantized and pruned SNNs.

To address your compatibility concerns, we conduct additional ablation experiments by switching the order of quantization and pruning. Experiments are performed on the CIFAR-100 dataset using a ResNet20 architecture. The results are summarized in the table below, from which two conclusions can be obtained.

• First, the proposed ReScaW and SVS can improve both performance, regardless of the order in which they are applied, leading to an 1.83% improvement in PQ-SNN and a 4.46% improvement in QP-SNN.
• Second, QP-SNN achieves the highest performance (surpassing PQ-SNN by 1.39%), which proves that ‘quantize first, then prune’ is a more effective approach. (For detailed theoretical analysis, see Response to W2.) | Method | PQ-SNN baseline | PQ-SNN | QP-SNN baseline | QP-SNN | | -------- | --------------- | ------------------------------ | --------------- | ------------------------------ | | Accuracy | 71.51% | 73.34% (PQ-SNN baseline+1.83%) | 70.27% | 74.73% (QP-SNN baseline+4.46%) |

I sincerely appreciate the authors' detailed response, which has addressed my concerns. I am willing to raise my score.

W2: To my knowledge, when two or more model lightweigh techniques are employed, compatibility issues often arise, such as the order of applying these techniques and the training strategies involved. Additional clarification is needed on these aspects.

Response to W2: Thank you for pointing out this issue. We have adopted the "quantize first, then prune" strategy based on the following two considerations.

• First, this strategy can better guarantee the effect of pruning technique. Specifically, if pruning is applied before quantization, important convolutional kernels identified in the full-precision parameter domain may become misaligned after quantization, as the quantization reintroduces additional errors. In contrast, by quantizing first and then pruning, redundant convolutional kernels are identified directly in the target low-precision parameter domain. This order allows for more accurate identification and preservation of critical kernels.

• Second, this strategy significantly reduces training overhead. Pruning before quantization requires three weight updates: "full-precision SNN training→ pruning with fine-tuning → quantization with fine-tuning," while "quantize first, then prune" only requires two adjustments: "quatized SNN training→ pruning with fine-tuning."

To more effectively address your concerns, we have conducted additional validation experiments by reversing the order of quantization and pruning. The experiments are conducted using the ResNet20 architecture on the CIFAR-100 dataset under the weight bit-width of 4, and the results are summarized in the table below. As shown in the following table, our ReScaW and SVS improve both performances irrespective of the order in which they are applied, yielding an 1.83% improvement in PQ-SNN and a 4.46% improvement in QP-SNN. QP-SNN achieves the highest performance of 74.73%.

Dear Changze,

Thank you for your interest and recognition of our work. I have organized the code and sent it to your email. I hope it will be helpful for your research.

Best regards, Wenjie.

Dear Do Thanh Dat,

Thank you for your interest in our work and your kind recognition. We are currently in the process of optimizing and organizing our research code for public release. We expect to make the latest version available within the next month.

Please feel free to reach out if you have any questions about our work in the meantime.

Best regards,

Eric