SpikeZIP: Compressing Spiking Neural Network with Paths-Ensemble Training for Optimized Pareto-front Performance

1. Some descriptions are unclear or confusing:

• Lines 17-20: "Nevertheless, existing works fail at providing theoretic equivalence between Quantized-ANN (QANN) and its converted SNN, while the SNN accuracy at small time-step (i.e., Pareto-frontier) can be further improved." The last row in Table 1: "Quantization levels in major-path and i-th sub-path."
• Lines 119-121: There are punctuation errors. It might be better to revise as: "ana. denotes analog input encoding; mem calib. is short for membrane calibration; and eq. stands for equivalence."
• Lines 250-252: The description is confusing and it's difficult to understand without reading subsequent content that convolutional layers in major-path and sub-paths share weights.
• Lines 327 and 328 mention the use of 'triangle symbol' and '-' symbol in the table to indicate SNN reaching equilibrium and most spikes not reaching SNN output, respectively. However, these symbols are not visible in Table 3. Is this an oversight?

2. In Table 4, the originally reported result for MS-ResNet34 shows timesteps equal to 6. The authors may have made an error in data collection here.
3. Lines 405-407 contain a descriptive error. The inference time steps of SNN should not be less than that of ANN with T=1.
4. The method of first training QANN and then converting it to SNN has already been proposed [1], so I don't consider this an innovation.
5. The motivation for proposing PET is insufficient and seems arbitrary. It's unclear what led the authors to choose this multi-scale parallel method. I understand that PET doesn't introduce additional inference computations for the conversion method, but the authors should provide ablation experiments to demonstrate the superiority of using PET compared to networks with only one stream. I believe that merely presenting data in Figure 6 without any analysis is insufficient to justify the necessity of PET.
6. Even though the authors state that their purpose for using the SEW residual structure differs from that in directly trained SEW-ResNet, I don't think this can be considered an innovation.
7. The effectiveness of this work is not satisfactory. In Table 3, it's difficult to see the superiority of the proposed method in terms of conversion or quantization. Regarding classification accuracy and reduction of inference time steps, this work's performance is inferior to some recent representative works not mentioned in this paper, such as [2].

[1]You K, Xu Z, Nie C, et al. SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN[J]. arXiv preprint arXiv:2406.03470, 2024.

[2] Lv L, Fang W, Yuan L, et al. Optimal ANN-SNN Conversion with Group Neurons[C]//ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024: 6475-6479.

This paper is not innovative enough, the RCR design has problems, and the PET is not convincing. More specifically:

1. There is a published paper [1] utilizing the same approach, i.e., ANN-QANN-SNN two-step conversion, for the objective of lossless conversion. This published paper has already been applied to complex Transformer architectures, while SpikeZIP has only been validated on simpler convolutional structures. It would be beneficial for authors to provide more specific details regarding how the approach in [1] compares with SpikeZIP. For instance, what are the key similarities and differences between the two? Additionally, are there any aspects of SpikeZIP that are novel compared to [1]?
2. The residual connection re-routing (RCR) method is originally proposed to be SNN-friendly; however, it clearly undermines this objective. As demonstrated in the SNN basic block in Figure 3, the vanilla structure preserves the spike-driven nature of SNNs. However, the output of the proposed SZIP is an integer, which disrupts the important spike-driven advantage of SNNs. So, why do authors think the designed RCR maintains SNN-friendliness? Additionally, the proposed RCR has already been extensively explored in SEW-ResNet (such as ADD, AND, and IAND).
3. The purpose of PET design is to improve performance. However, the author has not provided a clear explanation as to why dividing the backpropagation into three paths is necessary for achieving this performance improvement. The authors are encouraged to provide a more detailed justification for the three-path design of PET.

Ref: [1]: SpikeZIP-TF: Conversion is All You Need for Transformer-based SNN.

1. While paths-ensemble training is central to the SpikeZIP framework, a more in-depth explanation with specific pseudocode would make the method more accessible to readers. The authors should include a pseudocode algorithm or flowchart for the PET method in the paper or appendix. Additionally, a direct comparison between PET and conventional quantization-aware training is required, highlighting the key differences and advantages.
2. The paper would benefit from explicitly listing the assumptions and constraints under which the QANN-SNN equivalence holds. This would clarify the conditions necessary for achieving equivalence and enable more targeted reproduction. The authors should provide a clear list or table of the necessary conditions and constraints for QANN-SNN equivalence. This would be very helpful for readers trying to implement or build upon this work.
3. Although the focus is on conversion-based methods, a more extensive comparison with learning-based SNN training approaches could better contextualize the advantages of SpikeZIP, particularly in terms of energy efficiency and latency. It will be beneficial to include a comparative table or figure showing SpikeZIP's performance against leading learning-based SNN methods on standard benchmarks, focusing on both accuracy and energy efficiency metrics.

Despite the improved performance of this work, many concerns must be resolved for publication.

• Lack of novelty

  • Most of the components of SpikeZIP presented in this paper have been widely applied in other related studies.
  • What are the distinct features of SNN-friendly morphing compared to other studies?
  • Operator fusion and neuron replacing have also been widely applied in ANN-to-SNN conversion. What are the differences?
  • Input and bias encoding have also been widely used. What are the differences?
  • Residual connection re-routing is the same as SEW-ResNet as mentioned in the manuscript. The novelty of the proposed method does not exist just because of the different purpose. It is appropriate to judge that the existing method was applied. Additional explanations of the differences based on experimental results or theoretical analyses are required.
  • The theoretical proof in section 3.5 does not seem to be much different from the content published in the existing study. What is the difference from the proof in [2]?

• Lack of specific analysis of the proposed method

  • Why does the inference accuracy of the converted SNN increase when PET is applied? A specific and clear analysis is needed. Also, why were three paths {2, 4, 8} used? What effect does the number and composition of paths have on the conversion?

• Lack of logical connection between the theoretical proof and the proposed method

  • What relationship does the theoretical proof have with the proposed method? The proposed methods can be used without theoretical proof, and the method does not use the results of the theoretical proof.
  • Generally, T_{eq} > T_{off}, but how does T_{eq} affect the experimental results in the experiment?

• Manuscript revision

  • Last part of abstract
  • Missing definition of abbreviations
    • QANN, SOTA
    • line 183: LSQ-based
  • typos
    • line 39 - “))” → ‘)’
    • line 87
  • Duplicate notation
    • V^{in} in equations 1 and 7 have different meanings but are used in the same notation.
  • missing contents
    • line 327: The triangle symbol is not in the table. What does it mean?
  • Readability
    • The font size of Table 3 is too small to read.
    • The notation is concentrated in Table 1, so it is a bit inconvenient to look at the equation.
    • It would be better to switch the order of Figures 2 and 3. Section 3.3 is connected to Figure 3, and Section 3.4 is connected to Figure 2.
  • line 85: ST-BIF neuron reference required
  • line 347: no appendix provided

While the paper claims superior performance, it would be beneficial to see a more comprehensive comparison with other recent SNN conversion methods. For e.g. [1] has imagenet accuracy with <6 timesteps. Can the authors comment why is their latency so high as compared to [1]? Similar to [1], there are many other works that look into efficient and fast inference with low latency in SNNs even with transformers [2-4]. Can the authors comment on it? What about video data [5]?

The individual impact of each component in the SpikeZIP framework should be provided.

Will the technique scale to transformer architectures [2-4]?

[1] Li, Yuhang, et al. "Seenn: Towards temporal spiking early exit neural networks." Advances in Neural Information Processing Systems 36 (2024).

[2] Yao, Man, et al. "Spike-driven transformer." Advances in neural information processing systems 36 (2024).

[3] Zhou, Chenlin, et al. "Spikingformer: Spike-driven residual learning for transformer-based spiking neural network." arXiv preprint arXiv:2304.11954 (2023).

[4] Lee, Donghyun, et al. "Spiking Transformer with Spatial-Temporal Attention." arXiv preprint arXiv:2409.19764 (2024).

[5] Xiao, Shiting, et al. "ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition." arXiv preprint arXiv:2409.01564 (2024).