Response to Reviewer BB72

---

R1: Contributions reclaim.

Our contributions primarily focus on a logits-based distillation framework tailored for SNNs, effectively leveraging spatio-temporal features without incurring additional overhead. Specifically, we address two critical deployment concerns: accuracy degradation from ANNs and retraining costs when adjusting timestep settings, achieving this through a lightweight, end-to-end solution validated by theoretical and empirical analysis. Furthermore, the trained SNNs demonstrate a notable advantage in terms of robustness across inference timesteps, as exemplified by stable performance from T=1 to T=6 (Tab. 7), enabling practical, flexible trade-offs between inference cost and accuracy. Overall, our clearly articulated methodology, novel conceptualization of temporal robustness, theoretical and empirical validation meaningfully advance the field of SNN distillation. Connections and differences with related works are further detailed in responses R1.1-R1.3, positioning our contributions within the existing literature.

R1.1: Relation to TET [1] and its underlying concept

Our work relates to TET [1], which significantly advanced direct SNN training by highlighting temporal-wise decoupling of the classification objective. Inspired by this underlying concept, our paper introduces, for the first time, a clear definition and adaptation of temporal-wise decoupling into logits KD domain. Specifically, we demonstrate—both theoretically and empirically—the advantage of designing logits KD aligned with the spatio-temporal characteristics inherent to SNNs.

Unlike TET, which focuses exclusively on decoupling cross-entropy loss and convergence boundary analysis, our method emphasizes applying temporal decoupling explicitly within logits KD, highlighting that performance improvements can be obtained without additional complexity (“free lunch”, see R4 of Reviewer p1Y4). Moreover, our work uniquely emphasizes the robustness across different inference timesteps, showcasing stable convergence for submodels (Sec. 3.4 and Tab. 7), an advantage previously unexplored by TET or related studies. Additionally, by integrating self-distillation within our temporal-decoupling framework, we fully utilize supervision signals at all timesteps without introducing extra overhead, further extending beyond TET.

R1.2: Relation and comparison with TSSD [2]

We regard TSSD as representative work in self-distillation, exploring soft-label generation from temporal and spatial perspectives. Unlike TSSD, which introduces overhead through auxiliary classifiers and extra forward processes, our method achieves improvements by using existing supervision signals (teacher, ensemble, and task labels in logits-KD framework) within a fixed backbone, requiring no additional branches or training overhead.

R1.3: Ensemble perspective with [3]

We recognize [3] as an excellent contemporaneous study that similarly explores submodels within SNNs from ensemble perspectives. While [3] constructs KL between adjacent timesteps (t and t-1), our method builds KL between the overall ensemble output and each individual time t. We will include discussion in the updated version, enabling readers to more comprehensively grasp ensemble learning in SNNs. We plan a detailed comparison once [3] becomes publicly available.

R2: About experiment setup.

In setting up, our work has maintained strict alignment with recent SNN studies [OTTT, SLTT, ASGL, rateBP]. We replicated logits KD, TKS on C100 setup:

On Imagenet and C10-DVS (Tab. 2 & 3), base models do not exhibit special advantage. The result on large-scale Imagenet is indicative of performance.

R3: Ablation study on CIFAR10-DVS

R4: Results on spikingformer [4]

R5: Ablation on TWSD

R6: Relation of Fig. 4 to the main claim and Tab. 7 baseline results

Thank you for suggestions. Please refer to R1 of Reviewer p1Y4.

R7: Regarding reproducibility and minor errors

Following modifications will ensure the code runs correctly:

1. Inutil.py, changeargs.arch().lower()to args.stu_arch.lower()withinget_model_namefunction.
2. Deleteinit_model(model)inbptt_model_settingfunction.
3. In run command, replace--epochwith--num_epoch.

The appendix provides necessary details. We plan to open-source complete codes and ensure reproducibility in future releases. Minor errors will be fixed, and we appreciate your feedback.

Response to Reviewer zowF

---

R1: Is the proposed method still in MS-ResNet [1]?

Following the experimental setup in [3], our frameword can also support the MS-ResNet [4] structure as expected, with the experimental results as follows:

R2: Discuss the recent paper [2] using distillation in SNNs.

Thank you for your comment. We acknowledge that [2] is an excellent piece of work and represents a significant advancement in the study of spiking transformer architectures. We note that distilling spiking transformer architectures requires special consideration of the choice of the teacher model ANN, as the design philosophy of spiking transformer architectures differs from traditional ANN architectures. Unlike the ResNet structure, it is not possible to find an ANN counterpart with the same structure, making this a unique consideration for transformer distillation schemes. At this point, the differences in structural design necessitate a more ingenious and novel heuristic design for the loss objective aligned with features between the ANN teacher and the SNN teacher under the feature-based distillation framework, which introduces greater design complexity in practical implementations. Therefore, we believe that logits-based, end-to-end distillation offers more practical advantages in the direction of spiking transformers. Our work aptly discusses how to fully unleash the potential of logits-based distillation in exploiting the unique spatiotemporal characteristics of SNNs, thereby laying a solid foundation for further exploration of logits-based distillation in spiking transformer architectures.

We will update our paper with the above discussion to strengthen our positioning in the literature. Thank you for your suggestion.

R3: Discuss the recent paper [3] in learning methods for SNNs.

The work in [3] is quite fascinating. We understand it as implementing adaptive encoding of spike sequence inputs during the network training process by adding an attention module at the input layer, which significantly enhances the adaptability of SNN architectures to static inputs and better utilizes the SNN’s capability to process spatiotemporal information. Interestingly, under this encoding method, the characteristics of the intermediate features at different times change with time, and the spike representations carry temporal dimension information. In this scenario, the standard distillation framework would cause the features at different times to fit towards the same ANN features, thus diluting the desired diversity in the temporal dimension. The prposed distillation framework considered in our paper, which is designed for temporal-wise decoupling, is better suited for such scenarios. We think the great potential of the proposed distillation framework in this direction is worth further exploration.

We will include the above discussion in the updated version. Thank you for your suggestion.

---

Reference:

[1] Quantized Spike-driven Transformer. ICLR 2025.

[2] Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training. IEEE TPAMI 2025.

[3] Gated attention coding for training high-performance and efficient spiking neural networks. AAAI 2024.

[4] Advancing Spiking Neural Networks towards Deep Residual Learning. IEEE TNNLS 2024.

Response to Reviewer p1Y4

---

R1: Relation of Fig. 4 to the main claim and Tab. 7 baseline results

Thank you for your suggestion. We use Fig. 4 to show the distinction from the standard logits-based distillation. In the case of the proposed temporal-wise distillation in Fig. 4b, the clustering effects of the submodels exhibit a high degree of similarity. For example, the upper, lower, and central-left clusters in each subplot of Fig. 4b (which will be marked in the updated images) all display a similar pattern across the five sub-figures. This largely echoes our analysis in Section 3.4, where the loss of each submodel is essentially embedded within a larger submodel framework, resulting in a uniform convergence effect in their clustering outcomes. In the updated version, we will specifically highlight the observed clustering phenomena in Fig. 4b to better complement the discussion mentioned above.

Additionally, we think the following results can more intuitively demonstrate the effectiveness of the proposed method compared to standard logits-based distillation paired with Fig 4:

Table X1. Comparison of inference performance based on models trained over different timesteps. The experiments were conducted using ResNet-18 on the CIFAR-100 dataset.

The table above shows inference results, supplementing the baseline results shown in Tab. 7. Horizontally, it demonstrates how inference accuracy changes when adjusting timesteps after model training. Under standard logits-KD, different models excel within specific inference timestep ranges: T=2 model performs best at timesteps 1–2, T=4 at times 3–4, and T=6 at time 5–6. In contrast, ours consistently achieves optimal performance using the model trained at the maximum timestep (T=6), clearly demonstrating the effectiveness of our method in training robust SNNs that maintain accuracy across varying inference timesteps.

Thank you for your suggestion; we will include these results and analysis in the updated version.

R2: Practical significance of temporal robustness

Ensuring the robustness of SNNs models at different inference timesteps can provide the following two technical advantages:

1. Horizontal view from Table X1. Taking the model trained with T=6 as an example, it shows stable performance across the inference window from T=1 to 6. Once deployed, the model does not require additional considerations for adaptation and switching across different inference states. This allows us to practically balance inference costs and performance directly, providing a viable model approach for scenarios that require real-time control of inference costs based on computational resources.
2. Vertical View from Table X1. For the model trained with T=4, we can invest in greater training costs to distill the T=6 model, and use the submodel at T=4 (essentially the same model) to achieve better performance. This offers an effective and feasible way to enhance performance by leveraging surplus training resources, providing a viable technical solution for scenarios where large training resources are available and performance enhancement is a critical issue.

R3: Results of firing rates

R4: About training costs and practical effectiveness

Thank you for your suggestion. We have added measurements of training costs:

Consistent with the discussion, the extra training overhead from ours is negligible compared to the overall backbone. We will include results in the updated version to reinforce the lightweight nature.

Following your comments, we conclude that ours achieves a 0.73% performance improvement over logits-KD at virtually no extra cost and also introduces robustness across varying inference timesteps. We think advantages of ours in practical use are evident.

R5: Include logits-KD in Fig. 1

Thank you for your suggestion. We will add the performance of logits-based KD to Fig. 1 to more clearly show effectiveness. The necessary data is already available in Table X1.