H-Direct: Homeostasis-aware Direct Spike Encoding for Deep Spiking Neural Networks

Thank you for your constructive feedback on our manuscript. We have carefully reviewed your comments and would like to ask for clarification on several points.

1. “Many of the techniques~ neuromorphic computing.” To the best of our knowledge, we have not encountered cases where adaptive thresholds or feature diversity loss were applied specifically to the encoding layer. Existing studies have not addressed issues related to direct encoding, which has been widely used in input spike encoding for deep SNNs. In our work, we believe this work is the first attempt to identify the limitations of conventional direct encoding and propose methods to address them. If there are relevant papers discussing adaptive thresholds or feature diversity loss applied to encoding layers, please let us know.
2. “The experiments focus heavily~ applicability of H-Direct.” The datasets used in this study are benchmarks commonly used to validate performance in deep SNNs, as referenced in [Rathi & Roy, 2021; zheng et al.,2021; Feng et al., 2022; Guo et al.,2022a; Guo et al., 2023; Zhou et al., 2023]. If you are aware of benchmarks that could better demonstrate the generalization performance of our proposed method, we would greatly appreciate your suggestions. Additionally, CIFAR10-DVS is, to our understanding, a neuromorphic dataset with considerable room for performance improvement. While automotive detection tasks such as Gen4 exist as an event-based vision data, these tasks beyond the scope of this work.
3. “The performance gains~ adaptive mechanisms.”
   Our experiments were conducted with baselines achieving SOTA-level performance. Therefore, as you noted, the improvements may appear marginal. However, we note that it is similarly challenging for other SOTA papers to demonstrate significant gains over competitive baselines [Guo et al.,2022a; Guo et al., 2023]. As shown in Table 2 of our manuscript, despite only improving the spike input encoding, our method enhances performance while reducing the number of spikes, thus improving energy efficiency. Furthermore, Table 2 also demonstrates that our method can be universally applied to approaches employing direct training, achieving both performance and efficiency improvements. We anticipate that our approach can enhance future SOTA methods.
4. “The paper’s presentation~ explanations.” If you can clarify which parts need improvement, we will consider revising the manuscript.

We believe your responses to the above questions will be instrumental in enhancing our manuscript. Thank you for taking the time to consider our queries, and we look forward to your feedback.

Thank you for the constructive feedback on our manuscript.

Q1. This paper significantly reduces the number of spike data, but lacks theoretical power consumption calculation. Can it give specific power consumption results and compare them with some advanced works [1]?

A1. Following the calculation method mentioned in [1], we attach the power consumption results and comparisons as below. The comparison targets are studies that used the same architecture and reported power consumption results. We assume that the data for various operations are based on a 32-bit floating point implementation in 45nm technology, where E_MAC = 4.6 pJ and E_AC = 0.9 pJ as [1]. Experimental results show that our method achieves higher performance with lower energy consumption compared to previous studies, and we will reflect this in the manuscript.

Table R1. Comparison with previous studies in terms of power consumption and accuracy based on the ResNet34 architecture.

Q2. The experimental effect of this paper is limited, and it is not compared with some advanced works [2][3] on many datasets. I hope the author can introduce why H-direct works from a theoretical perspective.

A2. Our experiments were conducted using a baseline with state-of-the-art (SOTA) performance. Thus, as you mentioned, the performance improvement may not appear substantial, but other SOTA studies also do not show significant performance improvements compared to their baselines [Guo et al., 2022a; Guo et al., 2023]. Furthermore, as shown in Table 2 of the manuscript, despite only improving spike input encoding, our method achieved improved performance with fewer spikes compared to the baseline, thereby enhancing energy efficiency and reducing computational costs. Additionally, as also shown in Table 2, our method can be generally applied to methods that use direct training, achieving both performance and efficiency improvements. This approach will also be applied to other methods presented in the future.

We also thoroughly reviewed and compared with the papers you referenced for comparison [2], [3]. First, [2] proposes a hybrid coding method that combines ASNN, which includes learnable components with time-to-first-spike (TTFS) encoding and direct encoding. Implementing this method requires additional computations for TTFS coding (encoding and decoding). Moreover, hybrid coding involves using both encoding methods, which we believe leads to considerable computational overhead. Since this study does not provide spike counts or energy consumption details, a quantitative comparison is limited. In contrast, our method can be applied without modifying the deep SNN model structure or requiring additional overhead during inference, which is a significant advantage. The significance of our approach is that it leverages the model capacity of deep SNNs to improve encoding efficiency and performance, thereby enhancing the overall performance and efficiency of deep SNNs.

Next, [3] proposes a method to improve the training efficiency of deep SNNs based on a ConvNeXt-style [R1]. The proposed model has structural characteristics different from the commonly used ResNet and also deviates from the model proposed in [R1] by excluding Layer Normalization, among other changes. Such deep SNN model structures have not yet been widely adopted in related research. Despite the specificity of the model, this study also uses direct encoding to achieve high performance. Therefore, we believe that our methodology can be applied to the method and model structure. Even if BN is not used, the DFE methodology of regulating spike firing by adjusting the input distribution of neurons can still be applied, and in such cases, it could be implemented using a loss function such as KL-divergence. Our approach to enhancing the performance and efficiency of deep SNNs through improved spike encoding can be applied to various models, datasets, and algorithms that use direct encoding.

Our study represents early research of efficient direct encoding, and through comprehensive experiments and analysis, we have thoroughly demonstrated that performance and efficiency can be improved solely by enhancing direct encoding. We will reflect the impact of H-Direct from a theoretical perspective in future research.

Thank you for taking the time to review our work.

[R1] Liu, Zhuang, et al. "A convnet for the 2020s." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

The paper the reviewer mentioned is our work that will be presented at the NeurIPS 2024 workshop (NeuroAI).

As stated in the call for papers for that workshop, papers presented at this workshop are not considered for publication (”non-archival”).

Call for papers for the workshop (https://neuroai-workshop.github.io/call-for-papers/)

“… We invite researchers in the field of NeuroAI to submit their papers to this workshop. The workshop papers are non-archival and authors are also encouraged to submit short versions of their NeurIPS main conference submissions concurrently to this workshop. …”

Also, according to ICLR’s dual submission policy, presented papers at workshops without publication proceedings do not violate the submission policy.

Dual Submission Policy of ICLR (https://iclr.cc/Conferences/2025/CallForPapers)

“… However, papers that cite previous related work by the authors and papers that have appeared on non-peer reviewed websites (like arXiv) or that have been presented at workshops (i.e., venues that do not have publication proceedings) do not violate the policy. … ”

Therefore, there is no problem with our submission of this manuscript, and we ask that the reviewers appropriately evaluate this paper with our response above.

Thank you for the constructive feedback.

W1. Many statements in the paper lack sufficient persuasiveness, such as those described in the abstract and introduction.

A1. It seems our wording may have led to some misunderstanding. First, regarding “Despite the importance of encoding, efficient encoding methods HAVE NOT BEEN studied,” as you mentioned, temporal coding methods like TTFS have indeed been studied. We intended to highlight that efficient encoding methods specifically for direct encoding, which is widely used in deep SNNs, have not yet been thoroughly explored. Additionally, the statement “Thus, the energy problem has become the MOST urgent issue to be addressed for the sustainable development and utilization of AI in our lives” was meant to convey that the energy issue is one of the major challenges in AI. We apologize for the confusion caused by our wording and will revise the manuscript to make our intention clearer. Regarding “Direct encoding learns encoding methods from data, which leads to superior performance over other encoding approaches,” we can confirm through Table 3 of [Qiu et al., 2024] that direct encoding outperforms other encoding schemes. Moreover, almost all recent studies demonstrating state-of-the-art performance use direct encoding [Rathi & Roy, 2021; Zheng et al., 2021; Feng et al., 2022; Guo et al., 2022a; Guo et al., 2023; Zhou et al., 2023].

W2. The proposed method lacks sufficient innovation, since the approach of setting dynamic neuronal thresholds and adjusting loss functions to enhance performance is widely used.

A2. We have not encountered any previous studies where adaptive thresholds and feature diversity loss have been applied to the encoding layer. Existing research has not addressed the widely used input spike encoding method, direct encoding, in current deep SNNs. To the best of our knowledge, this work is the first attempt to identify issues in conventional direct encoding and propose methods to address them. If there are related papers on applying adaptive thresholds or feature diversity loss to the encoding layer that we are unaware of, please let us know.

W3. The experimental results do not demonstrate a substantial improvement in accuracy, and as model size and dataset scale increase, this improvement diminishes further. This raises concerns about the practical utility of the proposed method.

A3. Our experiments were conducted with a baseline of state-of-the-art (SOTA) performance. Thus, as you mentioned, the performance improvement may not appear substantial, but it is also difficult to see significant improvements in other related papers compared to their baselines [Guo et al., 2022a; Guo et al., 2023]. Moreover, as shown in Table 2 of the manuscript, despite improving only the spike input encoding, our method achieved better performance than the baseline with fewer spikes, thereby increasing energy efficiency and reducing computational costs. As can be seen from the same table, our method can be applied universally to methods using direct training, achieving both performance and efficiency improvements. This approach can also be applied to other SOTA methods presented in the future. Additionally, as shown in Table 2, the improvement effect does not always decrease as the model size and dataset scale increase. For instance, in the case of VGG16, the performance improvement is greater for CIFAR100 compared to CIFAR10, and on CIFAR100, the performance improvement for ResNet20 is greater than for ResNet19.

W4. Does the performance improvement depend on a large number of training epochs? Could the authors provide more experimental details? Additionally, does the introduction of the proposed loss function add computational overhead?

A4. The configuration of our experiments was set similarly to those commonly used in related studies, and training was conducted until the training loss sufficiently converged [Deng et al., 2022; Guo et al., 2023]. In addition, the experimental results for tdBN, IM-loss, and RMP-loss provided in Table 2 were all conducted under the same settings and were fairly evaluated. Under the same experimental conditions, applying our method showed improvements in performance. More detailed experimental setups are provided in Appendix A.4. If you have any further questions about specific details beyond the provided experimental information, please let us know, and we will be happy to answer them. Our approach only modifies the encoding layers. Let $N_e$ be the number of encoding neurons. Then, the computational overhead of AT and FD is O($N_e$). The overhead of DFE is O($C_e$), where $C_e$ indicates the number of channels in the encoding layer. Considering that training demands O(W) computations and W is generally larger than $N_e$ and $C_e$, the overhead of our method is insignificant. We also demonstrated the overhead using wall clock time during training. As you can see in Table R1, when our method is applied, training time increases by only 2.26%. According to our measurements, most of the overhead was caused by FD, likely due to more complex operations, such as fitting the probabilistic density function. Detailed results are shown in the table below.

Table R1: Ablation study for overhead on CIFAR10

Q1. In Figure 1, in the image to the left of (b), the red box denotes OFE (Over-fired Encoding). Why is the RED box filled with ORANGE blocks (Orange represents normal spike counts)?

A5. The colors of each block in Figure 1-(a) represent the spike count (refer to the spike count legend in the upper right of Figure 1). In the example of Figure 1-(a), the “RED box filled with ORANGE blocks (actually YELLOW blocks)” represents neurons with a spike count of four (refer to the legend in the upper right of Figure 1). In other words, the box (channel) marked in red represents OFE. We apologize for the confusion caused by the figure. We will update the related figure to more clearly explain our method.

Q2. The criteria for the divisions are not clearly articulated. For instance, what spike count corresponds to DSE? Furthermore, when the spike count is zero, how to distinguish between OFE and DSE?

A6. Based on our findings in direct encoding, we categorized encoding channels into four types (PE, DSE, UFE, and OFE) mainly depending on the encoding rate of each channel. Encoding rate $R_c^d$ is defined as follows:

$$R_c^d =\sum_{d} H(S_{c}^{d}) / |D|.$$ $$S^d_c=\sum_{i\in Channels_c}\sum_{t}^{T}s_{i}^{d}[t],$$ $$H(x) = \begin{cases} 0 & x\leq 0 \\\\ 1 & x>0 \end{cases}.$$

• UFE: $R_c^d$ == 0, meaning they cannot encode any spikes for all input data. This kind of channel prohibits fully utilizing of given model capacity.
• DSE: 0 < $R_c^d$ < 1, meaning they decide whether to encode or not depending on the input. To adjust the encoding rate appropriately, we let them learn whether to encode or not through training data.
• PE: $R_c^d$ == 1, meaning they always encode spikes for all input data.
• OFE: $R_c^d$ == 1 and standard deviation ($\sigma$) of $\sum_t^T {s_i^d[t]}$ is 0, meaning that all neurons in Channel c fire with the same spike value(e.g., 1,2,…,t) for all input data, resulting in the encoding as meaningless features.

As explained, channel categorization is not a concept that can be defined for a single input. We hope that our additional explanation has been helpful for your understanding.

Q4. What is the purpose of fitting FD to a probability density function, and which PDF is used? What is the impact of different PDFs?

A4. For encoding diverse features, we proposed FD loss that increased the diversity of each neuron’s firing in the encoding layer. In order to utilize this as a regularizer during training, in this study, we fitted the distribution with a normal distribution, which is differentiable and can well represent the spike count distribution of neurons. To investigate the effects of other PDFs, we conducted additional experiments using an exponential distribution, and the results are presented in Table R1. For a fair comparison, we report the average of four experiments conducted under the same experimental settings as in the paper. Compared to the baseline, the normal distribution showed an improvement of approximately 0.2%, while the exponential distribution showed an improvement of about 0.08%. The total number of spikes decreased by 4K for the normal distribution and 6K for the exponential distribution, while the encoding spikes decreased by 6K and 7K, respectively. Both distributions outperformed the baseline, but we chose the normal distribution for its superior performance.

Table R1. Comparison of accuracy, number of all spikes, and number of encoded spikes across different PDF (CIFAR10, VGG16).

Q5. Does H-Direct show improvements in other architectures, such as RNNs or Transformers?

A5. The input spike encoding method proposed in this study is model-agnostic. Thus, we believe that it can be applied not only to various CNN models demonstrated in our experiments but also to other architectures such as RNNs and Transformers. For comparison with closely related works, we primarily conducted experiments on CNNs as in [Rathi & Roy, 2021; Zheng et al., 2021; Deng et al., 2022; Guo et al., 2022a; Guo et al., 2023; Mukhoty et al., 2023; Jiang et al., 2024]. Like other related studies, we also thoroughly conducted experiments on various CNN models and datasets in this paper to ensure comprehensive evaluation. Furthermore, to demonstrate the agnostic property of training algorithms, we also performed experiments applying different training algorithms (IM, RMP). This experiment, which has not been shown in other related works, indicates the thorough validation of our methodology. Despite our comprehensive validation, we plan to conduct further validation on other model architectures in the future.

Q6. Could the authors analyze the robustness of hyperparameters for the DFE, AT, and FD losses?

A6. Table R2, R3, and R4 present the accuracy, the number of all spikes, and the number of encoded spikes for different hyperparameters of DFE, AT, and FD, respectively. The hyperparameters used in the paper are indicated in bold. Each experiment was conducted under the same conditions as in our main results, with only the corresponding hyperparameter modified. The results are the average of two trials. First, Table R2 shows the results of experiments conducted by varying the hyperparameter (α) of DFE. When α=−3 compared to α=−1, there is a difference of approximately 0.17% in accuracy, 19K in total spikes, and 8K in encoded spikes. PE tends to have relatively larger α values compared to DFE. Thus, as α increases, more channels fire in PE, whereas, as α decreases, more channels fire in DSE or UFE. Consequently, the number of encoding spikes and total spikes tends to increase as α increases. Next, Table R3 presents the results of experiments by varying the hyperparameter (η) of AT. When η=0.8 compared to η=1.6, there is a difference of approximately 0.32% in accuracy, 20K in total spikes, and 7K in encoded spikes. As η increases, the rate of threshold reduction for channels that fail to fire in the encoding layer becomes smaller (η ≤ 1 ) or the thresholds exceed their initial values ( η > 1 ). This leads to a decrease in spikes in the encoding layer as η increases. The differing trend observed in the total spike count can be attributed to the fact that our method operates only in the encoding layer. Finally, Table R4 shows the results for the hyperparameter (λ_FD) of FD. When λ_FD=5e−7 compared to λ_FD=5e−6, there is a difference of 0.25% in accuracy, 8K in total spikes, and 2K in encoded spikes. FD is a loss function designed to encourage neurons in the encoding layer to fire with diverse spike counts. As λ_FD increases, the spikes of each neuron are distributed more diversely across the range of 0 to timestep T . Thus, the encoding spikes tend to increase as λ_FD increases. Similar to AT, the differing trends in total spikes are likely due to our method being applied only in the encoding layer. We determined each hyperparameter value by considering both performance and efficiency.

Table R2. Comparison of accuracy, number of all spikes, and number of encoded spikes based on the hyperparameter (α) of DFE (CIFAR10, VGG16).

Table R3. Comparison of accuracy, number of all spikes, and number of encoded spikes based on the hyperparameter (η) of AT (CIFAR10, VGG16).

Table R4. Comparison of accuracy, number of all spikes, and number of encoded spikes based on the hyperparameter (λ_FD) of FD (CIFAR10, VGG16).

Thank you for taking the time to review our work. We hope our answer addresses your concerns. If you have any further questions, please let us know.

Thank you for the constructive feedback.

Q1. Why are the corrective mechanisms not applied beyond the encoding layer, given that over-firing and under-firing issues may also exist in deeper layers?

A1. Our research aims to emphasize the importance and necessity of efficient input spike encoding in deep SNNs. In this work, we discovered the problems in the conventional direct encoding and experimentally demonstrated that improving input encoding can lead to enhanced performance and efficiency. We anticipate that applying our proposed method to hidden layers will further enhance performance and efficiency. This will be investigated in our future research.

Q2. In Table 4, why do certain modules of H-Direct decrease performance in ResNet20 compared to the baseline? Does this suggest limitations in module effectiveness across architectures?

A2. There is a trade-off between the efficiency (i.e., the number of spikes) and performance (i.e., accuracy) in deep SNNs. To improve these two simultaneously, we proposed H-direct, which consists of complementary methods. The main purpose of DFE is to increase efficiency by reducing inappropriate encodings and encoding rates. In cases where the proportions of inappropriate encodings (OFE, UFE) are high, as in VGG16, DFE can improve both efficiency and performance. However, DFE alone cannot guarantee simultaneous improvement. Thus, we proposed AT and FD, which encourage diverse spike encodings at the cost of efficiency. These methods deteriorate the efficiency improvement due to DFE (increased the number of encoded spikes in Table 4) but increase the diversity of encodings (# of channels in Table 5), which results in improvement of the accuracy. In conclusion, contrary to the reviewer's concern, the effectiveness of our method is not limited to a specific model architecture as in Table 2. The experimental results in Table 4 show the validity of our method, which utilizes complementary mechanisms.

The characteristics of input spike encoding may vary depending on the model architecture. As shown in the baseline of Table 5, VGG16 and ResNet20 exhibit different distributions of encoding channels. We observed that the proportion of improper encodings (OFE, UFE) is lower in the ResNet architecture compared to the VGG architecture. Due to the reasons mentioned above, applying DFE alone improved both performance and efficiency in VGG, whereas in ResNet, only efficiency was improved. When all proposed methods were applied, we observed an improvement in both performance and efficiency compared to the baseline. Therefore, our approach can improve both efficiency and performance, regardless of the various encoding patterns depending on the models.

Q3. How would DFE perform in networks without Batch Normalization?

A3. DFE is a method that improves encoding efficiency by adjusting the input distribution of encoding neurons. In this study, we implemented DFE by utilizing the scale and shift parameters of Batch Normalization (BN). As most state-of-the-art deep SNN models currently utilize batch normalization (BN) in the encoding layer, we believe our method is both applicable and practical [Fang et al., 2021; Zheng et al., 2021; Zhou et al., 2023; Yao et al., 2024]. Furthermore, even if BN is not used, the methodology of DFE, regulating spike encoding by adjusting the input distribution of neurons, can still be applied. In such cases, it could be implemented using a KL-divergence loss.

The paper the reviewer mentioned is our work that will be presented at the NeurIPS 2024 workshop (NeuroAI).

As stated in the call for papers for that workshop, papers presented at this workshop are not considered for publication (”non-archival”).

Call for papers for the workshop (https://neuroai-workshop.github.io/call-for-papers/)

“… We invite researchers in the field of NeuroAI to submit their papers to this workshop. The workshop papers are non-archival and authors are also encouraged to submit short versions of their NeurIPS main conference submissions concurrently to this workshop. …”

Also, according to ICLR’s dual submission policy, presented papers at workshops without publication proceedings do not violate the submission policy.

Dual Submission Policy of ICLR (https://iclr.cc/Conferences/2025/CallForPapers)

“… However, papers that cite previous related work by the authors and papers that have appeared on non-peer reviewed websites (like arXiv) or that have been presented at workshops (i.e., venues that do not have publication proceedings) do not violate the policy. … ”

Therefore, there is no problem with our submission of this manuscript, and we ask that the reviewers appropriately evaluate this paper with our response above.