Thank you very much for your appreciation of our paper. In response to the issues you raised, we provide the following responses:

Q1: Mathematical Equivalence of RF Neurons and FT Operations

A: The mathematical proof demonstrating the equivalence of RF neurons as an efficient substitute for FT is provided in Appendix A, with the equivalence relationship expressed as follows:

$FT[k] = \sum_{n=0}^{N-1} x[n] e^{-\mathrm{i}\frac{2\pi}{N}nk} = \sum_{n=0}^{N-1} x[n] \left( \cos{\left(\frac{2\pi}{N}nk\right)} - \mathrm{i} \sin{\left(\frac{2\pi}{N}nk\right)} \right)$

$RF[k] = \sum_{n=1}^{T} \lambda_n \cos(n\omega_k \Delta t) - \mathrm{i} \sin(n\omega_k \Delta t) x[t-n]$

Moreover, In Figure 5(a), we validated the error between RF-ITD and FT-ITD across a substantial audio dataset, and the error ratio between these two methods was only 1%.

Q2: Why no Datasets for Robustness

A: Currently, there are no existing SSL datasets specifically designed for noisy environments. To validate robustness in all current SSL tasks, noise is artificially added, as demonstrated in the study [1,2,3]. Furthermore, we have detailed the process of adding noise to the SLoClas dataset and the preparation of robustness experiments in Appendix C.

$ComplexVideo_i (n) = mic_i (n) + \lambda noise(n), i=1,2,3,4$

Reference:

[1] Multi-tone phase coding of interaural time difference for sound source localization with spiking neural networks. In: TASLP (2021).

[2] A hybrid neural coding approach for pattern recognition with spiking neural networks. In: TPAMI (2023).

[3] SLoclas: A database for joint sound localization and classification. In: O-COCOSDA (2023).

Q3: Deploying the Proposed SSL Model on Hardware

A: We fully agree with you. However, we faced significant challenges in this area. Firstly, open-source edge devices, such as robots and robotic dogs, are costly and have inconsistent interfaces. Secondly, the optimal deployment of SNNs is contingent on neuromorphic chips or specialized hardware, which are both scarce and difficult to procure.

Q4: Further Analysis of Short-term Memory Structures

A: We fully agree with your perspective and have conducted extensive experimental studies in ablation study. Our research explored the performance of adjacent frames (our), multi-frame, and global frames (TA [1]). The ablation study results, shown in the below table, indicate that adjacent frames yield the best performance. This is because, while global and multi-step frames introduce additional useful information, they also introduce more noise, hindering the model's ability to quickly capture key ITD features. Moreover, the temporal interactions in global and multi-step frames require multiple audio frames for decision-making, undermining the model's real-time localization.

Reference:

[1] Temporal-wise Attention Spiking Neural Networks for Event Streams Classification. In: ICCV (2021).

Q5: A Biological Precedent for Similar RF Neurons in the Cochlea

A: Yes, RF neurons do have biological precedents in the cochlea. However, it's important to note that RF neurons are not a simplification of any specific biological cell. Instead, they are a mathematical abstraction of the combined functions of the cochlear basilar membrane and inner hair cells [1,2].

Reference:

[1] Dendritic channelopathies contribute to neocortical and sensory hyperexcitability in Fmr1−/y mice. In: NAT NEUROSCI (2014).

[2] An oscillator model better predicts cortical entrainment to music. In PANS (2019).

Thank you for the response, which has addressed my concerns very well, so I will increase the score.

Thank you very much for your recognition of our work. In response to the questions you raised, we provide the following replies:

W1: Enhance the Background Introduction

We strongly agree with your perspective. In the background section, we will incorporate additional research on SSL tasks and compare methodologies. Additionally, we will further augment the appendix with theoretical proofs of the formulas.

W2: Intuitive Demonstration of Model

We greatly appreciate your suggestions. We will revise Fig.1 to more clearly demonstrate the function of each part of the model.

Q1: How to Determine the Azimuth of the Sound Source

A: We will detail the function of each part of the model and its data format, as well as how to ultimately determine the azimuth.

• RF-PLC Method: An audio is initially segmented into a sequence of frames, denoted as $x\in R^{T \times C_{mic} \times L}$, where $T$ represents the number of frame sequences, $C_{mic}$ indicates the number of microphone channels, and $L$ denotes the length of each frame sequence. Subsequently, audio from different channels is paired and input into the RF-PLC encoding section, resulting in the encoding $E\in Z^{T \times C \times F \times \tau}$, where $C$ represents the number of channel combinations, $F$ denotes the number of frequency band decompositions, and $\tau$ indicates the number of detection neurons.
• Back-end Model: Encoding result $E\in Z^{T \times C \times F \times \tau}$ will be input into a network featuring the MAA module. It primarily consists of frequency band preference and short-term memory components. The frequency band preference ensures the network focuses more on crucial frequency band information, while the short-term memory enhances the temporal capabilities of neurons. Combined, these features significantly boost the environmental robustness of the backend network. Finally，the output layer of the network consists of 360 units, representing the predicted outcomes.

Q2: Effectiveness of the MAA Module

A: To ensure that the effectiveness of the MAA module is NOT merely due to an increase in parameter count, we conducted additional ablation experiments detailed in the supplementary materials. As depicted in Fig.9 and below Table, 'Local-XXX' represents the scenario where the module is replaced with a convolutional layer of equivalent parameter size. The results clearly demonstrate that even with an equal number of parameters, our module exhibits superior performance, underscoring its unique benefits.

Q3: Hardware-friendly of RF Neurons

A: We agree with your view. RF neurons and our SSL model are indeed compatible with current neuromorphic platforms like Loihi 2 [1] and Tianjic [2]. Yet, substantial challenges hinder their broader adoption. Primarily, the high costs and varied interfaces of open-source edge devices like robots limit practical deployments. Furthermore, the scarcity and procurement difficulties of neuromorphic chips or specialized hardware impede the effective utilization of SNNs.

Reference:

[1] Efficient Video and Audio Processing with Loihi 2. In: ICASSP (2024).

[2] Towards artificial general intelligence with hybrid Tianjic chip architecture. In: Nature (2019).

Thank you for acknowledging our work. As you per, traditional ITD encoding often relies on FT operations [1], which are difficult to deploy on neuromorphic hardware, resulting in a divided implementation. In contrast, our ITD encoding scheme employs RF neurons, which have been successfully deployed on Loihi 2 [2] with excellent performance. This demonstrates that our encoding method does not face technical barriers when deployed on neuromorphic chips. Additionally, our backend localization model is strictly spike-driven, with extensive research confirming that such models can be effectively implemented on neuromorphic hardware [3, 4]. Thus, our SSL model can be deployed as a complete system on neuromorphic hardware.

[1] A hybrid neural coding approach for pattern recognition with spiking neural networks. In: TPAMI (2024).

[2] Efficient Video and Audio Processing with Loihi 2. In: ICASSP (2024).

[3] 22.6 anp-i: A 28nm 1.5 pj/sop asynchronous spiking neural network processor enabling sub-o. 1 μj/sample on-chip learning for edge-ai applications. In: ISSCC (2023).

[4] Towards artificial general intelligence with hybrid Tianjic chip architecture. In: Nature (2019).

Thank you very much for your valuable feedback and for taking the time to read our paper. We hope the following responses will address your questions.

W1: How to Design Loss Function

A: To achieve a $1\degree$ resolution in SSL tasks, the output layer comprises 360 neurons, each representing a distinct azimuth angle[1]. Subsequently, we use the cross-entropy loss function to guide model training. Thanks to the design of the output layer, there is less need to focus on the discrepancies between the network's predictions and the actual outcomes.

However, during model performance evaluation, the similarity between results at $355\degree$ and $5\degree$ can pose issues in accurately assessing model performance. We utilize the evaluation metrics [2] to valiate the performance of model. Specifically, toaddress the aforementioned issue, MAE is further defined as:

$MAE = \frac{1}{N}\sum_{i=1}^{N} 180\degree -||\hat{\theta}-\theta|-180\degree|$

Reference:

[1] Multi-tone phase coding of interaural time difference for sound source localization with spiking neural networks. In: TASLP (2021).

[2] A hybrid neural coding approach for pattern recognition with spiking neural networks. In TPAMI (2024).

W2: Robustness Validation of Localization

A: Considering the potential presence of noise in SSL tasks, it is crucial to assess model performance in noisy environments. As described in Appendix C, we introduce the same noise information, $noise(n)$, into each microphone channel $mic_i(n)$ to simulate noise coming from various directions. The dataset can be represented as:

$ComplexVideo_i (n) = mic_i (n) + λnoise(n), i=1,2,3,4$

Additionally, we can utilize the Signal-to-Noise Ratio (SNR) to represent the complexity of the dataset after adding noise.

$SNR(dB) = 10 log_{10}(\frac{P_{signal}}{P_{noise}})$

W3: Validity of Experimental Methods

A: The SLoClas dataset stands as one of the most challenging benchmarks for SSL tasks. In contrast, other datasets, such as those employed in studies [1] and [2], suffer from limited localization precision, making them less representative of real-world scenarios. Moreover, we have also tested our model on datasets [1] and [2], where it achieved nearly 100% accuracy. Therefore, we are particularly focused on the performance of the SLoClas dataset, as it is the only one collected in a real-world environment.

Reference:

[1] Spiking neural network model of sound localization using the interaural intensity difference. In: TNNLS (2012).

[2] A biologically inspired spiking neural network model of the auditory midbrain for sound source localisation. In: Neurocomputing (2010).

[3] A hybrid neural coding approach for pattern recognition with spiking neural networks. In: TPAMI (2024).

Thank you very much for recognizing the strengths and quality contributions of our paper. In response to the questions you raised, we are providing further details and insights to clarify the points mentioned in your review.

Q1: Generalization

A: As per your suggestion, we evaluated our model using the HRTF, Single Words, and SLoClas datasets, assembled from both simulations and real-world data collections. As detailed in the subsequent table, our model demonstrates SOTA performance across all the three datasets, thereby validating its generalizability.

Reference:

[1] Spiking neural network model of sound localization using the interaural intensity difference. In: TNNLS (2012).

[2] A biologically inspired spiking neural network model of the auditory midbrain for sound source localisation. In: Neurocomputing (2010).

[3] A hybrid neural coding approach for pattern recognition with spiking neural networks. In: TPAMI (2024).

Q2: Incremental Costs in Module Stacking

A: We agree with the importance of balancing performance and computational costs. As per your suggestion, we conducted a detailed comparison across various SSL models. As shown in the following table, our model achieves competitive performance while reducing the model size by 1/10. The results demonstrate that our model achieves the best trade-offs between performance and computational overhead.

Reference:

[1] A hybrid neural coding approach for pattern recognition with spiking neural networks. In: TPAMI (2024).

[2] Multi-tone phase coding of interaural time difference for sound source localization with spiking neural networks. In: TASLP (2021).

[3] Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips. In: ICLR (2024).

Q3: Novelty of This Work

A: Compared to the work referenced in [38], our study demonstrates innovation primarily in two aspects:

• Energy-efficient ITD Encoding: Reference [38] uses FT operations during encoding, which needs massive computing resources. Our method innovatively employs RF spike neurons and phase-locking mechanisms to directly encode raw speech signals into sparse ITD spike trains. This approach is not only biologically plausible but also energy-efficient.
• Enhanced Performance in SSL Backend Networks: The backend network in [38] is limited to convolutional architectures, which do not adequately consider the dependencies and preferences between the frequency and time domains of sound signals, resulting in suboptimal localization performance. In contrast, our MAA module innovatively incorporates the frequency band preferences and temporal interactions found in biological auditory systems. Our ablation studies robustly validate this enhancement.

Q4: Training Cost

A: Based on your suggestion, we tested the training time overhead on ANN and SNN backends with identical architectures. Detailed experimental details are as follows:

• ANN: Training was conducted on a single NVIDIA RTX 3090 (24G) with a batch size of 16. Each epoch took approximately 30s, and the model required around 300 epochs to complete training.
• SNN: Training was conducted on a single NVIDIA RTX 3090 (24G) with a batch size of 16 and a time step of 4. Each epoch took approximately 60s, and the model required around 200 epochs to complete training.

As you produced, SNNs indeed require longer training times compared to ANNs. Efficiently training SNNs continues to be an unresolved issue within the field. Numerous studies have focused on mitigating the training challenges associated with SNNs through the development of efficient algorithms [1] and the implementation of dedicated hardware acceleration platforms [2,3].

Reference:

[1] Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting. In: ICLR (2022).

[2] Towards artificial general intelligence with hybrid Tianjic chip architecture. In: Nature (2019).

[3] 22.6 anp-i: A 28nm 1.5 pj/sop asynchronous spiking neural network processor enabling sub-o. 1 μj/sample on-chip learning for edge-ai applications. In: ISSCC (2023).

Q5: Why Our SSL Model Performance Better

A: This is a core highlight of our work. We attribute the superior performance of our model to the following three key parts:

• RF-PLC method: RF neurons are strategically deployed to precisely isolate ITD cues, effectively eliminating auditory information that does not contribute to localization. This provides a reliable data foundation for high-performance localization.
• MAA module: The MAA module draws on biological auditory principles of frequency band preference and short-term memory. The font enhances networks' focus on critical ITD cues within key frequency bands. The latter strengthens the model's memory for informed decisions across timeframes.
• Advantage of SNNs in SSL task: Information in SNNs is encoded within temporal spike trains, allowing these networks to directly and effectively capture temporal features. Given the high time resolution required for SSL tasks, SNNs are particularly well-suited for processing temporal audio signals.

Overall, the proposed SSL model well-orchestrates the proposed RF-PLC method, MAA module and the inherent advantage of SNNs, and this is the main reason for its high performance.

Thank you very much for your review comments. In response to the issues you have raised, we offer the following replies:

Q1: Limited Improvement

• Limited Improvement: In SSL tasks, Acc and MAE are the most important metrics. As shown in the following table, extensive comparative experiments have demonstrated that our model not only achieves SOTA accuracy among similarly sized models but also reduces MAE by approximately 50%.

Reference can be found in Table.1 of the submitted manuscript.

• Compared with Spike-Driven V2 Model: As per your suggestion, we replaced the back-end model with Spike-driven V1 and V2 models [1,2] for the ablation study. As shown in the following table, our model achieves competitive performance while reducing the model size by 1/10. Therefore, it achieves the best trade-offs between performance and computational costs.

By the way, spike-driven methods are generic and high-performing, but they rely on extensive parameters. In contrast, our model is specifically designed for SSL tasks, fully considering the characteristics of ITD pulse sequences, thus it is both lightweight and efficient.

Reference:

[1] Spike-driven Transformer. In: NeurIPS (2023).

[2] Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips. In: ICLR (2024).

Q2: Novelty of Attention Mechanisms

A: The novelty of our MAA module lies in three key aspects:

• MAC-free Computational Paradigm: Compared with those attention methods [1,2,3] in SNNs, our methods achieve MAC-free attention computation paradigm. Therefore, it possesses significant energy efficiency advantages.

• Task-Specific Design: Our methods are designed to work well with the specific patterns of ITD spike trains from the RF-PLC method. This helps the front-end and back-end of the SSL model work better together, making the system both efficient and effective.

• Biological Plausibility: The MAA incorporates frequency band preference and short-term memory, both grounded in strong biological evidence. Neuroscientific evidence indicates that frequency tuning varies across regions, highlighting the critical role of frequency selectivity in auditory processing [4]. Additionally, biological studies demonstrate that auditory short-term memory significantly enhances selective attention to auditory signals in noisy environments [5].

Reference:

[1] Temporal-wise attention spiking neural networks for event streams classification. In: ICCV (2021).

[2] TCJA-SNN: Temporal-channel joint attention for spiking neural networks. In: TNNLS (2024).

[3] Attention spiking neural networks. In: TPAMI (2023).

[4] Temporal coherence and attention in auditory scene analysis. In: TRENDS NEUROSCI (2011).

[5] Attention improves memory by suppressing spiking-neuron activity in the human anterior temporal lobe. In: Nature Neuroscience (2018).

Q3: Experiment Detail

A: As per your suggestions, all experiments in this paper were conducted at least 5 times to ensure reliability. For example, the follow table shows the robustness performance of our methods in five independent experiments. In addition, we have uploaded all the code to the supplementary materials to help others replicate our results more easily.

We greatly appreciate your recognition of the innovative aspects and motivation of our work. In response to the weaknesses and questions, we will provide further detailed explanations:

W1: Compare Analysis

A: Following your suggestion , we rigorously evaluated our work against previous SOTA ANN-based and SNN-based models in terms of both Acc and MAE. As demonstrated in the table, our model all achieved SOTA performance.

We attribute the superior performance of our model to the following two key parts:

• RF-PLC method: RF neurons are strategically deployed to precisely isolate ITD cues, effectively eliminating auditory information that does not contribute to localization. This provides a reliable data foundation for high-performance localization.
• MAA module: The MAA module draws on biological auditory principles of frequency band preference and short-term memory. The font enhances networks' focus on critical ITD cues within key frequency bands. The latter strengthens the model's memory for informed decisions across timeframes.

Overall, our SSL model effectively integrates the proposed RF-PLC method and MAA module, resulting in demonstrably superior performance.

W2: Trade-offs between Performance and Computational Costs

A: As per your suggestion, we conducted a detailed comparison across various SSL models. As shown in the following table, our model achieves competitive performance while reducing the model size by 1/10. The results demonstrate that our model achieves the best trade-offs between performance and computational overhead.

Reference:

[1] A hybrid neural coding approach for pattern recognition with spiking neural networks. In: TPAMI (2024).

[2] Multi-tone phase coding of interaural time difference for sound source localization with spiking neural networks. In: TASLP (2021).

[3] Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips. In: ICLR (2024).

W3: Generalization

A: As per your suggestion, we evaluated our model using the HRTF, Single Words, and SLoClas datasets, assembled from both simulations and real-world data collections. As detailed in the subsequent table, our model demonstrates SOTA performance across these datasets, thereby validating its generalizability.

Reference:

[1] Spiking neural network model of sound localization using the interaural intensity difference. In: TNNLS (2012).

[2] A biologically inspired spiking neural network model of the auditory midbrain for sound source localisation. In: Neurocomputing (2010).

[3] A hybrid neural coding approach for pattern recognition with spiking neural networks. In: TPAMI (2024).

W4: Biological Plausibility

A: The biological plausibility of our model is demonstrated in two key aspects:

• RF-PLC method: This method utilizes RF spiking neurons and phase-locking mechanisms, both of which are biologically plausible. Neuroscientific evidence demonstrates that RF neurons, a specific type of biological neuron, selectively respond to particular input frequencies and efficiently process oscillatory signals [1]. Furthermore, phase-locking mechanisms are prevalent in the biological auditory system and facilitate the capture and encoding of temporally correlated information by neurons [2].
• MAA module: The MAA module emphasizes frequency band preference and short-term memory. Neuroscientific research indicates the critical role of frequency selectivity in auditory processing [3]. Additionally, biological studies demonstrate that short-term memory enhances selective auditory attention in noisy environments [4].

Reference:

[1] Dendritic channelopathies contribute to neocortical and sensory hyperexcitability in Fmr1−/y mice. In: NAT NEUROSCI (2014).

[2] An oscillator model better predicts cortical entrainment to music. In PANS (2019).

[3] Temporal coherence and attention in auditory scene analysis. In: TRENDS NEUROSCI (2011).

[4] Attention improves memory by suppressing spiking-neuron activity in the human anterior temporal lobe. In: NAT NEUROSCI (2018).

Q1: Suggestions for Improving Writing

A: Thank you for your constructive feedback on our manuscript. We will carefully consider your recommendations to simplify technical descriptions and enhance discussions of limitations relative to existing SSL models.

Q2: Experimental Reproducibility and Future Discussion

A: As shown in the following table, all experiments were conducted at least 5 times to ensure reliability.Furthermore, the code has been uploaded to the supplementary materials to enhance the reproducibility. In future research, we aim to deploy and test our model on Loihi 2 or Tianjic, ultimately applying it to intelligent robots.

Thank you for your thoughtful review and valuable suggestions regarding our manuscript. We have carefully incorporated your recommendations and conducted extensive experiments to address your concerns.

We observed that the score for our manuscript decreased from 4 to 3. We are puzzled by this change and would like to understand the reasons. Could this reduction be a misunderstanding, or are there new concerns arising from our rebuttal? We would greatly appreciate any additional feedback that could help us improve our manuscript further.

Thank you very much for considering our request.

Thanks for rebuttal, but the results seem not convinced just as reviewer 2syp mentioned, so I will lower my score.

Dear Reviewer,

We would like to confirm whether our recent response has adequately addressed your concerns. Regarding the issue of convincing results, all the relevant codes and data are provided in the supplementary materials. May we kindly request that you review these materials at your earliest convenience? We appreciate your reconsideration and hope for a fair evaluation.

Thank you for your time and attention.