Enhancing Audio--Language Models through Self--Supervised Post--Training with Text--Audio Pairs

We are thankful for the insightful reviews and comments on our work. We are working on refining our subsmission and adressing all the questions marked in the review. Looking forward towards this discussion.

Thanks, --Authors.

We sincerely thank the reviewers for their thoughtful feedback, which has been invaluable in enhancing the quality of our paper. We deeply appreciate the recognition of our work's strengths and the constructive identification of areas for improvement. Below, we provide a detailed response to each identified weakness, address the specific questions raised, and outline the changes incorporated into the revised submission.

We respectfully request the reviewers to carefully consider our explanations and arguments, as we believe they are integral to advancing the understanding we aim to contribute to the community through this work. We remain fully open to further discussions and are committed to refining the manuscript to better align with the expectations of the reviewers and the broader research community

W1 & W2 Citations and Comparisons:

• The reviewer correctly points out the relevance of both [1] and [2], which we included as major contributions in this space. Additionally, we acknowledge that [3] is one of the earlier works in this domain. However, we did not include [3] in our original submission since it has neither been published in a peer-reviewed journal or conference nor has it made its code and results publicly available. We apologize for this oversight and have now cited [3] in the updated version of our paper.

• In response to the reviewer's comments, we have added a paragraph in section 1. There is indeed a distinct difference between our sequential learning approach and the modular contrastive approach mentioned by Compa [2]. While Compa trains the model to become modular by introducing progressively complex texts—essentially akin to introducing new types of labels in steps—our model adopts a fundamentally different strategy. We initially train the model to recognize that multiple distinct sounds can coexist, followed by training on these multiple sounds. This approach is more interpretable and structured in its methodology.

• In comparison to T-CLAP and work by Wu et. al., both trains a contrastive learning model without requiring to address the need of multiple sounds distinction which defeats the purpose of increasing the interpretability of the models. We are not able to include them in the results section as they neither have benchmark results nor publicly available model trained weights for us to use. (Line 93-109).

• Furthermore, In contrast, our approach achieves significant advancements within a limited computational budget, training around 10% of the total trainable parameters and utilizing a single dataset (ESC-50). Unlike prior works, such as those by Ghosh et al. (2023) and Wu et al. (2023a), which rely on more expansive datasets and substantial computational resources. Our focus is on developing a methodology that can effectively instill a sense of time in the model within acceptable computational constraints, rather than on generalizing over large, diverse dataset

  References:

  • [1] Wu et al., Audio-text models do not yet leverage natural language
  • [2] Ghosh et al., Compa: Model comparison
  • [3] T-CLAP: Temporal-Enhanced Contrastive Language

W3. Public Accessibility of T-CLAP:

• As noted by the reviewer, T-CLAP [3] does not provide publicly available data or a codebase. This lack of public resources limits a direct comparison with their implementation. The details of coparison with this model has been included in line [96-99] and [100-106] the revised manuscript. We also like to correct the reviewer that T-CLAP [3] is trained on 2.04M audio--caption pairs and not just on ESC--50 dataset. While with our mdoel, the focus is on developing a methodology that can effectively instill a sense of time in the model within acceptable computational constraints, rather than on generalizing over large, diverse datasets.

W4. Contributions:

Kindly check the following answer where we have detailed our contributions.

W5. Two-Stage Training Methodology:

The reviewer's comments on this aspect are highly insightful, and we have addressed this point in the updated manuscript.

• A joint training method would undermine the purpose of distinguishing between multiple sounds, which is a prerequisite for effective temporal instillation. Joint training enforces these conditions simultaneously, increasing the complexity for the model.
• Given the limited number of trainable parameters in our setup, we opted for a step-wise training approach, which is more controllable.
• Sequential training allows us to observe the model's progress at each stage, thereby enhancing interpretability.

Issues with the presentation :

We are grateful for the reviewer's detailed observations and have made the following corrections in our updated submission:

• Updated Figure 3 and 4 for clarity.
• Removed the extra "equation"
• Corrected the equation numbering.

Overall Remark:

W4. Details about our contribution:

We would like to elaborate on our objectives, the claims associated with each, and the strong proofs presented as results. We kindly request the reviewers to thoroughly review these points, as they should provide a clearer understanding of the contributions and significance of our work.

Our Objectives (as presented in the paper):

1. Establishing a multi-stage temporal instillation objective:

The first stage focuses on learning to distinguish multiple sounds, while the subsequent stage emphasizes learning the temporal arrangement of these sounds.

• Claim: Training a single model to directly learn temporal relationships is ineffective because models must first develop the ability to distinguish between different sounds. Without this foundational understanding, learning temporality becomes infeasible.
• Proof 1: To validate this claim, we compared the model's performance after applying only TeminAL B with the performance after employing the multi-stage training process (TeminAL A followed by TeminAL B). The results show that the multi-stage model significantly outperforms in all tasks defined by ZSTE—a more advanced progression of the classical zero-shot evaluation strategy widely used in prior work. The results are detailed in Table 2.
• Proof 2: We further compared our model with state-of-the-art (SOTA) contrastively trained ALMs across all ZS evaluation tasks. Specifically, in ZSTE Task 2 (subtasks A and C), which require distinguishing multiple sounds, our model outperforms models trained on large datasets. However, in subtasks 2B and 2D (which involve simpler mappings of individual sounds), models trained on massive datasets excel due to their extensive exposure. For Tasks 3A and 3B, all models perform comparably, highlighting the challenges of zero-shot evaluations when trained on similar examples. These findings underscore the necessity of multi-stage training for better generalization.
• Note: Our training was intentionally restricted to a subset of the total parameters and used only the ESC-50 dataset. This decision aligns with our goal of demonstrating results within a realistic compute budget. Reviewers are kindly requested to consider that the focus of this work is not on achieving SOTA performance across billion-sample datasets, but rather on addressing a critical gap in current ALM post-training methods. Our key contributions lie in introducing a novel multi-stage training strategy and an effective evaluation framework.

2. Demonstrating correct temporal relationships between audio and text:

• Claim: Our analysis indicates that current contrastive ALMs face challenges in accurately capturing temporal relationships between audio and text.
• Proof: This claim is best evaluated after understanding our first objective. We acknowledge that the presentation order of objectives in the manuscript might have caused confusion, and we apologize for any inconvenience caused. However, the current order was chosen to maintain the logical progression of ideas.

3. Proposing a general-purpose zero-shot evaluation scheme:

• Claim: We are not claiming anything here, but providing and using a structured way of evaluating our earlier claims.
• Proof: Existing evaluation metrics fall short of effectively assessing temporal relationships, particularly in our case. We propose this evaluation scheme as a simple yet intuitive method to capture model behavior accurately. While future enhancements to evaluation metrics are certainly necessary, this scheme is sufficiently robust for our purposes.

Final Note:

If the addressed weaknesses and clarifications satisfactorily resolve the reviewers' concerns, we kindly request an updated score to reflect the potential impact of this work in advancing the field of audio-language models. Thank you.

W3. Comparison and Fairness of evaluation :

• We thanks the reviewer for noting the impact of our post--training dataset size and the parity with other mentioned models. We acknowledge the reviewers suggestion of either training CompA on the same dataset or using CompA's dataset for comparison. To carefully adress this we have the following remarks.

• Results overfitting on ESC--50:

• We do partially accept this remark from the reviewer and think it's an important remark in general toward model post--training. Although, as we emphasis in our objective and scope of work, that our aim is not give a generalisable model which perform SOTA across all datasets. Rather our aim is to present the idea of multi--stage training of contrastive models and it should effectively be done in the sense of instillation of time. We can't really say it's overfitting as we perform all our multiple audio evaluation on the test set for ESC--50. Although we do accept that the model hasn't seen other data and so has a bias towards ESC--50 which does get reflected in the results. But given the compute budget, we determine training on a large dataset to be out of scope for the current work.

• Assertion: Evaluating results of other mentioned dataset would not be able to give better result than the origianl CLAP model, as we never post--trained on those specific datasets. Henceforth encountering any OOD sample would mean feeding unkown data to the model to hallucinate on. While the training can well be extended to different datasets or to a better mixture of datasets, but that defeats the purpose of being able to train within our compute budget. Although this point has been well noted for future work which will emphasis solely on giving generalisable post--trained models. We request the reviewer to share their view on our assertion.

• Thus, our final take on this is to not consider the model to be a generalisable SOTA ALM, rather consider the method to be as one which has novelty in contribution and succesfully defends the novelty as detailed in our previous comment (https://openreview.net/forum?id=nplYdpc1Pm&noteId=e4ylyCLZy1) without going out of scope. So, it's funadmentally neither fair nor reasonable to evaluate the model on various datasets testing it's generalisability.

Comment on joint training method:

• We would like to humbly counter the point mentioned by the reviewer, why are we focusing on the joint training mehtod? It doesn't take away from the fact that a multi-objective (either joint or sequential) is required.
• In this work we opted for the sequential training method as we wanted to focus on understandibilty of each of our models.
• The abalation on joint training would be a good addition, but it would not add knowledge to the eixting proofs of our claims. Our work objective is not to test the various training strategy which can be there, rather our objective is to assert that the current contrastive ALMs do not satisfatory capture the temporal instillation and for that we propose a method.
• Henceforth, we humbly request the reviewer to kindly view the current objectives as it would be out of our scope to tackle all the challenges in one work.

Final note and request: We sincerely appreciate the reviewer for taking the time to provide highly valuable comments on our responses. We have made all the necessary changes as requested by the reviewer. We request the reviewer to kindly review our updates and if they are able to convey the message then kindly update their scores.

W1: Distinction from other Related work : We request the reviewer to kindly check the updated document where we have specifically updated the following to inlcude all the distinction between current and related works. Kindly note the specific lines.

• (Line 93-101) : In response to the reviewer's comments, we have added a paragraph in section 1. There is indeed a distinct difference between our sequential learning approach and the modular contrastive approach mentioned by Compa [2]. While Compa trains the model to become modular by introducing progressively complex texts—essentially akin to introducing new types of labels in steps—our model adopts a fundamentally different strategy. We initially train the model to recognize that multiple distinct sounds can coexist, followed by training on these multiple sounds. This approach is more interpretable and structured in its methodology.

• (Line 102-108) : Furthermore, In contrast, our approach achieves significant advancements within a limited computational budget, training around 10% of the total trainable parameters and utilizing a single dataset (ESC-50). Unlike prior works, such as those by Ghosh et al. (2023) and Wu et al. (2023a), which rely on more expansive datasets and substantial computational resources. Our focus is on developing a methodology that can effectively instill a sense of time in the model within acceptable computational constraints, rather than on generalizing over large, diverse dataset

• (Line 97-101) and (Line 101-104) : In comparison to T-CLAP and work by Wu et. al., both trains a contrastive learning model without requiring to address the need of multiple sounds distinction which defeats the purpose of increasing the interpretability of the models. We are not able to include them in the results section as they neither have benchmark results nor publicly available model trained weights for us to use.

• (Section 5 ; Specifically in discussion to Table 2 and Table 3) : Non explicit comparison based on model results, the reviewer has already noted these comparative details as mentioned in their comment.

W2. Claim on current model : Yes, as mentioned in our previous comment.

• This claim is best evaluated after understanding our first objective. We acknowledge that the presentation order of objectives in the manuscript might have caused confusion, and we apologize for any inconvenience caused. However, the current order was chosen to maintain the logical progression of ideas.

• As mentioned by us it is clear from the result section that the models trained on larger dataset with large number of trainable paramters do perform well in subtask 2B and 2D, as it only requires them to map to at--least one of the sounds (There was a type-o in our initial response, we corrected it.). Now when we take a look at subtask 2A and 2C, it is evident that the other models do not satisfactorily capture the distniguishing multplie sounds.

• We thank the reviewer for taking this issue up we accept the suggestion by the reviewer and to address the issue which the reviewer has, we have updated the main paper (Line 131-133) : " Our analysis indicates that current contrastive ALMs face challenges in accurately capturing temporal relationships between audio and text, as shown in \cref{table5}, highlighting an area for potential improvement in existing models." We request the author to kindly review this and let us know if this addresses the concerns on our remarks given to the previous models.

Respected reviewer xfBd,

Along with the above updates on your questions, we have taken the time to adress all the reviews and provide an in-depth detail of the summary (The response can be found up top in the comment section as "Final updates and addressing the reviewers" , https://openreview.net/forum?id=nplYdpc1Pm&noteId=tGw3QZiByT, Detailed paper review: https://openreview.net/forum?id=nplYdpc1Pm&noteId=lfCYvH8OhT) along with the limitations (https://openreview.net/forum?id=nplYdpc1Pm&noteId=jj9j5D2lq6) of our work. Kindly review this and if it satisfactorily answers your concerns then reward us with a better score.

Thanks. --Authors.

Respected reviewer,

If you don't have any further arguments related to your queries, kindly update your score. If you do have arguments or queries, please post your rebuttal so that we can address them. We respect your position of responsibility and time, and humbly expect the same courtesy from your end.

From our side, we have addressed all your concerns and accepted all your insightful suggestions. However, on your end, we expected you to either acknowledge or update the score.

Thanks.

We are thankful for the insightful reviews and comments on our work. We are working on refining our subsmission and adressing all the questions marked in the review. Looking forward towards this discussion.

Thanks, --Authors.

We thank the reviewer again for the insightful questions. We wish to answer the above questions in the following order. Also, if all the concerns have been adressed well, then we humbly request the reviewer to update the current score. Thank you :

Answers to the Questions:

Q1. what is the advantage of 2-stage training vs 1stage training?

Answer. The reviewer has correctly noted the impact of our two-stage training approach. To address this, we have included results from both the single-stage (TerminAL B only) model and the two-stage (TerminAL A + B) model in Tables 2 and 3. These results clearly demonstrate the contribution of Stage A training to the overall performance of the Stage B model. Specifically, as shown by the results, we validate our hypothesis that the model struggles to distinguish between multiple audio sources when trained with only Terminal B. We kindly request the reviewer to refer to Table 3, where we present comparative results for the model trained without Terminal A (i.e., without prior knowledge of multiple sound distinctions) and the model trained with Terminal A + B (where Terminal A knowledge is first introduced). Our observations confirm that the two-stage model exhibits a better understanding of temporality across various ZSTE tasks, substantiating the importance of our proposed methodology.

Q2. COMPA-Order and COMPA-attribute missing in the main text

Answer The reviewer rightly points out the absence of zero-shot metrics like Compa-Order, Compa-Attribute (Ghosh. et. al.), and similar evaluations. Previous studies (Wu. et. al.) have demonstrated that existing zero-shot evaluation methods often fail to capture the general understanding of models comprehensively. While Compa includes such observations and establishes similar ZS metrics with slight variations in prompt design (e.g., Compa Order and Compa Attribute evaluations), their focus is primarily on assessing the model's general-purpose zero-shot capabilities.

In contrast, our current work introduces an improved evaluation metric that not only encompasses but also broadens the scope of these ZS metrics to provide more generalized and meaningful evaluation strategies. Additionally, we would like to highlight that we are also working on follow-up research to further enhance the generalizability of models beyond the scope of the current work. However, we believe that this paper represents a critical first step toward achieving more impactful and robust evaluations of general intelligence, and we appreciate the opportunity to advance the discourse in this area.

Overall Remark:

We would like to elaborate on our objectives, the claims associated with each, and the strong proofs presented as results. We kindly request the reviewers to thoroughly review these points, as they should provide a clearer understanding of the contributions and significance of our work.

Our Objectives (as presented in the paper):

1. Establishing a multi-stage temporal instillation objective:

The first stage focuses on learning to distinguish multiple sounds, while the subsequent stage emphasizes learning the temporal arrangement of these sounds.

• Claim: Training a single model to directly learn temporal relationships is ineffective because models must first develop the ability to distinguish between different sounds. Without this foundational understanding, learning temporality becomes infeasible.
• Proof 1: To validate this claim, we compared the model's performance after applying only TeminAL B with the performance after employing the multi-stage training process (TeminAL A followed by TeminAL B). The results show that the multi-stage model significantly outperforms in all tasks defined by ZSTE—a more advanced progression of the classical zero-shot evaluation strategy widely used in prior work. The results are detailed in Table 2.
• Proof 2: We further compared our model with state-of-the-art (SOTA) contrastively trained ALMs across all ZS evaluation tasks. Specifically, in ZSTE Task 2 (subtasks A and C), which require distinguishing multiple sounds, our model outperforms models trained on large datasets. However, in subtasks 2B and 2D (which involve simpler mappings of individual sounds), models trained on massive datasets excel due to their extensive exposure. For Tasks 3A and 3B, all models perform comparably, highlighting the challenges of zero-shot evaluations when trained on similar examples. These findings underscore the necessity of multi-stage training for better generalization.
• Note: Our training was intentionally restricted to a subset of the total parameters and used only the ESC-50 dataset. This decision aligns with our goal of demonstrating results within a realistic compute budget. Reviewers are kindly requested to consider that the focus of this work is not on achieving SOTA performance across billion-sample datasets, but rather on addressing a critical gap in current ALM post-training methods. Our key contributions lie in introducing a novel multi-stage training strategy and an effective evaluation framework.

2. Demonstrating correct temporal relationships between audio and text:

• Claim: Our analysis indicates that current contrastive ALMs face challenges in accurately capturing temporal relationships between audio and text.
• Proof: This claim is best evaluated after understanding our first objective. We acknowledge that the presentation order of objectives in the manuscript might have caused confusion, and we apologize for any inconvenience caused. However, the current order was chosen to maintain the logical progression of ideas.

3. Proposing a general-purpose zero-shot evaluation scheme:

• Claim: We are not claiming anything here, but providing and using a structured way of evaluating our earlier claims.
• Proof: Existing evaluation metrics fall short of effectively assessing temporal relationships, particularly in our case. We propose this evaluation scheme as a simple yet intuitive method to capture model behavior accurately. While future enhancements to evaluation metrics are certainly necessary, this scheme is sufficiently robust for our purposes.

Final Note:

If the addressed weaknesses and clarifications satisfactorily resolve the reviewers' concerns, we kindly request an updated score to reflect the potential impact of this work in advancing the field of audio-language models. Thank you.

We sincerely thank the reviewers for their thoughtful feedback, which has been invaluable in enhancing the quality of our paper. We deeply appreciate the recognition of our work's strengths and the constructive identification of areas for improvement. Below, we provide a detailed response to each identified weakness, address the specific questions raised, and outline the changes incorporated into the revised submission. (Answer to the questions in the next comment, thanks)

We respectfully request the reviewers to carefully consider our explanations and arguments, as we believe they are integral to advancing the understanding we aim to contribute to the community through this work. We remain fully open to further discussions and are committed to refining the manuscript to better align with the expectations of the reviewers and the broader research community.

Weaknesses:

W1. Comparison with Compa's modular contrastive training approach

Thank you for highlighting this point. There is indeed a distinct difference between our sequential learning approach and the modular contrastive approach mentioned by Compa. While Compa trains the model to become modular by introducing progressively complex texts—essentially akin to introducing new types of labels in steps—our model adopts a fundamentally different strategy. We initially train the model to recognize that multiple distinct sounds can coexist, followed by training on these multiple sounds. This approach is more interpretable and structured in its methodology. We have now included these comparison points in the main text (Line 93-109)

• Compa’s final objective remains the same across their training steps, and they do not provide a comparative analysis to demonstrate what the model gains in knowledge from the initial training on hard negatives. While they report percentage improvements in performance, they do not substantiate this with reasoning, which makes their method less interpretable.

• Unlike prior works, such as those by Ghosh et al. (2023) and Wu et al. (2023a), which rely on large datasets and substantial computational resources, our focus is on developing a methodology that instills a temporal sense within the model while remaining within realistic computational constraints. Our emphasis is on this methodological advancement rather than on generalization over expansive and diverse datasets.

W2. Rationale behind using ESC50

We deeply appreciate this important point raised by the reviewer. We have addressed this in Section B2 (line 894 onwards) of our paper. As the work is authored by student researchers, it is important to recognize the implications of using a single dataset. Expanding the dataset would necessitate an increase in the model size (i.e., trainable parameters) to counteract the high bias introduced by increased dataset diversity. This would go beyond the scope and budget of our current work. Hence, we strategically focused on highlighting critical gaps in the current state of Audio-Language Model (ALM) research, particularly in multi-modal research, while adhering to realistic training constraints. We are immensely thankful to the reviewer for raising this point, as it underscores the deliberate choices we made to balance the scope and impact of our project.

W3. ZS experiments on standard audio-classification

We acknowledge the reviewer’s observation regarding the absence of standard audio classification results on datasets like FSD50K and USD8K. Our primary focus in this paper was to address temporal instillation and move beyond standard Zero-Shot (ZS) tasks. The objective of our work is to provide a meaningful step toward achieving these goals, as evidenced by our results compared to other SOTA models. That said, we accept the reviewer’s suggestion and have now included classification results from ESC-50 and USD8K in Table 4. We are also planning to incorporate results from FSD50K, contingent upon the timeline for submission. It is important to note that due to our focus on training with limited computational resources, we included only ESC-50 in our post-training dataset. As such, we rely on the original training of the CLAP model over these datasets, which explains the observed decrement in performance. However, our results demonstrate that the model does not undergo catastrophic forgetting, as evidenced by the competitive results on benchmark retrieval datasets such as Clotho and AudioCap.

Respected reviewer isZu,

Along with the above updates on your questions, we have taken the time to adress all the reviews and provide an in-depth detail of the summary (The response can be found up top in the comment section as "Final updates and addressing the reviewers" , https://openreview.net/forum?id=nplYdpc1Pm&noteId=tGw3QZiByT, Detailed paper review: https://openreview.net/forum?id=nplYdpc1Pm&noteId=lfCYvH8OhT) along with the limitations (https://openreview.net/forum?id=nplYdpc1Pm&noteId=jj9j5D2lq6) of our work. Kindly review this and if it satisfactorily answers your concerns then reward us with a better score.

Thanks. --Authors.

We are thankful for the insightful reviews and comments on our work. We are working on refining our subsmission and adressing all the questions marked in the review. Looking forward towards this discussion.

Thanks, --Authors.

Overall Remark:

We would like to elaborate on our objectives, the claims associated with each, and the strong proofs presented as results. We kindly request the reviewers to thoroughly review these points, as they should provide a clearer understanding of the contributions and significance of our work.

Our Objectives (as presented in the paper):

1. Establishing a multi-stage temporal instillation objective:

The first stage focuses on learning to distinguish multiple sounds, while the subsequent stage emphasizes learning the temporal arrangement of these sounds.

• Claim: Training a single model to directly learn temporal relationships is ineffective because models must first develop the ability to distinguish between different sounds. Without this foundational understanding, learning temporality becomes infeasible.
• Proof 1: To validate this claim, we compared the model's performance after applying only TeminAL B with the performance after employing the multi-stage training process (TeminAL A followed by TeminAL B). The results show that the multi-stage model significantly outperforms in all tasks defined by ZSTE—a more advanced progression of the classical zero-shot evaluation strategy widely used in prior work. The results are detailed in Table 2.
• Proof 2: We further compared our model with state-of-the-art (SOTA) contrastively trained ALMs across all ZS evaluation tasks. Specifically, in ZSTE Task 2 (subtasks A and C), which require distinguishing multiple sounds, our model outperforms models trained on large datasets. However, in subtasks 2B and 2D (which involve simpler mappings of individual sounds), models trained on massive datasets excel due to their extensive exposure. For Tasks 3A and 3B, all models perform comparably, highlighting the challenges of zero-shot evaluations when trained on similar examples. These findings underscore the necessity of multi-stage training for better generalization.
• Note: Our training was intentionally restricted to a subset of the total parameters and used only the ESC-50 dataset. This decision aligns with our goal of demonstrating results within a realistic compute budget. Reviewers are kindly requested to consider that the focus of this work is not on achieving SOTA performance across billion-sample datasets, but rather on addressing a critical gap in current ALM post-training methods. Our key contributions lie in introducing a novel multi-stage training strategy and an effective evaluation framework.

2. Demonstrating correct temporal relationships between audio and text:

• Claim: Our analysis indicates that current contrastive ALMs face challenges in accurately capturing temporal relationships between audio and text.
• Proof: This claim is best evaluated after understanding our first objective. We acknowledge that the presentation order of objectives in the manuscript might have caused confusion, and we apologize for any inconvenience caused. However, the current order was chosen to maintain the logical progression of ideas.

3. Proposing a general-purpose zero-shot evaluation scheme:

• Claim: We are not claiming anything here, but providing and using a structured way of evaluating our earlier claims.
• Proof: Existing evaluation metrics fall short of effectively assessing temporal relationships, particularly in our case. We propose this evaluation scheme as a simple yet intuitive method to capture model behavior accurately. While future enhancements to evaluation metrics are certainly necessary, this scheme is sufficiently robust for our purposes.

Final Note:

If the addressed weaknesses and clarifications satisfactorily resolve the reviewers' concerns, we kindly request an updated score to reflect the potential impact of this work in advancing the field of audio-language models. Thank you.

We sincerely thank the reviewers for their thoughtful feedback, which has been invaluable in enhancing the quality of our paper. We deeply appreciate the recognition of our work's strengths and the constructive identification of areas for improvement. Below, we provide a detailed response to each identified weakness, address the specific questions raised, and outline the changes incorporated into the revised submission.

We respectfully request the reviewers to carefully consider our explanations and arguments, as we believe they are integral to advancing the understanding we aim to contribute to the community through this work. We remain fully open to further discussions and are committed to refining the manuscript to better align with the expectations of the reviewers and the broader research community

Weaknesses:

W1. Title of Figure 3:

Thank you for pointing this out. We accept this comment and have modified the title of Figure 3 for clearer message delivery. Since Figure 3 serves as an overview of the model architecture, we have provided an in-depth explanation of the loss function in Figures 6 and 7. Additionally, we have included references to these figures in Section 3.4 to improve ease of understanding and relatability.

W2. Loss Function Formulation for TeminAL A:

We acknowledge the reviewer's suggestion and have included a figure demonstrating the loss function formulation for TeminAL A in the relevant section. Due to space constraints, we have placed detailed derivations in the appendix, in line with current trends in conference papers. However, key equations are retained in the main section, starting from line 366, including Equations 8 and 9. Further derivations are provided in the supplementary material from Equation 22 onward. Morever we have also included Figure 7 for reference to understand TeminAL A loss function.

W3 & W4. Highlighting ZSTE in the Main Paper:

We appreciate the suggestion to better highlight ZSTE in the main paper. The whole Section 4.2 has been revised to incorporate necessary information from Appendix Section B.4. Due to space limitations, we have moved some background details and additional diagrams to the supplementary section. Nonetheless, all important impacts and applications of ZSTE are now detailed in the Results section (Section 5) as we have re-written the whole section. The results section includes Table 1, Table 2, and explanatory paragraphs in Section 5, which specifically highlight the role of ZSTE in our current work. The details of ZSTE are now shifted to main text from the tables as per suggestion.

W5. Figure Fonts:

The font size of the figures has been increased, following the reviewer's helpful suggestion.

W6. Clarity on Embeddings and Variables:

We agree with the reviewer's observation regarding the embeddings going into the loss functions, denoted as $z$, and the formulation of $l_{\text{text}}(C)$. These aspects have been clarified to ensure better understanding of the variable usage.

Questions:

Q1. Source of the results from table 1

Answer Yes, the results of the model in Table 1 have been taken from previous published results and the sources have been cited. The citations have been included the caption of the figure as well.

We thank the reviewer again for the insightful comments, if all the parts of the weknesses and the questions have been adressed well in our modifications , we request the reviewer to kindly update the score of our paper. In order to mark the impact of the work for the development of such models.

Respected Reviewer,

We sincerely thank the reviewer for their thoughtful feedback and guidance, which have greatly contributed to improving our work. We appreciate your acknowledgment of the updates made in response to your earlier comments. We kindly request additional feedbacks on how we can further refine our manuscript to address your concerns and ensure it aligns with the standards for acceptance. Based on the current score, we feel that we may not have effectively communicated the key contributions and strengths of our work.

Furthermore, if our efforts to address your concerns regarding novelty, the proofs provided, and the discussions in the paper have met your expectations and if the reviewer thinks the contribution in this work is novel and should be highlighted, then it's a humble request to score this work towards acceptabiity and help us share this knowledge towards the general development of the scientific community.

With gratitude,

--The Authors

Respected reviewer SKUo,

Along with the above updates on your questions, we have taken the time to adress all the reviews and provide an in-depth detail of the summary (The response can be found up top in the comment section as "Final updates and addressing the reviewers" , https://openreview.net/forum?id=nplYdpc1Pm&noteId=tGw3QZiByT, Detailed paper review: https://openreview.net/forum?id=nplYdpc1Pm&noteId=lfCYvH8OhT) along with the limitations (https://openreview.net/forum?id=nplYdpc1Pm&noteId=jj9j5D2lq6) of our work. Kindly review this and if it satisfactorily answers your concerns then reward us with a better score.

Thanks. --Authors.

We are thankful for the insightful reviews and comments on our work. We are working on refining our subsmission and adressing all the questions marked in the review. Looking forward towards this discussion.

Thanks, --Authors.

We sincerely thank the reviewers for their thoughtful feedback. Below, we provide a detailed response to each identified weakness, address the specific questions raised, and outline the changes incorporated into the revised submission.

We respectfully request the reviewers to carefully consider our explanations and arguments, as we believe they are integral to advancing the understanding we aim to contribute to the community through this work.

Weaknesses:

W1. Limited Novelty and Generalization

We thank the author for mentioning the two--stage training contribution for our work. And we do agree that the method used in our work has been explored in the past. However we don't think one can look at these two aspects individually, being able to instill time through contrastive training on modified samples is a general way of doin this, but doing it correctly is what we have emphasised in our work. And in order to provide evidence to our claims we have presented sufficient results in section 5. Although we humbly accept that earlier the results were not presented well in the document. However, we have made necessary changes in order to get across our message. The following is our take on how we prove our claims around novelty and generalisation.

• Claim: Training a single model to directly learn temporal relationships is ineffective because models must first develop the ability to distinguish between different sounds. Without this foundational understanding, learning temporality becomes infeasible.
• Proof 1: To validate this claim, we compared the model's performance after applying only TeminAL B with the performance after employing the multi-stage training process (TeminAL A followed by TeminAL B). The results show that the multi-stage model significantly outperforms in all tasks defined by ZSTE—a more advanced progression of the classical zero-shot evaluation strategy widely used in prior work. The results are detailed in Table 2.
• Proof 2: We further compared our model with state-of-the-art (SOTA) contrastively trained ALMs across all ZS evaluation tasks. Specifically, in ZSTE Task 2 (subtasks A and C), which require distinguishing multiple sounds, our model outperforms models trained on large datasets. However, in subtasks 2B and 2D (which involve simpler mappings of individual sounds), models trained on massive datasets excel due to their extensive exposure. For Tasks 3A and 3B, all models perform comparably, highlighting the challenges of zero-shot evaluations when trained on similar examples. These findings underscore the necessity of multi-stage training for better generalization.
• Note: Our training was intentionally restricted to a subset of the total parameters and used only the ESC-50 dataset. This decision aligns with our goal of demonstrating results within a realistic compute budget. Reviewers are kindly requested to consider that the focus of this work is not on achieving SOTA performance across billion-sample datasets, but rather on addressing a critical gap in current ALM post-training methods. Our key contributions lie in introducing a novel multi-stage training strategy and an effective evaluation framework.

2. Demonstrating correct temporal relationships between audio and text:

• Claim: Our analysis indicates that current contrastive ALMs face challenges in accurately capturing temporal relationships between audio and text.
• Proof: This claim is best evaluated after understanding our first objective. We acknowledge that the presentation order of objectives in the manuscript might have caused confusion, and we apologize for any inconvenience caused. However, the current order was chosen to maintain the logical progression of ideas.

3. Proposing a general-purpose zero-shot evaluation scheme:

• Claim: We are not claiming anything here, but providing and using a structured way of evaluating our earlier claims.
• Proof: Existing evaluation metrics fall short of effectively assessing temporal relationships, particularly in our case. We propose this evaluation scheme as a simple yet intuitive method to capture model behavior accurately. While future enhancements to evaluation metrics are certainly necessary, this scheme is sufficiently robust for our purposes.

In order to make the submission better and to encompass everything which we state above, the whole Section 4.2 has been revised to incorporate necessary information from Appendix Section B.4. Due to space limitations, we have moved some background details and additional diagrams to the supplementary section. Nonetheless, all important impacts and applications of ZSTE are now detailed in the Results section (Section 5) as we have re-written the whole section. The results section now includes Table 1, Table 2, and explanatory paragraphs in Section 5.

We sincerely appreciate the reviewer for taking the time to provide highly valuable comments on our responses. To address the concerns efficiently, we wish to provide our replies to the above-mentioned comments. If necessary, we will offer an extended response before the final interaction window to ensure that our message is conveyed completely and accurately.

1. Responding to the weakness mentioned by the reviewer :

• 1a. and 1b. Changes in the motivation: We accept that we have made changes in the motivation section, but we have not tried to re-align the paper. Although we do accept that the initial message was not coming though well, and all the credits for the update only goes to the insightful reviewers of the paper. But I wish to present examples from the original submitted paper, where we have explicitly mentioned the limited budget post--training of CLAP on a limited dataset (here being ESC--50) , <The following are lines and sections from the original submission on 10/01/2024>

• a) Figure 2 and 3 : Here we had schematically shown only a subset of the layers as trainable, while adapting the origianl encoders with temporality.

• b) Line 310 : Encoders are not trained from scratch; we extend our framework using a pre- existing audio-language model comprising an audio encoder $fc_\theta$ and text encoder $fc_\phi$ as shown in figure 3.

• c) Line 210 : Details of our study on the limited dataset ESC--50. Followed by section B2 for the details.

• d) Line 1271 : The whole section B.5 of the original submitted paper mentions our training paramters as well as cost of training.

• Accepting sections from our original submission which were confusing: We would also like to humbly accept the confusing section of the initial submission which may have resulted in the confusion of the intent. Again all thanks to the reviewers for their continued effort and insightful suggestions.

• a) Line 314 : Due to limited dataset size, selective refinement of specific layers within $\Theta$ is performed. Here we wanted to convey the message of trade-off between dataset size and trainable parameters in the perspective of overfitting and budget, but failed to do so in the original submission.

• b) Section 4.1 : We accept the confusing lines in this section, we wanted to convey what we train on the baseline model. We are very thakful for the the reviewer for pointing out this confusion. We have updated our section in the main paper to avoid this confusion.

• 1c. : Results overfitting on ESC--50:

• We do accept this remark from the reviewer and think it's an important remark in general toward model post--training. Although, as we emphasis in our objective and scope of work, that our aim is not give a generalisable model which perform SOTA across all datasets. Rather our aim is to present the idea of multi--stage training of contrastive models and it should effectively be done in the sense of instillation of time.

• Hypothesis for 1c. : Evaluating results of other mentioned dataset would not be able to give better result than the origianl CLAP model, as we never post--trained on those specific datasets. Henceforth encountering any OOD sample would mean feeding unkown data to the model to hallucinate on. While the training can well be extended to different datasets or to a better mixture of datasets, but that defeats the purpose of being able to train within our compute budget. Although this point has been well noted for future work which will emphasis solely on giving generalisable post--trained models. We request the reviewer to share their view on our hypothesis.

• 1d. Confusing statement on whether we have fully trained our own CLAP model or not?

We again thank the reviewer for pointing to this confusion. The reviewer is absolutely correct in noting that we never train the CLAP model from scratch, we rather align the pre-trained model with our post--training strategy. As mentioned in the response of 1.a. we humbly accept these confusing lines and have updated the section 4.1 for better clarity.

Note : We would also wish to make a remark on authors comment on our method being computational effective method, we will abstain ourselved for making that statement, as compute effective and compute efficient training are broader scope terms which would require heavy abalations for prooving. However, we would wish to only inlcude statements on compute limitation and budget, which does distinguish our work from others but only in terms of providing an effective multi--stage training strategy rather than giving a compute efficient generalisable model which is performant across all data distributions.

We highly value the reviewer's summary, although we would humbly wish to share our response such that our objectives and scope are communicated well.

Response to reviewer's summary

• We accept the remarks by the reviewer on viewing the effectiveness through post--training on multiple different datasets. We would like to present our thoughts both positive and counter-positive (not in the theoretical sense, but in the terms of scope of our current work.)

• Increasing knowledge base of the model: The positives is fairly straightforward, as we would get better underdantind on how the method would perform on variations of differnet kinds of datasets. However, we would like to comment that since the data space of current Audio--Language contrastive models are all based on single event monoaural sounds, with non-descriptive caption. The extension of the dataset would only increase the knowledge base of the current model and hence--forth better performance. Although it would not bring a fundamental change in terms of generalisable multi--modal learning. And as stated previously our work focuses only on providing an effective multi--stage training strategy rather than giving a compute efficient generalisable model which is performant across all data distributions.

• Inceraisng model complexity to overcome over-fitting : Extension on multiple different datasets would effectively involve incereasing the learnable parameters of the model to avoid over-fiting of the models which makes the work go out of scope in terms of compute. Hence, defeats our objective of the project.

Response to minor comments

2. Providing abalation for TeminAL A: We note the reviewer's concern on providing abalation study for TeminAL A. However, we would like to put forth our thinking about judging effectiveness through providing abalation on TeminAL B and TeminAL A+B. We would humbly like to suggest that, the abalation presented in Table 2 and results of Table 3 between TeminAL B and TeminAL A+B, actually serves this purpose and no new information can be broghut upon by abalation on TeminAL A.

• It may help by thinking this way, we wish to capture the gain due to TeminAL A, hence model with TeminAL A (A+B) and model without TeminAL A (Only B), are the necessary and sufficent requirements. Assume model tarining as vectors getting scaled and transformed, initial vector be $v_i$.

• Let TeminAL A transform it to $v_a$ (scale by $\alpha$ and rotate by $\theta$) and further teminal B transform it to $v_{ab}$ (further scale by $\beta$ and rotate by $\phi$), while direct transformation would be $v_b$ (i.e $v_i$ scale by $\beta$ and rotate by $\phi$) . Henceforth $v_b$ is actually $v_i$ scaled by $\beta * \alpha$ and rotate by $\phi + \theta$. Hence, even without the information of the transformation incurred due to TeminAL A (i.e $v_a :$ $\alpha$ and $\theta$), we can infer it through only $v_b$ and $v_{ab}$. Application of monotonic non-linearity will make the above non-computational and hence we won't comment on quantitative aspects (i.e how much effect, or exact values of $v_a :$ $\alpha$ and $\theta$), but qualitatively the general trend would preserve.

3. Addition of temporal words : We accept the suggestion of the author that increasing the context of temporal words would increase the generalisabilty. But would humbly argue that this would still not provide a generalisable solution (something which we are currently working on as well). Again circling back to the scope of the current work, that we neither wish nor are capable enoguh in this project to provide a generalisable model. Although we only focus on providing an interpretible way of multi--stage contrative training of ALMs to instill the specific case of temporality.

Final remarks and request : We sincerely appreciate the reviewer for taking the time to provide highly valuable comments on our responses. We highly value the challenges mentioned by the reviewer but we also like to mention that it would be out of our scope to tackle all the challenges in one work. However, If the addressed weaknesses and clarifications satisfactorily resolve the reviewers' concerns specifically to our current scope, then we kindly request the reviewer to reward us a better score.

Thank you.

I really appreciate the response from the authors on my concerns on this submission. I carefully read all these responses and list my feedback below:

I agree the below responses and will update the score accordingly:

1. The revision on section 4.2, the change of the title, and the revision on demonstrating the TB / T-AB experiments and 5 ZSTE tasks make the presentation of this paper more clear.

2. The contribution of proposing a ZSTE benchmark is valuable.

I disagree the below responses and give my reasons on it:

1. The underperformance from the proposed T-CLAP to the previous models (e.g., CompA) on subtasks 2B and 2D is due to to the limitation of the dataset and computational resources.

Concern: there are some big changes on the paper statements and motivations for the revised version. Unfortunately Openreview does not allow to access the paper at the version before rebutall (Nov 19). But I do have one saved copy when I reviewed the paper:

1.a. In the revised submission and the rebuttal, "In contrast, our approach achieves significant advancements within a limited computational budget, training around 10% of the total trainable parameters and utilizing a single dataset (ESC-50)."

1.b. In the original submission, the above statement was not presented.

The authors explained the reason of underperformance of subtasks 2B and 2D on the limited datasets and computational resources and this becomes one of their proposed contribution. I strongly disagree with this point not only because this was not stated in the original submission, but also given the fact that:

1.c. For a contrastive learning model that commonly requires a large-scale training paradigm, it remains very unconfident that if the proposed results actually overfit to the ESC-50. I would like to request the result of ZSTE on other testset (such as processed one from DESED, UrbanSound, AudioSet Strong, etc.)

1.d. Admittedly, the pre-training of the CLAP model involves more datasets listed in Appendix B.5-Table 6. In that, we actually cannot conclude that the proposed method is a computational effective method "just because the post-training process only involves ESC-50". In fact, this makes me confuse about another statement the authors made in section 4.1: "We train a CLAP model Elizalde et al. (2023) using transformer-based encoders: HTSAT Chen et al. (2022) for audio and BERT Devlin et al. (2018) for text, each with a projection layer." So it seems like the authors never trained the CLAP model from scratch but just used some pre-trained checkpoints from the previous works (and then did the post-training with ESC-50?

In summary, I still cannot fully assess the effectiveness of the proposed "correct method on temporal enhancement". This still aligns my initial statement: "it would have been more effective to apply the post-training method to 2-3 different CLAP models and compare their performance with and without the post-training."

Minor comments:

2. It is still valuable and necessary to have Teminal-A-only ablation study on the results to fully assess the effectiveness of Terminal-B. Otherwise it would be really uncertain how Terminal-B method can further improve the temporal awareness of the model reflected in the performance.

3. The generalization of the temporal template words (before, after, while) should be further considered to add more words.

In summary, I raise my score as 5 for the revision of the paper representation and organization, but remain unsatisfied with other aspects of the submission.

Respected reviewer YCxA,

Along with the above updates on your questions, we have taken the time to adress all the reviews and provide an in-depth detail of the summary (The response can be found up top in the comment section as "Final updates and addressing the reviewers" , https://openreview.net/forum?id=nplYdpc1Pm&noteId=tGw3QZiByT, Detailed paper review: https://openreview.net/forum?id=nplYdpc1Pm&noteId=lfCYvH8OhT) along with the limitations (https://openreview.net/forum?id=nplYdpc1Pm&noteId=jj9j5D2lq6) of our work. Kindly review this and if it satisfactorily answers your concerns then reward us with a better score.

Thanks. --Authors.

Dear Reviewers and Chairs,

We would like to ask the Chairs to kindly notify the reviewers as well. As the author interaction window approaches its conclusion (November 26 at 11:59 pm AoE), we kindly ask you to review our rebuttal responses and consider whether they adequately address your concerns. If you find that all your concerns have been addressed, we would humbly request you to consider updating your scores accordingly.

Thank you.

Respected Chair,

Thanks a lot for you help. Following the author's guide (https://iclr.cc/Conferences/2025/AuthorGuide), may you kindly tell us how to update our paper's title, we are unable to do so with the current frontend setup. We wish to change our paper's title to "Temporal Enhancement of Contrastive Audio--Language Models through Self--Supervised Post--Training with Text--Audio Pairs". This has already been communicated to the reviewers (https://openreview.net/forum?id=nplYdpc1Pm&noteId=04VFUnDzfs). Moreover, this is accounting the suggestion given by the insightful suggestion of Reviewer YCxA.

Sincerely,

--Authors.

Final Note:

As the updation of the drafts reach its final day, we have rigoursly went through each reviewer's comment and tried to address it systematically. If the final version of the paper and our supported responses address your concerns and questions, then we kindly request an updated score to reflect the novelty and potential impact of this work in advancing the field of audio-language models and multi-modality. Thank you.

• Summary:

1. [Lines 38-49]: Multimodal learning and its capabilities towards progressing aritifical general intelligence. Here, we particularly focus on contrastive learning models and its applications, specifically focussing on Image-Language CLIP [1].
2. [Line 50-86]: Extending contrastive models to different modalities like Audio-Language model (ALM) CLAP [2] and also highlighting issues in the current ALMs particularly having bias towards specific words rather than their arrangment or context [3]
3. [Line 87-96]: We describe various previous model and how they adress the drawbacks stated above. Mainly focussing on Compa-CLAP [4] and how it addresses the problem with attribution and composition training.
4. [Line 95-105]: We address the challenges which previous model face, specifically (a) Distinction of multiple sounds before instilling the understanding of temporality, (b) Extending upon this understanding towards building temporal understanding. Contrasting to the previous approaches we implement while operating in a limited computational budget while training only 10% of the total trainable parameter of the base CLAP model.
5. [Line 106-119]: A brief overview on our model and training for the introduction. Details of which is discussed later in specific section
6. [Line 120-127]: Our introduction to the ZSTE evaluation and why it's needed as cited be previous authors. We abstain from claiming it to be general purpose as we have not shown it to be. Although intuitively the scheme can be generalised to other modality with ease.
7. [Line 128-144]: Our main contribution, claims and proofs of which have already been detailed in https://openreview.net/forum?id=nplYdpc1Pm&noteId=YpDrcK1rig for the readers.
8. [Line 148-161]: Background on different audio-language models both open and closed ended and their applications. Emphasising on why the work is important and relevant for the multi-modal articifial intelligenece community.
9. [Line 164-176]: After intoroducing the importanace of ALMs, we wish to intorduce previous applications of Self-supervised learning and Post-training particularly in the domain of contrastive learning, so that the reader gain context on the domain we wish to operate on.
10. [Line 179-194]: Finally in the background section, we wish to cover Zero-shot models and its limitations, as we further propose our own scheme for zero-shot evaluation (ZSTE). Sumarising the limitations, classical ZS inference schemes may lack to properly capture the context (in whichever sense, be it attribution, POS or temporal).
11. [Line 199-220]: In order to address our objectives which may be used to solve the problems with ALMs and Evaluation schemes which have been listed in the previous sections, we propose our method focussing on contrastive training of a pre-trained ALM (CLAP) with temporally augmented samples. 12.[Line 296-313]: We introduce our objective function for training which contains a 2-step training objective TeminAL A which is used to distinguish multiple sounds followed by TeminAL B used to contrastively train to instill the sense of time in it.
12. [Line 314-315]: Information on how we form our batches of data following section 3.2 for the actual training, followed by introduction to our limited parameter training (limiting to specific few layers of the encoders.) with the net Loss function $L_B$= $L_{c_B}$ + $\beta$ $(L_{a_B})$
13. [Line 331-377]: Defining the textual Loss $(L_{c_B})$ of Terminal B in eq(3) followed by defining all it’s terms in eq 4,5 and 6. Followed by Defining audio loss $(L_{a_B})$ in eq(7) followed by all its terms in eq 8 and 9. Similar formulation of loss function for Teminal A can be found in appendix B.3.
14. [Line 407-422]: The impact of hyper-parameters $\{\alpha_s, \alpha_c, \alpha_{st}, \alpha_{ct}\}$ and $\{\beta_A, \beta_B\}$ on the temporal sensitivity of encoders. These coefficients enhance sensitivity to time-reversed samples and dataset adaptability, enabling the model to treat similar and dissimilar pairs differently. The inclusion of these hyper-parameters refines the encoder's temporal understanding and ensures distinct sample treatment.
15. [Line 429-431]: Introduction to our base model CLAP and its components.
16. [Line 434-443]: Our model evaluation method ZSTE and downstream tasks (Audio & Text retrieval and Audio Classification). ZSTE is divided into 5 tasks where Task 2,3,4 and 5 have 4,2,2,2 subtasks each. While downstream tasks are evaluated for T-A and A-T retrievals on AudioCap/ Clotho dataset. And classification is evaluated on (ESC-50 and US8K dataset).

We humbly accept the suggestions by the reviewers and have made a full list of our model's limitation. Importantly it is to be noted that all these limitations are well out of scope of our current work and hence doesn't take away any novelty from our work.

What our work doesn't cover? A note to all future authors interested in this work:

• General-Purpose ALM: Our proposed model is not a general-purpose Audio-Language Model (ALM) capable of performing all downstream applications across all datasets. The current implementation is specifically designed and validated on the ESC-50 dataset as a proof-of-concept and to achieve our defined objectives. Consequently, generalization remains a limitation, although addressing this was beyond the scope of our work.
• Temporality Beyond the Dataset: The model does not provide a general understanding of temporality beyond the ESC-50 dataset. Results from the ZSTE Task 4 and 5 confirm that neither does the model propose general temporality, nor does it achieve it. Notably, we emphasize that achieving general-purpose temporality would require larger, more comprehensive pre-trained text encoders. Specifically, open-ended text encoders (e.g., encoders from encoder-decoder models) would be more suitable than encoders trained on closed masked language modeling techniques, such as BERT.
• Zero-Shot Evaluation Scheme: While our zero-shot evaluation scheme is designed to be general-purpose, it is inherently limited to contrastive models. Furthermore, the evaluation has not been tested on domains beyond the ESC-50 dataset. The reported ZSTE results are restricted to this dataset because it aligns with the training data used in our model and those of prior works. To mitigate data leakage, we ensure that our evaluation dataset is separate from the training data. However, it is important to note that the primary objective of this evaluation is to validate the proof-of-concept rather than to set new benchmarks.
• Evaluation Dataset Overlap: A broader limitation, relevant to most models in this domain, is the overlap between evaluation datasets across various benchmarks. These overlaps can occur in terms of sound events or contextual similarities. Therefore, we caution against uncritical comparisons of model performance on classical benchmarks without careful consideration of dataset overlap.

By addressing these limitations, we hope to guide future researchers in expanding and improving upon the current work to achieve broader applicability and more generalized performance.