Overall, while the proposed method is solidly novel and seems to perform better than current SOTA training-free editing approaches, issues in the overall clarity of the paper, evaluation suite, and in particular the proposed benchmark overweigh the contributions and thus I recommend rejection.

Overall Clarity

---

The paper contains a number of grammatical errors, incorrect names of things, and incorrect citations.

• The “Branch” term (line 070) is introduced without explanation.
• "rigid" and "non-rigid" edits are introduced without explanation. These terms are not elucidated by figure 1, as there seems to be no difference between the input prompt and the "non-rigid" prompt.
• (Line 143) What is “plug-in-plus”? Is this a typo for plug-and-play?
• (Line 369) What is ZoMo-bench? Is this a typo from ZoME-Bench?
• It should be “MedleyDB” not “MelodyDB” (line 370)
• Appendix C seems to contain the same information duplicated twice (874-880 and 881-892)
• MusicCaps has the incorrect citation on line (161), which should point to Agostinelli et al.

While the above issues are minor, more importantly there is a distinct lack of clarity in the methodological contributions and what are contributions from the authors.

• In particular, it feels like the paper suffers from overuse of acronym-like terms. Between Disentangled Inversion Control, Harmonized Attention Control, and Disentangled Inversion Technique, it is hard to tell what is a subset of what.
• In section 3.3.1. even more acronym-like terms are introduced (Global Attention Refinement and Local Attention Blend), and it is unclear both A) why are these getting special names if they are from existing work where these terms are not used and B) if these are novel contributions, this is not made explicit.
• In Equation 2, there is a lone $M_t$, and it is unclear which attention map this refers to
• In equation 3/4, the threshold functions takes $w_{src/tgt}$ as argument but in equation 6 it is only a function of the mask and the threshold
• Both 3.3.1. and 3.3.2 are realtively hard to follow without an intimate knowledge of past (image-domain) works. These sections as a whole could be made more clear by drawing specific examples to editing tasks.
• 3.3.3 (and algorithm 2), it is a bit unclear whether “forward” refers to the forward diffusion process (i.e. data $\rightarrow$ noise) or the “forward” pass of the model. I think it is the latter, and if so I think this nomenclature should be fixed to the “backward” step of the diffusion process to bring it in line with standard diffusion terminology.
• Paragraph 2 continuously refers to some caption include “with noise”, but this does not exist anywhere
• What is MEDIC? It is sometimes used as (what I can infer) to be the main method, but this is never stated and it is unclear what this refers to specifically.

Evaluation Suite

---

• A key hyperparameter in all these editing-through-inversion methods is the $T_{start}$ parameter, which should in theory determine overall edit strength. It is unclear how this hyperparameter is chosen for the proposed method (and admittedly it may be ignored with using all 200 steps), but it is unclear how it was chosen for the baseline comparisons (such as DDPM-friendly).
• As DIC is a text-based editing method, it is unclear how the MusicDelta section of MedleyDB was used for the task. If the authors used the prompts generated from Manor and Michaeli, this should be explicitly stated. In general, it is unclear what sorts of edits this dataset even contains other than that the samples are longer.
• LPAPS results seem odd when comparing to Manor and Michaeli, as the values for table 2 are in theory on the same dataset, yet are all considerably lower (by about 10x) than the values reported in Manor and Michaeli. Similar inconsistencies hold for FAD, and in general it is unclear why the results for DDPM-friendly are different from the original paper (as supposedly this is the same data).
• Standard error bars should be reported for the subjective listening results, as all the values (with the exception of SDEdit) are quite close, and it is thus unclear whether the differences are statistically significant. In particular, it is also not stated how many subjects they used for the listening test and how many samples of each editing task were tested, as statistical significance should be calculated on average scores for each stimulus aggregated across users to assess whether such results would extrapolate to additional results from the proposed method.
• FAD is a rather odd metric to be using in this context with paired samples, both because A) it is an unpaired, distributional metric that ignores the structural relevance between pairs of samples and B) FAD has documented clear instability in small sample sizes for estimating the covariance matrix of each Gaussian (with generally needing 2.5k-5k+ samples to estimate this reliably [1]), thus making results somewhat unstable given ZoME-Bench’s size of only 1100 samples. Other metrics such as CLAP Maximum Mean Discrepancy (MMD) could be better suited, but in general it would make more sense to compare the FAD of generated samples to some existing reference set (such as Song Describer), as it is really only a measure of audio realism than anything on similarity to reference signals (which the text should reflect).
• The argument in the “Qualitative Results” section is reasonably heuristic. From visually inspecting the spectrograms, it is not clear of any structural relevance to the original source audio, and simply saying “the change of ___ can be seen in the Mel-spectrum” is insufficient to point to anything meaningful (though admittedly, the utility of spectrograms as a visualization here is not great). However, I think the overall success of the proposed method is somewhat overstated, as most of the examples provided in the audio demo samples do not actually perform the target edit and preserve the non edited content at the same time, with most samples sacrificing one of these two facets (which the authors identify as the most important parts of the editing process).

Proposed Benchmark

---

My biggest issue with the paper is the proposed benchmark dataset of ZoME-Bench, as it seems to contain a number of logical flaws that severely limit its utility as a public benchmark.

• It is odd that as the first “editing benchmark”, there is no actual ground truth information about the correctness of the edit itself. If it is only being assessed by CLAP score, this implicitly assumes that 1-2 word changes in the CLAP text prompt return meaningful changes in the input embedding in order to assess this change, which is assumed without support here. One could imagine here that in a truly suboptimal sense, a model could simply prioritize an edit that does absolutely nothing to the source audio but is able to change the CLAP embedding of the output, which would theoretically achieve perfect results on the benchmark. As the benchmark is a core contribution of the present work, the authors should either have ground truth target audio samples for each source audio (which would be easily doable for some of the instrument tasks if one had source-separated tracks), and/or at least followed the growing standard practice [2] in editing evaluation and use pretrained discriminative models to help assess the edit fidelity of more fine-grained tasks, which is fully doable in this case (such as using instrument tagging models for edits 0/1/2/8 or genre classifiers for edits 3/4).
• Many of the tasks are similar to previous work (AUDIT / InstructME) in being source separation tasks (0/1/2/8), and thus a much more natural choice for this benchmark would include source separated tracks in order to actually assess these edits that have real ground truth answers. While it is still unclear where the text captions came from for the MedleyDB subset (i.e. if they came from the Manor and Michaeli), it is odd that the MedleyDB subset was not used for creating the benchmark, as it seems readily available and has separated tracks for the instrument-based tasks, thus giving a possible avenue for ground truth targets.
• The paper in particular is missing a rigorous definition of what “rigid” and “non-rigid” mean in the context of text-based audio editing, and why they have deemed certain editing tasks in one category or another. For the rest of my point here, I assume rigid means “content-based” and non-rigid means “style-based,” as that is what the paper seems to imply and is inline with past image domain works. For example, it is unclear why “instrument change” is referred to as a non-rigid task, given that in theory the change of an instrument should preserve all harmonic and melodic content and only reflect timbral changes (as it would be if a guitar player was changed to a violin but played the same part). Unlike in image domain tasks (where edits can mostly be grouped into those than edit a particular masked / bounding box region of the source image vs. ones that make global edits), this notion of region-concept cooccurrence does not exist in audio and thus porting over the definitions of “rigid” and “non-rigid” is not applicable out of the box.
• In general, I think a number of the tasks proposed do not make for an informative benchmark. Tasks 3/4/5/6/7 (genre/mood/rhythm/background/melody) are ill-defined, as these conceptual buckets do not disentangle content vs. stylistic changes in the audio, and seem to be rather divorced from how actual musicians talk about semantic changes in music. As examples:
  • For genre, if “blues” changes to “rocks” then what changes? Is this a reflection of content changing (such as chords being simplified and melodic lines using fewer blues scales) or of stylistic changes (micro-timing, ornamentation, guitar fx)?
  • If “fast” is changed to “slow”, should the entire content be slowed down (thus reflecting a content-based change that can only be seen as “stylistic” if realignment is assessed between the original and now slower edit) or is this just a measure of decreased density of perceptual onsets?
  • If a “relaxing” melody is changed to a “cheerful” one, does this reflect changes in the pitch values, the rhythmic interpretation, both, or neither?
• While this is somewhat up to subjectivity, none of these semantic tasks seem non-rigid to me (as they all involve some amount of content preservation with stylistic changes). If these are meant to allow content-based changes, this should be explicitly stated, and in general, I’m hesitant to even phrase such hypothetical tasks as “edits” in the first place (as something has to stay the same for it to be considered an edit).

Between the issues with the non-rigid tasks, lack of ground truth for the rigid tasks, and over-reliance on CLAP similarity as a measure of edit accuracy, the overall use of this benchmark for standardizing and improving music editing is quite limited. To improve the paper, I think that either completely focusing on the MedleyDB subset and mostly dropping the proposed benchmark from the paper (as the methodological contributions stand on their own) or performing the significant work to improve the benchmark (and/or justifying why CLAP similarity can be used as a ground truth signal so heavily) would both be valid options, as the present version of the benchmark is my main concern.

[1] Jayasumana, Sadeep et al. “Rethinking FID: Towards a Better Evaluation Metric for Image Generation.” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023): 9307-9315.

[2] Basu, Samyadeep et al. “EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methods.” ArXiv abs/2310.02426 (2023): n. pag.

Overall:

The following points represent the overall weaknesses in the current manuscript. Please refer to the detailed explanations in the latter part of the Weaknesses and Questions sections.

1. Insufficient or unclear validation of the effectiveness of the proposed method, which is directly related to the originality of this work. (For more details, see A. in Weaknesses and 1. in Questions.)
2. Unclear motivation for incorporating the inversion process (L3 in Algorithm 2) within the problem setup (where a source prompt $\mathcal{P}$ is provided). (See more details at 2. in Questions.)
3. The contribution of ZoME-Bench to the music-editing field seems somewhat limited. (For further details, see B in Weaknesses.)

Details:

A. The validity of the objective metrics used for evaluation remains unclear. For more details, please refer to Question 3.

• Given the ambiguity of these objective metrics, the experimental justification for the advantages of using the Harmonic Branch and introducing the Disentangled Inversion Technique seems insufficient.
• On the other hand, the benefits of combining Prompt-to-prompt and MasaCtrl appear to be adequately validated in subjective evaluation. However, this aspect alone may not be sufficient to fully support the originality and strengths of this work.

B. While I agree with the importance of introducing standardized benchmarks in audio/music editing and appreciate the effort to create a dataset, ZoME-Bench still has some limitations. ZoME-Bench includes original audio samples, original prompts, editing prompts/types/instructions, etc., but it lacks edited audio samples that define how the source samples are supposed to be edited in a certain sense. In this respect, although ZoME-Bench contributes to standardizing editing instructions, it leaves unresolved the larger issue of verifying the edited results, which remains a significant challenge in audio/music editing evaluation. Therefore, while ZoME-Bench contributes to the audio/music-editing benchmark to some extent, its impact is limited. (I understand how difficult it is to construct such edited audio samples. I mention this point to assess and clarify the degree of the contribution and originality of the ZoME-Bench proposal.)

C. To improve the paper's presentation quality, I recommend revising the current manuscript. (See Comments in Questions.)

1. While the experiments are about music editing, the evaluation only uses metrics for general audio editing. Music content-related metrics like chroma distance [1] are missing.
2. The paper does not seem to be clear enough. See questions.
3. The values and effects of the hyperparameters in the paper are unclear, like $k, \tau_c, L$ and $S$. Ablation study or case study by changing these hyperparameters would be helpful to understand the model.
4. While the methodology seems to be general-purposed, the experiments only focus on music editing. This is okay but limits the impact of the paper a bit.
5. Typos and formatting errors. Algorithm 1: Inconsistency use of fonts; formatting error of \hat in $\hat{M}^{\textrm{tgt}}$; $\epsilon_{c_{\textrm{tgt}}}$ should be $\epsilon_{\textrm{tgt}}$. Figure 2: inconsistenct notation $M_{\textrm{tgt}}$ vs. notation in text $M^{\textrm{tgt}}$. Equation 9: $l$ is not defined. Table 6: not referenced in the appendix text.

[1] Zhang, Y., Ikemiya, Y., Xia, G., Murata, N., Martínez-Ramírez, M. A., Liao, W. H., ... & Dixon, S. (2024). Musicmagus: Zero-shot text-to-music editing via diffusion models. arXiv preprint arXiv:2402.06178.

Although this paper is a strong empirically-driven study, there are certain hypothesis-related issues that could be improved.

• First, the paper needs to clarify what is meant by “rigid” and “non-rigid” tasks. These terms appear throughout the paper, but after re-reading the entire text, I still found no clear explanation of what these tasks entail, which left me quite confused.

• The paper actually addresses a text-guided music audio editing task. However, the language and context in the main body do not consistently maintain this focus. Given the current context, I suggest aligning terms in the main text to match the title, shifting from “audio editing” to “music editing.”

• While the proposed multiple control method indeed focuses on different aspects through each control mechanism, whether this approach achieves “disentangled” control is debatable. To demonstrate that the controls are disentangled, the paper should include experiments showing that one control does not interfere with another. While these controls focus on different levels conceptually, they do not intuitively seem orthogonal, making the term “disentangled” potentially misleading. I suggest either adding experiments to confirm this or revising the terminology.

• The paper includes a subjective evaluation, which is commendable. However, the description of this evaluation is incomplete. Typically, subjective evaluations should also describe the gender, age, music background, and musical training distribution of the subjects, which helps with the interpretation of the results. Unlike data annotation, where these factors might be less crucial, they are important here due to potential biases introduced by AMT, and these underlying biases should be considered.

• In addition, hypothesis tests should be conducted for all reported results.