All You Need Is A Reference: Cross-modality Referring Segmentation for Abdominal MRI

1. While the method appears novel, it seems the authors are tackling a relatively simple problem in a convoluted manner. The issue could be effectively addressed by training a segmentation model for T1w images and using a rigid registration method to transfer labels from T1w images to other modalities.
2. The workings of paired data augmentation are unclear. Translating a T1w image into an image resembling T2w, DWI, In-phase, and Opposed-phase MRI is not straightforward, especially at high resolutions. Are there any existing methods that can perform this task accurately and reliably? How does they work?
3. There is a lack of an overview of the proposed approach, making it unclear what role each subsection in Section 3 plays.

1. Despite the authors mentioning that they do not focus on multi-stage cross-modality approaches, the introduction and analysis are overly simplified. The experimental section lacks performance and computational complexity comparisons with cross-modality methods. Over-emphasizing the superiority of the method under its own setup makes the results less convincing. It is recommended to include relevant multi-stage cross-modality methods and conduct comparative analysis of computational speed and parameter counts between methods.
2. In the experimental setup, the authors only consider the scenario where T1w is used as the reference modality. This raises questions about whether the method is effective in other scenarios. More reference modality settings need to be considered to validate the generalizability of the method.
3. While contributing a new dataset is commendable, conducting experiments solely on a private dataset makes it difficult to obtain a fair and convincing evaluation of effectiveness. It is suggested to perform comparisons using other public datasets such as BraTS, as well as on lesion segmentation tasks.

Minors: The introduction of Table 4 seems to be mismatched with its content.

1. A major concern with the task definition is the unclear motivation for referring segmentation when target images are well co-registered with the reference images. In a co-registered setting, where both the reference and target modalities share the same ground truth, the output masks should be identical for both modalities. If co-registration is strong, a straightforward solution would be to perform segmentation directly on the reference image (e.g., T1w) and apply the same output mask to the target images. In this case, as shown in the results (Table 1), nnUNet would be the preferred model based on its performance on the T1w reference modality. This raises the question: Why is referencing segmentation necessary in such a scenario, and how does it add value over simpler approaches like direct segmentation on the reference modality?

2. The paper lacks significant methodological novelty. The approach essentially adapts existing natural image segmentation techniques (i.e., referring segmentation) to the medical imaging domain without introducing substantial innovations. The proposed method relies heavily on a segmentation framework designed for natural images, such as SEEM, with only minor adjustments for multi-modal MRI data. There is no substantial architectural change or novel mechanism introduced for the medical imaging context.

3. The model is evaluated on a single dataset that contains only four labeled organs. This is a significant limitation, as it restricts the generalizability of the results to other medical imaging datasets or tasks. To demonstrate the broader applicability of CrossMR, the authors should test the method on other datasets or scenarios with more organs or different imaging modalities (e.g., CT or x-ray).

1. Motivation of the method: motivation and details about the proposed method should be illustrated more clearly.
2. Weakness of ablations: the results are not well illustrated, and the effectiveness of augmentation is doubtful.
3. 2D model: the method is purely based on 2D slices. Experts still need to provide a scribble for each organ in every slice.

1.While the paper mentions automatic scribble generation, it doesn't provide sufficient information on how well this mimics real clinical annotations or its impact on model performance.

2.While the writing is generally good, some technical aspects would benefit from further details. For instance, section 3.4 (Automatic Scribble Generation) could provide more details on how different scribble patterns impact segmentation accuracy.

3.More study of the performance heterogeneity between modalities (such as opposed-phase vs. in-phase) will be better. Investigating if modality-specific factors like as contrast variations affect segmentation accuracy could provide more insight into the model's robustness and limitations.