Causal Frameworks and Feature Discrepancy Loss: Addressing Data Scarcity and Enhancing Medical Image Segmentation

1. I know it is not good to just say lack of novelty, but the segmentation area is developing so fast nowadays. I think the loss design is practical but not eye-attracting at all. The problem setting like data scarcity is something every medical image segmentation paper will mention, and claim their methods will help. The author has made this "data scarcity" in their paper title. I think more experiments showing that this proposed loss is performing well in this "data scarcity" situation should be given at least. Just showing the performance in each dataset is not enough to persuade me. Since the loss can handle data scarcity will contribute to better performance, but better performance does not mean the methods can handle data scarcity. At least more ablation study is needed( ex. Varying Training Data Size, Low-Shot Learning Setup, Comparison with Data Augmentation....)
2. Meta's segment anything is something that changed the game in image segmentation. Why not also give us the score segment anything will perform in this competition.

• I find the idea conceptually similar to entropy minimization, which has been a long-standing concept in training deep neural networks for image segmentation [1]. I also find the idea similar to pixel-level contrastive losses for semantic segmentation [2]. Both entropy minimization and pixel-level contrastive losses aim to push apart pixel-level representations of different classes, as do other previous works in semantic segmentation (e.g., [3]). The proposed work takes an alternative angle at this idea—by encouraging foreground/background classes to be far apart at many layers of the network—but a discussion of how the proposed idea relates to similar historical ideas in semantic segmentation would be appropriate.
• Evaluation focuses primarily on comparing networks trained with the loss function vs. without the loss function. While this is convincing for showing that the proposed loss function improves performance, it does not give a good understanding of how the proposed loss function compares to other loss functions or approaches developed to deal with limited labeled data. As a result, I do not have a sense for how impactful the proposed loss function is.
• Minor formatting note: the authors appear to use \citet instead of \citep, resulting in in-text citations where they are meant to be in parentheses.
• Minor note on framing: the Introduction focuses on breast cancer, however the paper and contributions are not particularly focused on breast cancer. You evaluate the loss function over multiple datasets (breast cancer and otherwise), and the primary contribution of the paper is a loss function that can be applied to any semantic segmentation network. Further, you show that the loss function can be extended to solve other problems in semantic segmentation, such as handling distribution shifts. Perhaps an alternative framing that focuses on the loss function and its applications would be more appropriate.
• Minor note: In the title you refer to your loss as “feature discrepancy loss,” while in the main text you refer to your loss as “feature distance loss.” Staying consistent would help the reader.
• Minor note: Figure 6 seems to be missing a legend.

In summary, I find the loss function useful and believe it improves performance. However, it is conceptually similar to past loss functions that have been developed and it is not evaluated well enough for me to be convinced of its impact. Given the incremental nature of the contribution and the minimal evaluation, I recommend to reject.

[1] Grandvalet, Yves, and Yoshua Bengio. "Semi-supervised learning by entropy minimization." Advances in neural information processing systems 17 (2004).

[2] Chaitanya, Krishna, et al. "Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation." Medical image analysis 87 (2023): 102792.

[3] He, Xingjian, et al. "Consistent-separable feature representation for semantic segmentation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 2. 2021.

1. The theoretical basis on causal inference appears disconnected from the method itself. The only connection is the acknowledgment that segmentation results rely on latent representations, highlighting the need for their improvement—an idea that is widely recognized. Therefore, reiterating this from a causal perspective seems unnecessary. Additionally, the propositions mentioned in the paper, such as functional relationships and conditional independence, do not seem relevant to the method.
2. The design of the method appears trivial, as it merely adds a loss term at intermediate levels to increase the distance between foreground and background representations. Many previous works have explored similar concepts, albeit in different forms, including deep supervision [1], local contrastive learning [2], and learning-to-rank [3].
3. There is some ambiguity in the methodology (see questions).

[1] Dou, Qi, et al. "3D deeply supervised network for automatic liver segmentation from CT volumes." Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19. Springer International Publishing, 2016.

[2] Chaitanya, Krishna, et al. "Contrastive learning of global and local features for medical image segmentation with limited annotations." Advances in neural information processing systems 33 (2020): 12546-12558.

[3] Gong, Shizhan, et al. "Segmentation of Tiny Intracranial Hemorrhage Via Learning-to-Rank Local Feature Enhancement." 2024 IEEE International Symposium on Biomedical Imaging (ISBI). IEEE, 2024.

The central claims of the paper are adequately supported with evidence. It is said In Sec. 2.2 that the paper addresses the issue through causal mediation. However, the connection between the proposed feature discrepancy loss and causal medication is not explained clearly. The empirical study and theoretical analysis of the feature discrepancy loss only lead to the conclusion that imposing label supervision on intermediate features could help improve the Dice score, which has already been validated in the previous work (Isensee et al. Nature Methods, 2021)