Explainable medical image clustering

The paper is not easy to follow. There are many parts missing which makes the paper hard to understand fully. My detailed comments are as follows.

1> The paper proposes a new cell dataset, however, it does not discuss the drawbacks of previous datasets. Authors should provide a comparative study and explain why a new dataset is necessary. It is not clear if annotations are available for the dataset. If yes, how annotations are prepared should be mentioned. In Fig. 3 cancer cell and healthy cell annotations are given, however, they overlap. For example, in ‘Day0’ image, the same cell is marked as both healthy and cancer cell. 2> Fig.2 is not clearly explained. Forward pass and backpropagation is not clear. Also, loss function is not defined. 3> ‘Preprocess’ mentioned in Fig. 2 is not explained in the paper. 4> Network architecture and training are not clearly mentioned. 5> How are human-designed features computed and why can't neural networks capture these features? 6> Are explanations generated using Grad-Cam and text modules verified by humans?

weaknesses can be grouped into four categories as follows, in order of importance:

1. novelty

Each module in the proposed framework is largely leveraging existing work, with basic adaptation to the studied image data. Feature extraction seems to be u-net with multi-head attention and some hand-designed features. Clustering is T-SNE followed by affinity propagation. Visual explanation is GradCam using assigned cluster labels as target concepts. Text explanation is looking at the output of auxiliary image classification models.

2. clarity of details

Important details in the paper are not clearly conveyed and/or missing. As examples: The description of the feature extractor is unclear in multiple respects - what are the specifics of the architecture, what are the specifics of the pseudo-labels, what are the specifics of the hand-designed features and how are these features integrated, what is the loss function, what is the training data. What is the dimensionality of the feature extractor output? How are the cancer and healthy cell annotations obtained, are they dense (complete) labelings, are they noisy? In the result tables, the headings are not explained - SE, SP, PC, JS, DC.

3. experimental validation

Very limited quantitative evaluation and comparisons against existing baselines are given, particularly for the main goal of the paper which is the clustering of the images. As far as qualitative evaluation, there are not concrete examples of interesting findings or observations that can be derived from the final clustering.

4. clarity of writing

I would encourage the authors to focus on clear, simple writing, with the goal of communicating information in a clear and concise manner. As a practical specific example, many adjectives and adverbs can be dropped. For instance, "comprehensive" is used 11 times in the paper, "meticulously" is used 4 times in the paper; simply dropping these terms has no impact on the meaning but makes the writing more concise.

• The method is not clear. For instance:

  • It is not clear how the pseudo labels for live/dead cells are derived and how they are used. Later there is only mention of using HSV to separate foreground and background.
  • How is the feature extractor trained on and what is the loss function?

• The text explanation is based on manual features such as density where a model is trained to predict them. They don't have anything to do with the features used in the clustering. So they really don't explain the clustering.

• In Fig 3, the annotations for cancer and healthy cells overlap. This is especially obvious in the day 9 images.

• In Table 1, the authors compare unet encoder with other architectures including resnet. Without the skip connections, a UNet encoder is basically a large CNN.

• The authors present the dataset as temporal sequences based on variation in treatment. However, the results do not show the clustering correlate with any outcome or treatment.

• Minor: In page 2:

  • Typo in the sentence "However, owing to the absence of ground truth-remove"
  • Typo in the sentence "a text explanation module-remove sentence."

There is very little offered in terms of methodological novelty. The paper is a collection of existing (and widely used) ideas in literature glued together in a manner to address the proposed application. As such, this would be a nice contribution in a more focused venue that brings together practitioners in the domain addressed by the paper (and the introduction of the dataset would indeed be a great addition).

The writing and elucidation of ideas needs reworking. Several sections are not clearly written. One issue that crops up is whether there is sufficient clarity with regards to the biological concepts described in relation to the dataset proposed, assuming the typical ICLR audience. The machine learning methods used can be described more clearly. Clarity of the images used needs improvement as well - this is important since a new dataset rich in imaging features and artifacts is being introduced as a key contribution.

The ablation studies need to be more detailed and described better. Currently, the section is written in a manner that is not particularly informative or helpful in discerning the key messages of the paper. Please consider using Supplementary materials to address content that doesn't fit within page limits.

Typo: 4.2.1 'IMPLEMENT' DETAILS --> IMPLEMENTATION DETAILS