CELL-Diff: Unified Diffusion Modeling for Protein Sequences and Microscopy Images

1. The paper describes that CELL-Diff can generate protein sequences based on microscopy images. However, it is not clear how the model ensures that these generated sequences are biologically valid and functional. Given that small changes in amino acid sequences can have significant effects on protein function, how does the model account for this? Are there mechanisms in place to validate the functionality or structural stability of the generated sequences?
2. Proteins often undergo post-translational modifications (PTMs) that affect their localization and function. Does CELL-Diff consider PTMs in the generation process? Additionally, how does the model perform with proteins that have complex quaternary structures or those that form part of larger protein complexes?
3. The paper mentions that "the variability in equipment and experimental conditions limits the availability of robust image encoders". How does CELL-Diff address this variability in the training data? Does the model generalize well to images obtained under different imaging settings, and is there any preprocessing required to standardize the inputs?
4. The conditional generation relies on cell morphology images such as nucleus, ER, and microtubule markers. How sensitive is CELL-Diff to the choice or quality of these images? For instance, if only partial or noisy cell morphology images are available, how does that affect the quality and accuracy of the generated protein images?
5. Has the potential incorporation of other types of biological data (e.g., gene expression profiles, protein-protein interaction networks, or 3D structural data) been considered to enhance the model's capability? How might the addition of such multimodal data affect the training complexity and the interpretability of the results? I would be pleased if the author could consider raising my score after addressing my concerns.

1. Given the fact that the compared algorithm CELL-E 2 produces binary image outputs, how fair is using MSF to evaluate the models? Could the authors please discuss the potential limitations or biases of using MSF for comparison when one model produces binary outputs and the other doesn't?

2. The balancing coefficient $\lambda$ is introduced in equation 12 and is set to 100 in the experiments. How does the value of $\lambda$ affect the results? Could the authors please provide an ablation study showing how different values of $\lambda$ impact the model's performance across various metrics?

3. What do the authors mean by this in line 418: “To calculate IoU, we apply median value thresholding to the original protein images to generate binary masks, while for CELL-E2, we use the predicted thresholding image"? Do you mean that median value thresholding was applied to both the original and generated protein images to obtain ground truth and predicted masks for IoU? Furthermore, CELL-E 2 generates masked (binary) images. When assessing the FID-O, is it an unfair comparison for CELL-E 2? If so, this should be stated clearly. Please provide a step-by-step explanation of how IoU is calculated for both models, and discuss any potential biases in the FID-O comparison due to the differences in output types between the models.

4. From Table 1, CELL-E 2 outperforms CELL-Diff in terms of IoU when using the Nucleus image alone as input. However, it is not tested how CELL-E 2 performs when using the additional channels. Could this method be tested with these additional channels?

What images did you use? Please describe the exact conditions, modalities, equipment, and setting. Make sure to note if the images are quantitatively comparable. If not what steps did you take to make them quantitatively comparable? What was the inductive bias? Showcase exact proteins and morphology or actual function as controls. What are the conditions when it works, and what are the conditions when it fails? What was the molecular source of fluorescence? Were these chimeric proteins? Did sequences contain the fluorescent protein parts? If so this constitutes a data leakage. How did you mitigate this?

L180: Delete the word "where".

L210: Typo: "involves model a continues" --> involves modelling a continuous.

L215: Typo: L^{port} --> L^{prot}.

Fig. 2 and upper panel of Fig. 3 and their captions are clear. It would help the reader if the different parts of figures 2 and 3 are named (a, b, c, ...), and then referenced in the text where appropriate as Fig2a, Fig2b, Fig3a, etc.

Fig3: Clarify and identify the lower panel of Fig. 3 (below the dashed line) and describe it in the caption, and describe how it is related to the top panel, and related to the overall method?

Eq. (13): small i is not (explicitly) defined.

Eq. (13): it unclear how to apply this equation. How is the f=I/(image size x pixel size) related to the "where {FRPSD...}" part?

L418: You write that the thresholding is applied of the original protein images. Clarify that/whether the thresholding was also applied to the generated (predicted) protein image.

The authors do not explain why they extract the masks (via thresholding) and apply IoU, noe they do explain why they measure FID limn the masks (i.e. FID-T).

What does the column "Cell image" mean in Table 1? How were these images used; and why aren't the same entries listed next to the competing method CELL-E2 (e.g. "Nucl,ER" vs only "Nucl")?

The bolded 'winning' values in table 1 should be restricted to the top two rows. The rows 3,4,5 (Cell-Diff with ER or/and MR) should be isolated from the other results.

Table 1:

• The way this table is presented raises questions about the fairness of comparison, i.e. could the improved results in Table 1 - HPA (the bold values) be attributed to using more/richer types of training data (ER and MT) and not to the methodology itself? The authors do state, on lines L430-431, that the competing methods CELL-E2 and the proposed method CELL-DIFF achieve comparable results, and that [only] when other images are used (ER and microtubules) do the proposed method performance surpasses the competition.

• The bolded 'winning' values in Table 1 - HPA should be restricted to the first two rows (CELL-E2 Nucl vs CELL-Diff Nucl), and the comparison (bold text) should be based on these two rows.

• The rows 3,4,5 (Cell-Diff with ER or/and MR) should be isolated from the other results and the values for those rows should not be used in the comparison. Examining the first 2 rows of results in Table 1, we see that CELL-E2 has better IoU than CELL-Diff (0.461 vs 0.448).

• How do explain that using ER and MT images give worse FID-T then when using only ER (34.1 vs 32.4; in Table 1, HPA, FID-T)?

Figure 4 show that CELL-Diff produced sharper images and qualitatively images that are more similar to the real images than CELL-E2. However, it is noticeable that there are structures / pixels that are clearly not faithfully reconstructed. Perhaps this localized differences are critical for a particular application. Similar observations can be made on the results in the Appendix (Fig. 7 and Fig. 8). So this raises the question of how would one trust the predicted images and utilize them for subsequent tasks? The authors do mention different potential downstream applications.

Whether the CELL-Diff images do in fact perform better when used in some subsequent tasks when compared to CELL-E2 remains to be seen. For example, how will the results look like if a figure like Figure 5 is produced but for CELL-E2?

The competing method CELL-E2 is not compared against under the "virtual screening" application.

The "visual staining" application is assessed qualitatively (again without comparison to CELL-E2). For this application, the authors write: "we solve this problem..."; I don't believe the authors provided anywhere close to sufficient evidence to justify the use of these words. The authors also state "clearly shows the association of LIF18A and EML4... consistent with known biological function" - a reference is needed here. Further, the overlaid image in Figure 6 does not, at least to the reviewer, clearly show the association. Other better ways to clearly visualize, and even quantify, the association is warranted.

For the "localization signal generation", the generated sequences are reported in a table in the appendix, but there is no assessment of their correctness.