Predicting Spatial Transcriptomics from Histology Images via Biologically Informed Flow Matching

Weakness: The novelty in the method is limited / incremental.

Respond: We respectfully disagree with the statements and would like to emphasize the contributions of our paper. As for the motivation, previous slide-level studies have primarily focused on designing better encoders to capture the global context, such as Hist2ST and TRIPLEX. However, they overlooked two critical aspects: (1) leveraging the representational power of pathology foundation models to improve prediction accuracy, and (2) explicitly incorporating gene-gene dependencies to infer gene expression. Inspired by these gaps, STFlow builds on top of pathology foundation models and introduces a generative approach to learning the landscape of gene regulation. This enables more accurate predictions. I believe our work will inspire the development of generative models in this domain.

As for the technical contributions, we do not consider the introduced flow matching to be trivial:

• We use the ZINB distribution to model the overdispersion and sparsity of gene expression effectively.
• Our proposed spatial attention mechanism is specifically designed for flow matching. The E(2)-invariant attention integrates gene expression to regulate the attention map, akin to the way cells regulate each other.

Weakness: The motivation for ZINB priors is not clear given the ablation study showing that using zero priors gives almost the same results. In the ablation studies, using "zero distribution, where all samples are zero". I'm not sure what it means to apply Log1p to the samples. That would mean Log 0 which is zero. This needs to be clarified. Also, the results with the zero distribution see very close to ZINB. So why go through the trouble of esimtating the parameters for ZINB? and they are better than Gaussian which sounds counterintuitive but is not explained.

Respond: We apologize for any confusion. Zero distribution is indeed all zero distribution so Log1p will produce the same results. This statement is removed from our updated manuscript.

As for the motivation for using ZINB, gene expression in ST data exhibits a similar pattern to scRNA-seq data:

1. Non-activated genes dominate: Most of the data consists of non-activated genes with expression values equal to 0.
2. Over-dispersion: Gene expression variance exceeds the mean value.

Below are the statistics for HEST-1k. Here, Exp=0(%) represents the average proportion of zero-expression values in the gene expression data across samples, while mean/variance indicates the average mean and variance of gene expression across samples.

Such an observation motivates us to use the ZINB distribution, where the negative binomial component explicitly models dispersion with an additional parameter to adjust the variance, and the zero-inflated condition captures the sparsity in gene expression.

In terms of performance, we would like to clarify that comparing the average Pearson correlation across datasets might somewhat underestimate the contribution of flow matching since there are significant scale differences between datasets (e.g., the model achieves 0.707 on SKCM but only 0.128 on HCC). Here, we present the relative improvement of the model with flow matching on each dataset, where flow matching can provide around 5% improvement on UNI, which is non-trivial.

We will incorporate these analyses in our manuscript.

Weakness: Author’s approach assembles many prior off-the-shelf methods for ST prediction, including a two-stage approach for histology, tile-level foundation models, and flow matching. Notably, the approach uses a frozen patch encoder, that does most of the heavy lifting in the representation learning, leaving it frozen inhibits the model’s ability to learn.

Respond: As mentioned in Line 159, our goal is to propose a pathology foundation model-agnostic spatial transcriptomics prediction framework, rather than to develop a better spot-level image encoder. While fine-tuning pathology image encoders with gene expression data could potentially improve performance, it is impractical, particularly for billion-level foundation models and standard gigapixel slides containing tens of thousands of image tiles. Moreover, the benchmarking is fair as all models utilize frozen image encoders.

Weakness: The model performance boost is not substantial, and are often within error bar of the much simpler baselines. The comparison of the proposed method doesn’t have the proper slide-based baselines using the same patch encoder. For example, comparison to Hist2ST and HistToGene in table 1 does not make sense because the patch encoder is different.

Respond: Regarding the error bar, we clarify that the variance in Table 1 represents the standard deviation of Pearson correlations across folds within a dataset, not across random seeds. Performance variance between folds may arise from batch effects or sample size differences. For example, a sample with few spots in the training set might lead to under-training and lower performance, while the same sample in the test set could yield better results. This pattern is also evident in UNI's performance on the READ dataset. For a clear demonstration, we here present the variance across random seeds:

Specifically, the standard deviation across 5 random seeds is the variance in mean performance under different initializations. The results demonstrate that our model is stable across various seed initializations.

As for the Hist2ST and HistToGene, these two models use a variant of ViT as their image encoder, making it unsuitable to replace their encoder with other pathology foundation models. However, we agree that the same pathology foundation models should be applied to other baselines for a comprehensive comparison. The results of BLEEP and TRIPLEX using Ciga and UNI are shown below:

Notably, STNet simply trains a linear head on top of the image encoder, so replacing its image encoder is equivalent to the case of UNI and Ciga with a linear head. Therefore, we only present the performance of BLEEP and TRIPLEX here. All these results will be included in our updated manuscript.

I think the authors for the response, and have addressed some of my concerns regarding benchmarking. However, I remain concerned with:

• the author's approach is not quite agnostic to foundation models (FMs), as it requires a new ST prediction module for each FM. Changes in FM leads to a much more significant performance boost compared to changes in the ST prediction module, and that suggests that the FM may be doing the heavy lifting. I encourage the authors to explore methods to fine-tune the patch/spot level encoders as well.
• Performance boost over baseline TRIPLEX is marginal.

Furthermore, upon reading other reviewer's comments, I concur with their concerns on the lack of novelty, specially with the use of the ZINB distribution, where the ablation studies shows that the ZINB's contribution is limited.

Therefore, I maintain my score of marginally below the acceptance threshold.

Weakness: Motivation of using ZINB prior is not strong: Is there a specific reason of using this distribution? Cited literatures are single cell RNA seq which is not ST. In the Table 2, it's also clear that ZINB is not helping especially for UNI and gigapath.

Respond: Gene expression in ST data exhibits a similar pattern to scRNA-seq data:

1. Non-activated genes dominate: Most of the data consists of non-activated genes with expression values equal to 0.
2. Over-dispersion: Gene expression variance exceeds the mean value.

Below are the statistics for HEST-1k. Here, Exp=0(%) represents the average proportion of zero-expression values in the gene expression data across samples, while mean/variance indicates the average mean and variance of gene expression across samples.

Such an observation motivates us to use the ZINB distribution, where the negative binomial component explicitly models dispersion with an additional parameter to adjust the variance, and the zero-inflated condition captures the sparsity in gene expression.

In terms of performance, we would like to clarify that comparing the average Pearson correlation across datasets might somewhat underestimate the contribution of flow matching since there are significant scale differences between datasets (e.g., the model achieves 0.707 on SKCM but only 0.128 on HCC). Here, we present the relative improvement of the model with flow matching on each dataset, where flow matching can provide around 5% improvement on UNI, which is non-trivial.

We will incorporate these analyses in our manuscript.

Weakness: Notation is not easy to follow: It's claiming that algorithm 2 has a sampling factor which is gradually decrease. However, the last step is basically, Yt+1←Yt+(Y^−Yt)/(T−t), I don't think it's decreasing?

Respond: We apologize for any confusion. The coefficient indeed increases as the time step progresses, ensuring the prediction gradually converges to the final result. The prediction at the final step is directly returned (refer to Line 6 in Algorithm 2). The description has been updated in the revised manuscript, highlighted in red.

Thank you for recognizing our efforts! Please feel free to let us know if you have any further questions.