Q1: For ab initio: (1) why not use noisy particles? (2) why not use cryoFM as refinement as well?

(1) Our decision not to utilize real particles for ab initio reconstruction is based on several technical challenges that would significantly affect the accuracy and feasibility of the model:

1. A large number of particles with low SNR complicate the sampling process: Generally speaking, the time complexity of cryoFM for ab initio modeling is "sampling-step x projection-num". Dealing with a few 2D averages within the standard framework is considerably easier. However, when applied to tens of thousands of real particles, it necessitates numerous modifications to the algorithm to keep it robust. Moreover, the low SNR makes the pose estimation more arduous to implement, which may undermine the accuracy of likelihood computation. Conducting experiments on 2D averages with high SNR would be easier to validate the effectiveness of our model. In future work, we will refine our code for pose determination to be more robust.
2. Inconsistent value range between the flow model and real particles: When training cryoFM, we standardize the maps in the training set to ensure that their values range from 0 to 1. This is distinct from particles in the real world, the value of which may exhibit significant variance. Such a disparity makes it difficult to estimate the likelihood term since we cannot compute the MSE between two entities with different amplitudes. A workaround is to use correlation loss, which seems to alleviate the problem. However, we empirically find that due to the extremely low SNR of particles, the correlation loss may also fail. Given these challenges, we decided to use clean class averages for ab initio reconstruction and plan to focus on real particle reconstruction in future work. We also believe that incorporating CryoFM as a prior could potentially enhance the refinement process in addition to ab initio initialization. However, for the reasons outlined above, we will reserve this exploration for future work.

(2) We agree with the reviewer and believe that CryoFM can be used for refinement as well. However, the details of the best way to incorporate CryoFM into refinement are not obvious and not in the scope of this paper, which we will leave for future work. For example, it seems feasible to embed CryoFM empowered denoising into each iteration in the refinement process, but this also introduces additional challenges, such as potential inaccuracies in estimating the forward model. We conducted additional experiments to evaluate the robustness of the estimated forward model, by introducing perturbations to the noise power of the estimated operator in the spectral denoising task. The results show that the model performs robustly with perturbations, suggesting that CryoFM has the potential to be applied to real data in refinement as well.

Spectral noise denoising (reported numbers are $FSC_{AUC}$)

We thank the reviewer for the valuable questions. We have updated the manuscript and here is the detailed response:

Weakness 1: The motivation for using a flow-based model is not very clear

Thank you for your fruitful comments. The main motivation lies in the following two points: (i) generative modeling is a trend in constructing modern foundation models, and (ii) flow matching excels in generative modeling. Two representative foundation models (based on generative modeling) are AlphaFold3 and ESM3. Both of them incorporate an iterative refinement strategy to generate proteins (compared with their previous works, AlphaFold2 and ESM2). AlphaFold3 employs a diffusion module, and ESM3's refinement strategy resembles that of discrete diffusion. This indicates that the diffusion model has a great potential in building modern foundation models. Flow matching is a variant of diffusion models, and their training strategies are very similar. However, flow matching is easier to train and converges faster. We have added a discussion part in Appendix A.2 to discuss the motivation of using the flow matching framework for building foundation models.

W2: Related work discussion could be improved

Thank you for your helpful feedback. We add the missing references and related discussions in the sections of introduction and related work. Besides that, (i) We add some discussion on foundation models in Section A.2 (ii) We originally intended to cite the DiffModeler in Section A.4 (which is Section B.4 in the updated manuscript), but we messed up the citation key and ended up citing another paper by mistake.

W3: More explanation of the baselines

Thank you for your suggestions. We have added more explanatory descriptions of these baselines in Appendix B.1.

W4: Motivation of anisotropic denoising

The anisotropic noise task is related to the preferred orientation problem, which is very common in single particle cryo-EM. During pose estimation, poses are often not uniformly distributed. That is, more particles may be assigned to a specific pose. Under a certain orientation, a density will have a higher signal-to-noise ratio if more particles are correctly assigned to these poses. This will make the final density have anisotropic noise instead of an isotropic spectral noise, which is the motivation of this task. From the measurement standpoint, this is very similar to the spectral denoising task, but from the application perspective, this addresses a more complicated problem that often happens in the real world.

W5: Specialized versions of the algorithm in the main text

We thank the reviewer for the fruitful suggestions. We have updated the manuscript and added some descriptions regarding the modifications. This undoubtedly makes our manuscript clearer.

Weakness minor points 1: ... “degradation operator” ... Calling A a forward model/operator or measurement model/operator would be more consistent with the inverse problems literature.

Thank you for your constructive suggestion regarding the terminology of the forward operator A. We appreciate your insight and have revised the manuscript to consistently use the term "forward model" instead of "degradation operator" to enhance consistency.

Weakness minor points 2: There is a typo on line 231; Transformer is misspelled.

Thanks, we have corrected the word.

Weakness minor points 3: Figure 3: the colors for CryoFM and DeepEMhancer are too similar in the FSC curve. Another strategy would be to change the opacity or style of the lines so that both lines are visible even when they overlap.

We thank the reviewer for the suggestion and have updated the figures to improve the clarity.

We thank the reviewer for the valuable questions. We have updated the manuscript and here is the detailed response:

Weakness 1: ... lacks demonstrated general applicability and versatility ... such as 2D-to-3D reconstruction ...

Thank you for your insightful suggestions. We appreciate your recognition of the foundational aspect of cryoFM. Moving forward, we plan to investigate additional practical applications of cryoFM to demonstrate its broader utility further. Regarding the 2D-to-3D reconstruction, our current experiments focus primarily on using clean class averages as observations. We acknowledge that there are challenges when using experimental particle data directly. We conclude the challenges may come from the following reasons:

1. A large number of particles with low SNR complicate the sampling process: Generally speaking, the time complexity of cryoFM for ab initio modeling is "sampling-step x projection-num". Dealing with a few 2D averages within the standard framework is considerably easier. However, when applied to tens of thousands of real particles, it necessitates numerous modifications to the algorithm to keep it robust. Moreover, the low SNR makes the pose estimation more arduous to implement, which may undermine the accuracy of likelihood computation. Conducting experiments on 2D averages with high SNR would be easier to validate the effectiveness of our model. In future work, we will refine our code for pose determination to be more robust.

2. Inconsistent value range between the flow model and real particles: When training cryoFM, we standardize the maps in the training set to ensure that their values range from 0 to 1. This is distinct from particles in the real world, the value of which may exhibit significant variance. Such a disparity makes it difficult to estimate the likelihood term since we cannot compute the MSE between two entities with different amplitudes. A workaround is to use correlation loss, which seems to alleviate the problem. However, we empirically find that due to the extremely low SNR of particles, the correlation loss may also fail. However, we will reserve this exploration for future work.

W2: Testing on real datasets

The main reason that we did not use "experimental" noise for denoising is the lack of ground truth in this scenario, making result evaluation and comparison difficult. Although it is possible to apply cryoFM and other baselines to the density maps with a lower resolution, it is impossible to evaluate whether the authentic signals are enhanced and restored. Visual inspection can be deceptive and unreliable, as the seemingly improvement in the denoised density maps could simply be a hallucination. Dealing with experimental noise also introduces additional challenges, such as potential inaccuracies in estimating the forward model. On the other hand, spectral noise is a widely accepted noise model in cryo-EM reconstruction, and the anisotropic spectral noise is also reasonable when the pose distribution is uneven. Therefore, in order to provide a trustworthy metric evaluation compared to the ground truth, we adopted synthetic noise for denoising tasks.

To further test the performance of cryoFM and flow posterior sampling in the case that the forward model may not be inferred accurately, we have conducted additional experiments to evaluate the robustness of the estimated forward model. We introduced perturbations to the noise power of the estimated operator in the spectral denoising task, and the experimental results indicate that the model performs robustly with the perturbations, showing that the model does not strictly rely on the precise estimate of the operator.

Spectral noise denoising (reported numbers are $FSC_{AUC}$)

We thank the reviewer for the valuable questions. We have updated the manuscript and here is the detailed response:

Weakness 1: ... the theory of the flow-matching approach for training the prior very difficult to understand.

Thank you for your fruitful suggestions. We have updated the manuscript as follows: (i) Sections 3.1, 3.2, and 4.3 have been rewritten to make them more rigorous. (ii) A figure has been added in Appendix A.1 to illustrate the concepts of flow matching. (iii) A discussion part has been added in Appendix A.2 to discuss the motivation of using the flow matching framework for building foundation models.

Weakness 1: ... a definition or explanation of the variable (or function?) x^0 ...

Thank you for remarking that we missed some words for defining the function. $\hat{\mathbf{x}}_0(\mathbf{x}_t)$ represents the posterior mean of the flow (or the DDPM). To be more concise, it means "given a sample $\mathbf{x}_t$ at timestep $t$, what is the expected value at the final timestep?" It is not the output of the neural network $\mathbf{v} _\Theta$, but it can be computed analytically. For instance, $\hat{\mathbf{x}}_0(\mathbf{x}_t)=\mathbf{x}_t-t\cdot \mathbf{v} _\Theta(t, \mathbf{x}_t)$. We have added some words into the manuscript.

Weakness 2: Cannot judge the novelty of the variant of flow matching.

We thank the reviewer for the helpful feedback. In this manuscript, we did not alter the training framework of the original flow matching (Section 3.1). Our novelty lies in (i) exploring architectures and strategies (Section 4.1) for training flow matching models on 3D protein density data and (ii) following the DPS framework (Section 3.2) to derive a FPS framework (Section 4.3) and Algorithms 1/2/3 for solving specific inverse problems in cryo-EM.

Weakness 3 (Feedback 1~3): Details missing.

Feedback 1: More explanation of the baselines

We thank the reviewer for this helpful feedback. We have added more explanatory descriptions of these baselines in the Appendix B.1.

Feedback 2: Test set in denoising experiments

The test set is 32 density maps which were selected and excluded from training. We have added a new section (C. Additional data collection and processing) in the Appendix discussing the curation of the data. And we have also uploaded the EMDB IDs of the training and test data used in this paper for readers to reference if needed https://figshare.com/s/9ef2614108391c04d910

Feedback 3: Motivation for anisotropic denoising

Yes, this is related to the preferred orientation problem. During pose estimation, poses are often not uniformly distributed. That is, more particles may be assigned to a specific pose. Under a certain orientation, a density will have a higher signal-to-noise ratio if more particles are correctly assigned to these poses. This will make the final density have anisotropic noise, which is the motivation of Section 5.2.

We thank the reviewer for the valuable questions. We would like to clarify that for tasks including spectral noise denoising, anisotropic noise denoising, and missing wedge restoration, all 32 cases in the test set were used to calculate the statistics of the metrics and reported in the tables in the paper.

Weakness 1 (and Feedback 1~8): the mathematical derivation of the algorithm seems to be a bit sloppy

There are mainly two concerns raised by the reviewer:

• The sign error of vector field and the likelihood term in Section 3.2 and Section 4.3.
• Some incorrect terms in the algorithms, including the missing time increment $\Delta t$, the confusing meaning of the subscript $t$. We find that all of your suggestions are correct and deeply appreciate the reviewer. We have updated the manuscript and hope it addresses your concerns. Here are the detailed responses:

Feedback 1: the sign error of the vector field

• We thank the reviewer for pointing this out. We changed the sign of the vector field in the updated version.

Feedback 2: the oversimplified loss

• Thank you for your feedback. We have updated the manuscript to make it more rigorous.

Feedback 3: the sign of the likelihood term

• This is our mistake. We are grateful for you pointing this out. The error arises from line 192 where the gradient to the log-probability should be proportional to the negative loss function. We have corrected both line 192 and line 196 in the updated manuscript.
• the version in the original manuscript: $\nabla_{\mathbf{x}_t} \log p(\mathbf{y}|\mathbf{x}_t) \propto \nabla _{\mathbf{x}_t} \\|\mathbf{y}-\mathcal{A}\hat{\mathbf{x}}_0(\mathbf{x}_t)\\|^2_2$
• the correct version (updated): $\nabla_{\mathbf{x}_t} \log p(\mathbf{y}|\mathbf{x}_t) \propto -\nabla _{\mathbf{x}_t} \\|\mathbf{y}-\mathcal{A}\hat{\mathbf{x}}_0(\mathbf{x}_t)\\|^2_2$

Feedback 4: the missing explicit time increment in the algorithm

• Thank you! This makes the algorithms significantly more rigorous. We have updated Algorithm 1, Algorithm 2, and Algorithm 3.

Feedback 5.1 & Feedback 6: please clarify the meaning of $t-1$

• We are extremely grateful to the reviewer for bringing this to our attention. The subscript $t$ denotes a continuous time rather than a discrete index. We overlooked this when writing the manuscript, and we have updated all algorithms to enhance the clarity.

Feedback 5.2 & Feedback 6: the loop should end at $\Delta t$

• Thank you! You are correct and we have updated them.

Feedback 7: symbol $\theta$ is overloaded

• Thank you for your careful reading. We have changed the symbol of the network paramter from $\theta$ to $\Theta$. In cryo-EM, $\theta$ is widely adopted to denote a certain angle. Therefore, we change the symbol of the parameter.

Feedback 8: multiple versions of the flow, $\mathbf{v}_t(\mathbf{x}_t)$ and $\mathbf{v} _\theta(t, \mathbf{x}_t)$, and the flow should be bold face

• On page 3 of the previous version, we wrote a footnote that "for ease of notation, sometimes we omit the $\theta$ and use $\mathbf{v}_t(\mathbf{x}_t)$ for abbreviation (of $\mathbf{v} _\theta(t, \mathbf{x}_t)$) in later sections." However, we concur with you that this is sometimes perplexing. Therefore, we have updated the manuscript to maintain consistency among all of them.
• We have also changed all of the flow to bold face. Thank you for your careful reading!