Thank you for your appreciation and insightful comments. We will improve the clarity of the paper based on them.

Clarification of Notations

Thank you for thoroughly reading our paper and the supplementary material. We truly appreciate your suggestions.

For mutual information extraction (Equation 5), we followed CUT [1], where $\boldsymbol{v}, \boldsymbol{v}^+ \in \mathbb{R}^{N}$ represent an arbitrary pair of feature vectors of the query and the positive sample from the whole set. However, in our case, we have $Q$ pairs, and it will be clearer to rewrite $\boldsymbol{v}, \boldsymbol{v}^+ \in \mathbb{R}^{N}$ to $\boldsymbol{v}_q, \boldsymbol{v}_q^+ \in \mathbb{R}^{N}$. We will fix this in the revised version.

The "selecting" operation in Equation 5 corresponds to a multi-class classification task, aiming to maximize the probability of correctly matching the corresponding positive $\boldsymbol{v}_q^+$ from a set of it and $K$ negatives, given the query $\boldsymbol{v}_q$. Therefore, we formulate Equation 5 to minimize a $K+1$-class cross-entropy loss.

In Equation 13, $v$ is a unit vector $(0,0,1)$.

We will clarify all of these points in the revision.

[1] Park T, Efros A A, Zhang R, et al. Contrastive learning for unpaired image-to-image translation[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16. Springer International Publishing, 2020: 319-345.

The Origin of Coarse Models

Thank you for your attention. We will add this description to the Datasets section for better clarity. The coarse 3D model is obtained by running an ab-initio reconstruction of CryoSPARC with its default setting, followed by cryoDRGN to handle heterogeneous cases. We train cryoDRGN for 50 epochs using particles at a resolution aligned with the low-resolution 3D volume.

Thank you for your thoughtful suggestions. We will improve the paper based on them.

On the Relationship between Gaussian and Actual Physical Noise

We follow the common practice in the literature of using Gaussian noise to model the reconstruction problem in cryo-EM. For example, cryoDRGN [1] and RELION [2] model image noise in the Fourier domain as zero-mean, independent Gaussian distributed noise. The inverse Fourier transform of this noise in the real domain is also i.i.d. Gaussian noise.

We'd like to stress that CryoGEM can also accommodate other noise models, such as signal-dependent Poisson noise [3], by replacing the current Gaussian noise model with the new one during training. If space permits, we will include an example of this.

[1] Zhong E D, Bepler T, Davis J H, et al. RECONSTRUCTING CONTINUOUS DISTRIBUTIONS OF 3D PROTEIN STRUCTURE FROM CRYO-EM IMAGES[C]//8th International Conference on Learning Representations, ICLR 2020. 2020.

[2] Scheres S H W. RELION: implementation of a Bayesian approach to cryo-EM structure determination[J]. Journal of structural biology, 2012, 180(3): 519-530.

[3] Vulović M, Ravelli R B G, van Vliet L J, et al. The Supplementary Material of Image formation modeling in cryo-electron microscopy[J]. Journal of structural biology, 2013, 183(1): 19-32.

On the Effort to Capture More Samples

We agree that capturing more micrographs can improve the resolution of the final results. However, this approach leads to longer capture time on expensive cryo-EM equipment and demands substantial computational resources for iterative optimizations, often taking several days for a human expert.

Complementarily, CryoGEM aims to enhance downstream results without the need for a large amount of raw data, thereby improving the final resolution of resolved structures. This aligns with the recent generation-based reconstruction approaches [1], where sparse view reconstruction is achieved by generative models.

In terms of resource consumption, CryoGEM is a lightweight generative model that can be trained on 100 real micrographs using a single NVIDIA RTX 3090 GPU in just two hours. After training, it can rapidly generate annotated synthetic datasets with minimal additional cost. Therefore, the best practice should combine efficient data capture with advanced data processing, such as inputting as many as possible samples into a CryoGEM-improved pipeline for optimal reconstruction results.

[1] Wang S, Leroy V, Cabon Y, et al. Dust3r: Geometric 3d vision made easy[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 20697-20709.

Thank you for your valuable suggestions. We appreciate the opportunity to address these concerns.

On the Pipeline Efficiency

We acknowledge that CryoGEM indeed takes longer than manual annotating, as shown in the following table. However, particle picking is a tedious, labor-intensive, and time-consuming task for technicians. Even with a blob picker, identifying target particles from the candidates requires excessive effort. CryoGEM enhances productivity by automatically generating annotated data for particle picking with high precision, although at the expense of speed.

The bottleneck in CryoGEM's pipeline is the particle labeling (generation) time, which includes ab-initio reconstruction to get the coarse volume, as well as the training and inference time. Currently, CryoGEM's training and inference are conducted on a single RTX 3090 GPU. To improve speed, we are developing a more parallelized GPU version, which could potentially accelerate the process by an order of magnitude. Additionally, AlphaFold3 serves as a potential alternative to CryoSPARC, which could further speed up ab-initio reconstruction.

Comparison with Template Matching

The particle picking approach of CryoGEM is not compared with template matching from the coarse cryo-EM input map, which could provide a more comprehensive evaluation of its advantages. Thank you for the suggestion. It is indeed an excellent point. The following table shows the suggested quantitative comparison of our finetuned Topaz (Ours, by CryoGEM's synthetic annotated datasets) with cryoSPARC’s template-based matching method (Template Picker, using the coarse volume as an input). CryoGEM consistently outperforms the Template Picker in nearly all examples in both AUPRC and resolution, except for the AUPRC of Proteasome and the resolution of Integrin. We will add Template Picker as a picking baseline in the revision.

Comparisons with Pre-trained CryoFIRE

Thank you for your insightful comments. We did not include this comparison because we focused on evaluating the performance improvement between our method (Ours) and the original CryoFIRE. However, it is indeed more convincing to compare our method with the pre-trained cryoFIRE (CryoFIRE*) using the particle images whose poses are estimated at the ab-initio reconstruction stage. As shown in the following table, the pre-trained cryoFIRE exhibits a reasonable performance improvement compared to the original CryoFIRE. However, our method still outperforms both schemes by a large margin in most cases.

On the Code Availability

All of our code and datasets will be released to the public upon acceptance for further evaluation of CryoGEM. In the meantime, we will provide an anonymous version of our code for training and evaluating CryoGEM to the area chair.