Mixture of neural fields for heterogeneous reconstruction in cryo-EM

We thank our reviewer for their comments on the significance and the difficulty of the problem our method addresses. We hope to address their concerns in the following response.

Synthetic Datasets

To further validate Hydra on the tomotwin dataset, we generated a larger version of this dataset (30,000 particles) with the same SNR. Results are shown in Figure A2 (see document attached to the shared rebuttal).

For synthetic data like tomotwin, we use the ground-truth maps to quantitatively evaluate the reconstruction results. Following our reviewer’s suggestions, we provide 3D resolutions for the larger tomotwin dataset (30,000 particles) in Table A1 of the attached document.

The tomotwin and ribosplike datasets are built with a uniform distribution over the three possible states, while the three states in the experimental dataset (RyR) are present in very different proportions (RyR: 71%, p97: 8%, CIII: 13%, junk: 8%, according to cryoSPARC and Hydra, Fig 3.b). We demonstrate Hydra’s ability to process all these datasets, confirming the possibility to handle different types of distributions over classes.

We acknowledge the existence of a gap between the SNR in synthetic datasets vs. experimental datasets. However, for synthetic datasets, we compare Hydra to baselines (cryoSPARC, cryoDRGN2, DRGN-AI) using the same level of noise and show an improvement in reconstruction quality (Table 1, Table 2, Fig S7). Our experiments on the real dataset validate Hydra’s ability to handle realistic noise levels and to outperform existing methods (Fig 3.b, S2, S6).

Reconstruction Quality Evaluation

• We will include movies of the states shown in Fig 4 to better show their conformational changes.

• Per-image FSC measures per-particle 3D resolution on synthetic datasets (where ground truth maps are available) and is a standard metric for assessing the reconstruction quality of methods that can represent conformational (continuous) heterogeneity. We refer, for example, to Fig 3 and Supplementary Figure 1 of the cryoDRGN paper [1]. For the tomotwin dataset, which only contains compositional heterogeneity, we provide per-class 3D resolutions with a threshold at 0.5 FSC in Table A1 (see document attached to the shared rebuttal).

[1] Zhong, E. D., Bepler, T., Berger, B., & Davis, J. H. (2021). CryoDRGN: reconstruction of heterogeneous cryo-EM structures using neural networks. Nature methods, 18(2), 176-185.

Choice of Metrics for Pose Evaluation

In order to better assess pose accuracy, we provide per-class angular and translation errors for the tomotwin dataset in Table A1, and for the ribosplike dataset in Table A2 (see document attached to the shared rebuttal).

Ablation Study

Fig 2 provides an analysis on the influence of $K$. When $K$ is too low (Fig 2.a), the reconstruction is inaccurate. When $K$ is too large, some of the capacity of the model is not used, but the reconstructed states are accurate (Fig 2.b). To further stress-test Hydra, we provide additional results using $K=7$ on the tomotwin dataset in Fig A1 (see document attached to the shared rebuttal).

To obtain the optimal value of $K$, one can run Hydra several times with increasing values for $K$. Although choosing a larger-than-optimal value for $K$ would lead to unnecessary computation, the quality of the reconstruction does not degrade when $K$ is too large, as shown in Fig 2 (b vs. c).

We acknowledge that this sweeping procedure requires time, energy and memory and could be avoided with a principled way of choosing $K$. This limitation is currently mentioned in the Discussion section (L311-317), and potential mitigation strategies are suggested.

Miscellaneous

• We thank our reviewer for pointing out the wrong chirality of the spliceosome in Fig 4. Since the chirality of a molecule is not identifiable from cryo-EM projections, we showed the chirality that we got out of Hydra without modification. We will flip (mirror) the density map to show the correct chirality.

• While the choice of proteins in the ribosplike dataset may be unrealistic, having a mixture of different particle types in cryo-EM is not unrealistic, as evidenced by imaging of a lysate sample in the RyR experimental dataset. We agree with our reviewer: adapting this method to sub-tomogram averaging of in situ mixtures is a promising avenue for future work (L320).

• We always use the default parameters of cryoSPARC, both for synthetic and experimental datasets. We will mention this in the supplements.

• Table 1 focuses on the tomotwin dataset, a synthetic dataset with strong compositional heterogeneity. The Table shows previous neural-based methods fail on this dataset. CryoSPARC performs well, but is unable to reconstruct conformational heterogeneity, which is quantitatively shown in Table 2.

• We will clarify the contributions in L57-62.

Questions

• We thank our reviewer for suggesting to provide access to the reconstructed maps. We will follow this suggestion and add the reconstructed maps to the supplementary material, in the mrc format. To make our method fully reproducible, we will provide access to the code upon publication.

• We include qualitative comparisons to DRGN-AI and cryoDRGN2 in Fig S2, S4.b, S6 and S7.

• In Fig 4, the conformational states are manually selected in the latent space. In other figures, we use k-means clustering (L247, L940, caption of Fig S2, S4, S7).

• We evaluate the efficiency of Hydra by comparing GPU runtimes on the tomotwin dataset using a single NVIDIA A100 GPU. CryoDRGN2 required 1h50min. DRGN-AI required 1h20min. Hydra required 4h when $K=3$ and 6h20min when $K=5$. We will include this information on runtime efficiency in the supplemental.

We thank our reviewer for their comments. We hope to address them in the following response.

Distinction with Previous Works

Hydra uses the pose search strategy and autodecoding framework from DRGN-AI [1]. It primarily differs by using several neural networks, “latent scores” (L197) and a new loss function involving a marginalization over the possible classes (Eq 7). Compared to DRGN-AI, Hydra broadens the scope of datasets that can be handled. We will clarify that the pose estimation strategy was already used in DRGN-AI (it is only adapted to consider the simultaneous representation of several classes).

We thank our reviewer for pointing out the relevant reference [3], which we will cite. Although the references [2] (3DVA) and [3] reconstruct a combination of 3D arrays, those are not density maps but rather vectors with the same dimension as density maps, representing the basis of a low-dimensional linear space. The conformation heterogeneity is then represented in the linear space spanned by these vectors (the “structural basis” [3]). Unlike Hydra, 3DVA (cited on L34 and L76) does not handle compositional heterogeneity (L38).

EMPIAR-10076

We thank our reviewer for this suggestion. EMPIAR-10076 (assembling ribosome) would indeed be a relevant dataset to demonstrate our method. We will run additional experiments on this dataset and try to include results in the supplement.

We thank our reviewer for their constructive comments and overall positive rating of our submission.

Validation on an Experimental Dataset Combining Compositional and Conformational Heterogeneity

As mentioned by our reviewer, processing real cryo-EM data often comes with unforeseen and significant challenges. By demonstrating Hydra’s ability to process the “ryanodine receptor” (RyR) mixture dataset, we hope to provide evidence that our method can process experimental data, in spite of the challenges and non-idealities that come with it. We agree with our reviewer on the fact that demonstrating the unique capabilities of Hydra on an experimental dataset with mixed heterogeneity, and potentially revealing motion or complexes that could not be seen before, would constitute an exceptional scientific leap. By sharing our work (and our code), we hope to provide new capabilities to the cryo-EM community and enable it to make this leap.

Questions

• We will clarify the main differences between Hydra and DRGN-AI in L49: use of several neural networks, the presence of “latent scores” and a new loss function involving a marginalization over the possible classes.

• We will clarify the term “sequential order” on L79. The explanation given by our reviewer is correct.

• We will add a reference to Moscovich et al (Cryo-EM reconstruction of continuous heterogeneity by Laplacian spectral volumes, Inverse Problems, 2020) on L82. Thank you for pointing this out.

• We will add the missing character on L130.

• We will add “Section” in the caption of Fig 1.

• In the current architecture, the conformational latent vectors are stored in a $N$-by-$d$-by-$K$ array, meaning that all classes have the same latent dimension. We hypothesize that the latent dimension $d$ must be greater or equal to the largest number of degrees of freedom among all compositional states. However, it would be possible to use per-class dimensions $d_k$, for example using a dictionary of $K$ arrays of dimensions $N$-by-$d_k$. We will mention this in the Discussion section.

• We will fix the typo in Eq 4, thank you for pointing it out.

• For poses, the point estimates are only used in the second phase of optimization (the Stochastic Gradient Descent phase). During the first phase (Hierarchical Pose Search), poses are exhaustively searched over using the strategy described in Section B (supplementary material). This HPS phase enables Hydra to perform ab initio reconstruction.

• The exhaustive search (Eq 9) is only applied to poses on a predefined number of images (~1e6). The schedule describing the switch from HPS to SGD is further described in L869-871 (supplementary material).

• Following our reviewer’s suggestion, we report per-class resolutions at an FSC cutoff of 0.5 for the tomotwin dataset in Table A1 (see document attached to shared rebuttal). We note that a per-image FSC metric is better suited for the ribosplike dataset as it accounts for conformational differences in the true structural distribution.

• We recognize that the term “single pass” is misleading in L227. We will replace it with “single run”.

• The SNR for the dataset described in Section 4.3 is 0.1. We will add this information to the supplementary information, where the generation protocol is described (Section F).

• The mention of “4 GPU-days” relates to the experimental dataset (85k images). The 3,000 image dataset tomotwin3 was processed using a single NVIDIA A100 GPU and required 4h when $K=3$ and 6h20min when $K=5$. DRGN-AI required 1h20min. We will clarify this point in the supplement.

We thank our reviewer for their comments. We hope to address their questions and concerns in the following response.

Distinction with DRGN-AI

Hydra uses the same pose search strategy and autodecoding framework as DRGN-AI. It primarily differs by using several neural networks, “latent scores” (L197) and a new loss function involving a marginalization over the possible classes (Eq 7). By doing so, Hydra broadens the scope of datasets for cryo-EM reconstruction. In particular, we demonstrate that, unlike Hydra, DRGN-AI fails at processing datasets with strong compositional heterogeneity, both on synthetic (Fig 2.a, S7.b) and experimental (Fig 3.d, S2, S6) datasets.

We will clarify, in the description of the method, what elements are borrowed from DRGN-AI.

Determination of $K$

The optimal value of $K$ can be obtained by running Hydra several times with increasing values for $K$. Although choosing a larger-than-optimal value for $K$ would lead to unnecessary computation, we find that the quality of the reconstruction does not degrade when $K$ is too large, as shown in Fig 2 (b vs. c). We provide additional results using $K=7$ on the tomotwin dataset in Fig A1 (see document attached to the shared rebuttal).

We acknowledge that this sweeping procedure requires time, energy and memory and could be avoided with a principled way of choosing $K$. However, other conventional approaches for heterogeneous reconstruction (cryoSPARC, RELION) are limited in the same way, while not being able to handle conformational heterogeneity (Fig S7.c). This limitation is currently mentioned in the Discussion section (L311-317), and potential mitigation strategies are suggested.

Comparison to Conventional Approaches

Hydra is qualitatively compared to cryoSPARC on the experimental dataset (Fig S5) and on the synthetic ribosplike dataset (Fig S7). We thank our reviewer for suggesting to provide access to the reconstructed maps. We will follow this suggestion and add the maps reconstructed with cryoSPARC, DRGN-AI and Hydra to the supplementary material, in the mrc format.

Metrics on Pose Estimation

To better assess the accuracy of pose estimation, we provide both translation and rotation errors for the tomotwin (Table A1) and ribosplike (Table A2) datasets. For rotation errors, we use the geodesic distance in $SO(3)$ (angular error), in degrees, which is a more interpretable metric than the Frobenius norm. We will add these metrics to the supplementary material.

We thank our reviewer for their constructive feedback and suggestions on ways to improve the clarity of the manuscript.

Clarification of Prior Work and Contributions

We thank our reviewer for pointing out the lack of clarity in the presentation of prior works and apologize for the absence of important context. We will review our introduction and make an effort to make it clear for a broader audience.

Clarification of Field-Specific Terminology

Again, we apologize for the oversight on using field-specific terminology (pose, conformational/compositional heterogeneity). We will clarify these terms in order to allow non-cryo-EM experts to appreciate the presented work and potentially borrow from its ideas.

Distinction with DRGN-AI

Hydra uses the pose search strategy and the autodecoding framework from DRGN-AI. It primarily differs by using several neural networks, “latent scores” (L197) and a new loss function involving a marginalization over the possible classes (Eq 7). By doing so, Hydra broadens the scope of datasets for cryo-EM reconstruction methods. As suggested by our reviewer, we will clarify that we use the pose estimation strategy in DRGN-AI and that it is adapted to cope with the simultaneous representation of multiple maps.

Standard Deviations in Table 1

The standard deviations reported in Table 1 characterize the spread of the area under the Fourier shell correlation obtained over 20 images (with different conformational states $z$) per class. This can only be measured for neural-based methods (Hydra, DRGN-AI, cryoDRGN2), hence the absence of standard deviation for cryoSPARC, which outputs a single conformational state per class.

Code Release

Yes, we will release our code upon publication.

Questions

• We agree with our reviewer’s intuition. The use of $K$ neural networks increases the expressivity of the representation and probably decreases the number of latent dimensions required to capture conformational motion. Empirically, we found that a dimension of two was sufficient to obtain accurate density maps, and to capture continuous motion in the ribosplike dataset (Fig 4). However, if Hydra had to process a dataset where one of the compositional states had strictly more than two degrees of freedom, the dimensions of the latent space would have to be increased. We hypothesize that this dimension must be greater or equal to the largest number of global degrees of freedom among all compositional states. We will add a brief discussion on the choice of the latent dimension in the last section.

• Our reviewer’s intuition on the correlation between pose error and image-FSC is correct. In order to provide a clearer assessment of each method’s performance, we show updated metrics for the ribosplike dataset, including rotation and translation errors, in Table A2 (see document attached to the shared rebuttal).

• To show that the poor performance of DRGN-AI on the ribosplike dataset is not linked to the low dimension of the latent space, we give eight dimensions to the latent space of DRGN-AI (L244). We therefore hypothesize that Hydra’s ability to reconstruct this dataset can be ascribed to the mixture model. We provide qualitative reconstructions and a UMAP plot of the conformations obtained with DRGN-AI in Fig S7.