TEMPO: Temporal Multi-scale Autoregressive Generation of Protein Conformational Ensembles

We sincerely thank the reviewer for their detailed feedback on our modeling framework choices, evaluation methodology, and concerns about result consistency and generalization performance. Below are our detailed responses addressing each point.

W1

We appreciate this comment. While our current implementation uses pre-specified timesteps, this design choice reflects the established biophysical principle that protein dynamics naturally separate into slow collective motions and fast local fluctuations. The framework can be naturally extended to multiple scales by adding coupled SDE terms, similar to adaptive resolution schemes in image generation. Unlike ITO, which focuses on coarse-grained Cα representations and uses exponential distribution sampling, our approach models full backbone dynamics while maintaining temporal coherence and biological interpretability. ITO shows challenges in long-time scale prediction (as evidenced by negative VAMP-2 gaps that increase with lag time). In contrast, our hierarchical approach decomposes complex long-timescale dynamics into more tractable multi-step processes. Additionally, ITO was only evaluated on 12 folding proteins, making it difficult to assess its model capacity and generalization ability on larger protein datasets, whereas our model is trained on the comprehensive mdCATH dataset and tested on novel proteins to demonstrate robust generalization capabilities. We will provide the discussion of these methodological differences in the revision.

W2.a

We appreciate this observation and note that unlike ensemble methods that perform independent sampling, TEMPO generates temporally coherent MD trajectories while maintaining thermodynamically consistent free energy surfaces, demonstrating comparable ensemble quality and superior trajectory fidelity through lower RMSD error compared to MDGEN, and while our clash ratio is higher than MDGEN, it remains significantly lower than ensemble-based baselines like BioEMU and AlphaFlow, with visualizations revealing that MDGEN‘'s lower clash ratio comes at the cost of severely restricted sampling confined to very small conformational regions on the mdCATH dataset, and our clash loss has already substantially improved inference stability with future incorporation of enhanced physical constraints expected to further reduce the clash ratio.

W2.b

The smoothed curve in Figure 2 occurs because we compare our coarse-resolution trajectories (20ns intervals) with full MD ground truth (1ns intervals), as our coarse model is designed to capture slow collective motions rather than high-frequency fluctuations.

W2.c

We thank the reviewer for raising this concern about mode coverage challenges and acknowledge that capturing all distribution modes represents a fundamental difficulty in protein dynamics modeling, as even extensively pretrained methods like BioEMU show significant FES deviations (Appendix Figure 9). While our primary objective focuses on generating realistic conformational transitions rather than perfect mode coverage (Figure 4 and Appendix Figure 8), we recognize the performance gap between training and test scenarios reflects the complexity of generalizing to novel protein architectures in mdCATH where most proteins contain distinct energy basins. We have added TEMPO's performance on the training set in the table below, evaluated on 60 randomly selected proteins, where the training set demonstrates significantly better mode coverage accuracy. Due to conference policy restrictions on supplementary figures, we will provide comprehensive training set visualizations in the revision to illustrate this performance.

W2.d

We clarify that "focused exploration" refers to TEMPO's adaptive sampling around physically relevant conformational regions rather than uniform dispersion, which enables more efficient capture of large-scale transitions when they occur (Figure 4 and Figure 8 in the Appendix). This contrasts with baseline methods that show more dispersed but less physically meaningful patterns, particularly evident in Figure 9 in the Appendix, where our trajectory-aware approach maintains tighter distributions around energetically favorable states while still capturing the essential conformational transitions that define protein dynamics.

We appreciate the reviewer's feedback and respectfully note that protein dynamics generation represents an emerging and challenging field where our work addresses the more complex task of capturing temporal conformational evolution rather than just equilibrium ensemble generation, and our comprehensive evaluation demonstrates consistent improvements over baselines across multiple metrics on the demanding mdCATH dataset, with additional experiments on the ATLAS dataset(kindly refer to our respose to Q4) showing our model's substantial performance advantages over baselines through both quantitative metrics and visualizations, confirming our model's advanced ability to capture conformational transition pathways, particularly for proteins with complex free energy landscapes featuring multiple basins, while our multi-scale SDE formulation significantly reduces computational demands during training and maintains high efficiency during inference compared to direct trajectory decoding approaches used in prior work.

Q1

We thank the reviewer for this question about our modeling framework choice. Our deterministic prediction approach is theoretically justified because we are essentially learning the conditional expectation E[X_{t+1}|X_t] in the finite time-step regime, which mirrors how numerical MD integrators operate with deterministic updates plus controlled stochastic components for temperature regulation[1,2]. Additionally, our approach parallels transformer-based language models, where deterministic training (cross-entropy loss) still enables diverse generation during inference - similarly, we preserve stochastic diversity via Langevin noise while learning conditional expectations through MSE loss. At the 1ns sampling interval used in mdCATH, consecutive frames exhibit strong deterministic correlations due to physical inertia, making our model appropriate for capturing dominant drift dynamics, and our SDE formulation dX_t = μ(X_t)dt + σdW_t directly mirrors actual MD implementation, where deterministic force calculations drive conformational changes and stochastic thermostats introduce thermal effects.

[1] Effective stochastic dynamics on a protein folding energy landscape.

[2] Memory effects in irreversible thermodynamics.

Q2

We appreciate this question regarding trajectory continuity, and while potential discontinuities could theoretically occur in our hierarchical approach, our evaluation demonstrates that such instances are infrequent and remain within acceptable error ranges as evidenced by our low RMSD error metrics (Table 1) and smooth trajectory coherence (Figure 4 and Figure 8 in the Appendix), and developing more sophisticated continuity constraints represents an important direction for future work.

Q3

During training, we can optimize through approximately 20 autoregressive steps corresponding to our trajectory length under the multi-scale setting without encountering memory or gradient stability issues, and while we have not systematically explored reducing teacher forcing probability or extending backpropagation through longer sequences, this represents an important direction for future investigation that could potentially improve long-term trajectory coherence.

Q4

We appreciate the reviewer's question. We chose mdCATH over the widely used MD dataset ATLAS because its free energy landscapes contain more distinct energy basins with clearer conformational transition pathways better suited for our dynamics modeling task. At the same time, we evaluated baselines using their provided checkpoints since BioEMU has already been pretrained on diverse MD data and PDB structures; ESMFlow and AlphaFlow were not designed for MD generation. MDGEN encounters severe memory limitations on mdCATH due to its full-trajectory attention mechanism (requiring batch size of 1 and still experiencing out-of-memory errors on 8 A100 GPUs when training 400-frame conformations), but to address fairness concerns, we also trained TEMPO on ATLAS following MDGen's splits and results in table below demonstrate our method's strong performance across different datasets, achieving state-of-the-art results on ATLAS, with FES visualizations of generated ensembles corroborating our quantitative results, comprehensive visualizations will be provided in the revision due to current supplementary figure limitations.

Dear Reviewer HVNL,

With the final hours of the discussion period approaching, we appreciate your engagement with our work. We're standing by if you have any additional questions or reflections.

Thank you for your thoughtful comments.

Best regards,

The Authors

We sincerely thank the reviewer for their thoughtful suggestions on adaptive timescale learning, trajectory continuity concerns, and potential extensions to full-atom modeling. Below are our detailed responses addressing each point.

W1

We appreciate this constructive suggestion and acknowledge that while our current fixed two-scale decomposition reflects the established biophysical separation of protein dynamics into slow collective motions and fast local fluctuations, our hierarchical SDE formulation can naturally be extended to learnable or adaptive timescales, and we view this as a promising direction for future work that could enhance the framework's robustness across diverse protein systems with varying characteristic timescales.

W2

We appreciate this question regarding trajectory continuity, and while potential discontinuities could theoretically occur in our hierarchical approach, our evaluation demonstrates that such instances are infrequent and remain within acceptable error ranges as evidenced by our low RMSD error metrics (Table 1) and smooth trajectory coherence (Figure 4 and Figure 8 in Appendix), and developing more sophisticated continuity constraints represents an important direction for future work.

W3

We thank the reviewer for this valuable suggestion and clarify that, as an initial attempt, we focused on validating backbone trajectory generation within the TEMPO framework since backbone atoms determine the primary conformational changes of proteins, but we fully agree that sidechain generation is equally important and valuable. Extending to full-atom representation requires significantly larger model capacity and more physical constraints, making the incorporation of sidechain dynamics generation one of our key objectives for future model improvements.

W4

We appreciate this observation regarding data dependency, and note that while the current scarcity of comprehensive MD benchmarks and large-scale pretraining datasets for protein dynamics generation constrains our evaluation to mdCATH, our framework is designed to leverage temporal dependencies that become increasingly valuable as larger and more diverse MD datasets emerge, and we believe our approach will demonstrate advantages as the field develops richer benchmarks for protein dynamics modeling.

W5

The requirement for initial frames is inherent to the nature of molecular dynamics simulation and trajectory generation, as our method aims to generate temporally coherent MD trajectories that capture physically realistic transitions from a given starting conformation, and importantly, our model's performance is not limited by the source of the initial structure, allowing it to be coupled with any structure prediction method to generate dynamics from sequence-predicted conformations.

Q1

We appreciate this valuable suggestion and acknowledge that leveraging pretrained structure prediction models could potentially enhance our framework's performance, though we have not yet explored this direction due to computational constraints, representing an important avenue for future investigation.

Q2

Extending TEMPO to full-atom structures is conceptually straightforward, as our framework can accommodate higher-dimensional representations, though computational constraints currently limit our focus to backbone dynamics, which capture the essential conformational changes for most biological processes, and full-atom extension represents a key objective for our future model improvements.

Q3

We empirically chose the 20ns/1ns hierarchy based on established biophysical principles where the 20ns interval effectively captures major conformational transitions between different states in the free energy surface as visualized in Figure 1a, while the 1ns resolution represents the dataset's finest temporal sampling, and as a model hyperparameter, different datasets may require empirical exploration of various resolutions to optimize performance since protein dynamics timescales can vary significantly across different systems and functional contexts.

Q4

We have explored training single-resolution models at 1ns intervals and found that autoregressive error accumulation becomes problematic for extended trajectories, with single-scale generation beyond approximately 400 frames resulting in structural deviations exceeding 8Å RMSD relative to the native structure compared to ground truth. We also conducted ablation studies comparing our hierarchical TEMPO framework against single-scale baselines where the high-resolution model directly generates all frames without slow-scale guidance, as shown in the table below, demonstrating the importance of our multi-scale design.

We sincerely thank the reviewer for their insightful questions regarding our method's design rationale, evaluation metrics, generalization capability, and ablation studies. Below are our detailed responses addressing each concern.

W1

We thank the reviewer for this question and note that MDGen represents the current sota open-source method designed for protein dynamic generation, while other temporal methods were either not publicly available or published after our submission, and we have conducted ablation studies comparing our hierarchical TEMPO framework against single-scale baselines where the high-resolution model directly generates all frames without slow-scale guidance as shown in table below, demonstrating the importance of our multi-scale design.

W2

We appreciate this question, and our evaluation framework appropriately focuses on statistical and thermodynamic consistency rather than exact trajectory reproduction by measuring free energy surface fidelity, dynamic consistency, and physically plausible conformational transition patterns in PCA space, recognizing that even independent MD simulations from identical starting conditions would show different specific trajectories while maintaining the same underlying statistical distributions and exploring similar regions of conformational space.

W3

We apologize for the insufficient description and clarify that these metrics, adopted from AlphaFlow's evaluation framework, collectively assess different aspects of protein dynamics: pairwise RMSD measures conformational diversity, RMSF correlation captures local flexibility patterns, Wasserstein distances quantify distribution similarity in conformational space, contact dynamics evaluates biologically relevant inter-residue interactions, RMSD error measures structural deviation accumulation during trajectory evolution, and clash ratio assesses physical plausibility. We will provide more detailed descriptions of their biological significance in our revision.

W4&Q7

Our training set of 1,000 proteins spans diverse structural classes from the CATH database representing different folds, topologies, and homologous superfamilies, with the mdCATH dataset inherently ensuring structural diversity through non-homologous protein domains, and we rigorously quantified sequence similarity using mmseqs2, finding averaged 18.93% sequence similarity between training and test sets, which is well below standard thresholds(40%) for sequence relatedness.

W5

The smoothed curve in Figure 2 occurs because we compare our coarse-resolution trajectories (20ns intervals) with full MD ground truth (1ns intervals), as our coarse model is designed to capture slow collective motions rather than high-frequency fluctuations. While our cosine similarity of 0.41 improves over MDGen (0.36), we acknowledge this reflects the fundamental challenge of capturing large-scale conformational changes and believe future improvements through enhanced model generalization, physical priors, and pretraining strategies could further advance this metric.

W6

The requirement for initial frames is inherent to the nature of molecular dynamics simulation and trajectory generation, as our method aims to generate temporally coherent MD trajectories that capture physically realistic transitions from a given starting conformation, and importantly, our model's performance is not limited by the source of the initial structure, allowing it to be coupled with any structure prediction method to generate dynamics from sequence-predicted conformations.

Q1

Following AlphaFlow's evaluation framework, Root mean W₂ is computed as the average Wasserstein distance between 3D-Gaussians fitted to positional distributions of each atom across ensembles, while MD PCA W₂ measures the Wasserstein distance in the space of the first two principal components derived from PCA of joint Cα position distributions, where PCA is computed from the MD ensemble and the distance quantifies distributional similarity in this reduced conformational space.

Q2

We appreciate this question regarding trajectory continuity, and while potential discontinuities between fast-scale segments could theoretically occur in our hierarchical approach, our evaluation demonstrates that such instances are infrequent and remain within acceptable error ranges as evidenced by our low RMSD error metrics (Table 1) and smooth trajectory coherence (Figure 4 and Figure 8 in Appendix), and developing more sophisticated continuity constraints represents an important direction for future work.

Q3

Since fast fluctuations are more stochastic, we increase noise levels to encourage diverse local conformational sampling. We empirically found that a higher noise scale in the high-res model provides an optimal balance between conformational diversity and trajectory fidelity, maintaining conformations within reasonable proximity to the data distribution while enabling sufficient local exploration, as validated by our low clash ratios and strong correlation metrics in Table 1.

Q4

TEMPO results use our complete hierarchical pipeline where the low-resolution model generates coarse trajectories that are then fed into the high-resolution model to produce full trajectories, while TEMPO(up) represents an up-sampling task using ground truth coarse-resolution conformations as anchors with only the high-resolution model filling intermediate frames, demonstrating that our fast-scale dynamics modeling component can effectively reconstruct complete trajectories when given accurate anchor points.

Q5

We quantitatively measure FES differences using the MD PCA W₂ metric reported in Table 1, which computes the Wasserstein distance in the space of the first two principal components derived from PCA of joint Cα position distributions, where our FES visualizations in Figure 3 are constructed using inter-residue Cα distances as features projected onto these same first two principal components. The TEMPO(up) results in Table 1 provide the quantitative performance evaluation corresponding to the upsampling task demonstrated in Figure 3, achieving an MD PCA W₂ distance of 0.60, which demonstrates strong distributional similarity in the reduced conformational space.

Q6

We thank the reviewer for this question. Our evaluation framework appropriately focuses on statistical and thermodynamic consistency rather than exact trajectory reproduction by measuring free energy surface fidelity, dynamic consistency, and physically plausible conformational transition patterns in PCA space, recognizing that even independent MD simulations from identical starting conditions would show different specific trajectories while maintaining the same underlying statistical distributions and exploring similar regions of conformational space, which is why we evaluate against ensemble properties rather than deterministic trajectory matching.

Q8

We have added the ablation study with our multi-scale decomposition, showing significant performance improvements over single-scale baselines, and the table below compares error propagation when directly generating 400 frames without multi-scale guidance. During model training, we found that both the spatial encoder (multi-head attention) and temporal encoder (GRU) components are essential for effective training, as removing either component leads to poor training convergence and substantially reduced model fitting capability on the training data.

Q9

The sequence embeddings are learned through a standard nn.Embedding layer during training.

Q10

Please refer to our response to W6. Our method aims to generate temporally coherent MD trajectories that capture physically realistic transitions from a given starting conformation, and we use two initial frames as the optimal hyperparameter (with slight performance degradation observed when using only one frame), as these initial conditions enable the temporal dependencies that are essential for dynamics modeling, which differ from the equilibrium ensemble generation approach targeted by BioEMU and AlphaFlow.

We sincerely thank the reviewer for their valuable feedback on our evaluation framework, dataset fairness concerns, and methodological rigor. Below are our detailed responses addressing each point.

W1&Q5

We appreciate the reviewer's concern. The mdCATH dataset is composed of non-homologous protein domains at the CATH S20 level (sequence similarity <20%), ensuring low sequence similarity between any two domains, and we rigorously quantified sequence similarity using mmseqs2, finding averaged 18.93% sequence similarity between training and test sets, which is well below standard thresholds (40%) for sequence relatedness and ensures our evaluation truly tests generalization to novel protein systems, with the ATLAS dataset showing similar results at 18.3% average sequence similarity.

W2

We appreciate the reviewer's concern. We chose mdCATH over ATLAS because its free energy landscapes contain more distinct energy basins with clearer conformational transition pathways better suited for our dynamics modeling task. At the same time, we evaluated baselines using their provided checkpoints since BioEMU has already been pretrained on diverse MD data and PDB structures; ESMFlow and AlphaFlow were not designed for MD generation. MDGEN encounters severe memory limitations on mdCATH due to its full-trajectory attention mechanism (requiring batch size of 1 and still experiencing out-of-memory errors on 8 A100 GPUs when training 400-frame conformations), but to address fairness concerns, we also trained TEMPO on ATLAS following MDGen's splits and results in table below demonstrate our method's strong performance across different datasets, achieving state-of-the-art results on ATLAS, with FES visualizations of generated ensembles corroborating our quantitative results, comprehensive visualizations will be provided in the revision due to current supplementary figure limitations.

W3

We appreciate the reviewer's comment regarding evaluation rigor, and following BioEMU's established methodology, we quantitatively compute free energy differences by extracting reaction coordinates (fraction of native contacts) to compute the folding probabilities ($p_{\text{fold}}$), calculating free energy differences as $\Delta G = -kT \cdot \log(p_{\text{fold}}/(1-p_{\text{fold}}))$, and measuring $\Delta\Delta G = \Delta G_{\text{ground truth}} - \Delta G_{\text{predicted}}$, with TEMPO achieving an averaged $\Delta\Delta G$ of 0.67 kcal/mol on the mdCATH test set, which demonstrates good agreement with reference MD simulations within the acceptable range for biological applications. BioEMU's evaluation framework further reveals that our generated trajectories exhibit high similarity to ground truth in 1D free energy profiles constructed using RMSD, radius of gyration, and fraction of native contacts, 2D free energy surface plots, and fraction of native contacts time series analyses, with comprehensive visualizations to be provided in the revision due to current supplementary figure limitations. As protein dynamics generation is an emerging field where standardized benchmarks are still under development, we agree that advancing more sophisticated biophysical evaluation frameworks would benefit the entire community.

Q1

We thank the reviewer for this question and clarify that while our approach shares conceptual similarities with coarse-graining methods, we employ temporal coarse-graining (sampling at 20ns vs 1ns intervals) combined with backbone-only spatial representation rather than traditional spatial coarse-graining of molecular interactions.

Q2

mdCATH proteins range from 50-482 residues with sequences truncated to a maximum of 240 residues during training for computational efficiency, but inference is performed at the original protein lengths, and since we generate 4 backbone atoms per amino acid residue, the average protein in our dataset contains approximately 548 atoms (137 average sequence length).

Q3

While autoregressive methods typically suffer from error accumulation during sequential generation, our hierarchical multi-scale design mitigates this issue by anchoring fine-scale dynamics to coarse-scale predictions, resulting in significantly lower RMSD error (1.78Å) after ~400 frames and demonstrating controlled error propagation that maintains trajectory stability over extended generation periods. We provide additional ablation analysis below showing error propagation when directly generating 400 frames without multi-scale guidance.

Q4

The mdCATH dataset contains 5,398 protein domains with 134,950 fully unbiased molecular dynamics trajectories simulated using CHARMM22 force field and TIP3P water model in the NVT ensemble with Langevin thermostat, neutralized and ionized with Na+/Cl− at 0.150 M concentration, and "standardized to 400 frames at 1ns intervals" means trajectories longer than 400ns were truncated while shorter ones were cyclically repeated to reach uniform 400-frame length for batched training, with 1ns representing the dataset's finest temporal resolution as detailed in the original mdCATH paper.

Thank you for the thoughtful questions and valuable suggestions in your review of our paper, and your feedback has been instrumental in improving our work. We have carefully addressed all the concerns you raised in our rebuttal and would greatly appreciate knowing whether our responses have adequately resolved your questions. If our responses have addressed your concerns, would you consider re-evaluating our work's contribution? Your expert evaluation is crucial for finalizing our contribution, and we thank you for your time and dedication.

Thank you very much for your thoughtful feedback and valuable insights. We completely agree with your assessment that better evaluations would greatly benefit this field and set the stage for more meaningful ML work.

We will include a detailed explanation of the evaluations and their biophysical relevance in our revision as you suggested. However, given that we are in the final day of the rebuttal period, it is hard for us to implement additional reliable metrics for evaluating structural diversity within this timeframe.

As the rebuttal period is nearing its end, do you feel that our current rebuttal experiments and explanations have adequately addressed your concerns about our work? If so, would you consider updating your rating accordingly?