We greatly appreciate the reviewer’s kind recognition of our work. We believe that we have carefully addressed your concerns and responded to your questions. The detailed responses are as follows:

W1: generalization of CrysBFN to other symmetrical data types

We sincerely appreciate your interest in exploring the potential generalization of CrysBFN to other symmetrical data types—a direction we also regard as promising for future research. Below, we offer a detailed explanation along with some initial experimental results:

• Molecular conformer generation： Jing et al [1] proposed generating conformations by diffusing only on the periodic torsion angles and leaving the other degrees of freedom fixed, the generation quality was proved to be superior. In this setting, the proposed Periodic Bayesain Flow of CrysBFN could be directly applied to the task for its high sampling quality and efficiency.

• All-atom 3D protein/peptide structure generation: the side-chain geometry can be represented using up to four torsion angles corresponding to rotatable bonds between side-chain atoms $\chi_i\in[0,2\pi)^4$[2]. Similarly, CrysBFN can be applied to model these torsion angles as well.

• Experiments on peptide generation task: Moreover, we have already conducted initial experiments applying the proposed periodic Bayesian flow to the generating the side-chain angles of peptides. Specifically, we align the setting with torsional flow component of PepFlow [2] and evaluate the effectiveness of periodic Bayesian flow in CrysBFN. And we reported the MAE between predicted angles and ground truth angles along with the correct rate where a prediction is considered as correct with MAE < 20°.

  	$\chi_1$ MAE	$\chi_2$MAE	$\chi_3$MAE	$\chi_4$MAE	Correct % $\uparrow$
  Rosseta	38.31	43.23	53.61	71.67	57.03
  DiffPack	17.92	26.08	36.20	67.82	62.58
  PepFlow	17.38	24.71	33.63	58.49	62.79
  CrysBFN	13.20	19.98	28.11	49.56	66.36

W2: additional comparative analysis of entropy-conditioning mechanism across varied datasets

Thank you for your valuable suggestions! We have conducted an ablation study experiment on Perov-5 (simpler structure with only 5 atoms) and MPTS-52 (more complex structures compared to MP-20) and tabularize the results as follows:

The results imply that the effectiveness of entropy conditioning. It worth noting that on more complex structures, such as MPTS-52, the model could benefit more from the entropy conditioning.

W3: detailed analysis of the overall computational cost for training and deployment

That's an excellent question, and we agree that evaluating the training and deploying efficiency of CrysBFN is a crucial aspect. Below, we provide a thorough comparison of the training time to convergence across different methods, using a single 24GB NVIDIA RTX 3090 GPU in our experiments (details are provided in Appendix C):

As for the deployment cost, we evaluate the required GPU hour to generate 10000 MP-20 crystals with the NFEs determined by the highest performance of different approaches, and the results are listed as follows:

We observe that: 1) FlowMM requires significantly more training and inferencing time compared to DiffCSP and CrysBFN, primarily due to its larger parameter count; 2) CrysBFN shows comparable training time with DiffCSP on simpler datasets and enjoys better training efficiency on more complex dataset, such as MPTS-52; 3) CrysBFN requires fewer GPU hours than the current method to generate 10,000 MP-20 crystals, making it more efficient for deployment.

Q1: improve the related work section and conduct a survey of the past work in crystal generations

Thank you for your suggestion! We have made an improvement on the related works section and further do a survey of the past works especially those based on implicit representations in Appendix F (due to the page limitation) as you suggested, including the papers you mentioned, and retrieved more related works. We also include the discussion on how CrysBFN complements current works by proposing a novel periodic Bayesian flow with higher sampling quality and efficiency which can be utilized in various real-world applications as mentioned in W1. We kindly hope that they could bring better clarity for you!

Q2: more discussion on related works and how this work is different and takes the research forward

We thank the reviewer for the interest in our work! As addressed in Q1, we have improved the related work section and included a more detailed discussion in Appendix F. Additionally, we have incorporated a comparison in the preliminary section, explaining how BFNs fundamentally differ from flow matching and diffusion models, with precise guidance for each generation transition.

Our paper advances research in the field from both theoretical and practical perspectives. First, we first address the challenge of extending BFN to a non-Euclidean manifold, which sets the foundation for future exploration of BFN's potential on general manifolds. Second, our proposed frameworks serve as an efficient generative model for the real-world task of crystal generation, achieving state-of-the-art results with notable improvements in sampling speed.

As you suggested, we have also elaborated on how CrysBFN contributes to addressing the periodic variable generation problem by effectively overcoming the challenges associated with constructing a periodic Bayesian flow in related work section and Appendix F. We hope these additions could provide better insight for you!

References:

[1] Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. Advances in Neural Information Processing Systems, 35:24240–24253, 2022.

[2] Li, J., Cheng, C., Wu, Z., Guo, R., Luo, S., Ren, Z., ... & Ma, J. Full-Atom Peptide Design based on Multi-modal Flow Matching. In Forty-first International Conference on Machine Learning.

We sincerely thank the reviewer for the recognition of our work and your concerns are addressed as follows:

W1: About the definitions and exposition on Bayesian Flow Networks

Sorry for causing confusion. We have refined the presentation of BFN in the preliminary section of the revised paper version and added two toy examples (Gaussian BFN and von Mises BFN) in the anonymous code link located in the toy_examples folder. We hope they could bring greater clarity and a better understanding of how BFN works.

W2&Q4: about the stability rate for CSP task in Table 2

Thanks for pointing it out! We calculate the stability rate for the CSP task using the same method in Table 5 following Gruver et al[1]. The results are as follows:

It could be found CrysBFN achieves consistently better performance than DiffCSP and FlowMM using stability as the CSP task metric.

We also observe that the stability rates on CSP task are generally lower than ab initio generation task. We consider that this is because the CSP task has a stricter reference to compute the stability; specifically, each composition in the CSP task has a well-optimized structure based on the energy hull from DFT calculations as its reference.

In contrast, ab initio-generated crystals tend to have novel compositions that are not included in the known stable structures resulting in a possibly higher reference hull energy. This is because the calculation of hull energy for a novel composition requires decomposition (like interpolation) from known stable crystals and such calculated energy may not be as low as the well-optimized reference structures.

W2&Q5: add FlowMM in the sampling efficiency experiment

Thank you for your suggestion! We initially did not include FlowMM in Figure 4 due to its larger parameter size, which makes a comparison based on NFE less fair. To address the reviewer's concern, we make a comparison in Figure 7 (revised version) and observe that FlowMM performs poorly in extremely low NFE settings (e.g., 20 steps), achieving only a 16.18% match rate. In contrast, CrysBFN achieves a significantly higher match rate of 60.02% with just half that NFE (10 steps) and consistently delivers the best sampling quality across all NFEs.

Q1: Can the torus example in Figure 3 be explained through geodesic interpolations of $\theta_{i-1}$ and $y$?

Yes, you are correct! And we believe that your intuition provides an insightful perspective on explaining how the Bayesian update works on the torus. Specifically, using the notation from the paper $\theta_{i}=\\{m_{i},c_{i}\\}$, $m_{i}$ lies on the geodesics between $m_{i-1}$ and $y_i$, the location of which is determined by interpolating $m_{i-1}$ and $y_{i-1}$ based on previous belief entropy $c_{i-1}$ and $\alpha_i$ which is the accuracy of $y_{i-1}$.

Q2: About the preliminaries, summarizing the basic BFN framework, and explaining why extending to periodic is not easy in 2-3 sentences in layman terms?

Summary of the basic BFN framework: Unlike well-established SDE-based approaches, e.g., Diffusion Models, and ODE-based approaches, e.g., Flow Matching, Bayesian Flow Networks define a generative process driven by successive Bayesian updates. Based on noised samples, the uninformative prior distribution (pure noise) $\theta_{0}$ is progressively updated to new posteriors $\theta_i$ with higher confidence and more information on the data (similar to the reverse process of Diffusion models).

The challenge of extending BFN to periodic: The uncertainties/entropies after consecutive Bayesian updates were guaranteed to be additive and monotone proved by Graves et al [2] in Euclidean space, which provides numerous desirable mathematical properties for theoretical derivation, training, and sampling. However, we find and prove that for random variables with periodicity, such additivity and monotonicity do not hold. Consequently, we need to rethink the whole BFN framework, construct the periodic Bayesian flow from scratch, and propose measures to tackle problems incurred in every component (including the entropy conditioning mechanism, reformulations of Bayesian flow, the fast sampling algorithm, and a numerical method for determining the accuracy schedule, etc).

Q3: about the sampling in Eq (3) including the definition of $y_1$, the way to sample the $y_i$ sequence, and whether that is based on Phi

Sorry for causing confusion.

Definition of $y_i$ (including $y_1$): Actually, $y_i$ is the noisy version of ground truth $x$ sampled from so-called sender distributions $y_i \sim p_S(y_i|x;\alpha_i)$ (similar to the forward process in diffusion models), where the parameters $\alpha_i$ control the signal-noise-ratio. For example, when $\alpha = 0$, $y_i$ is pure uninformative noise samples and $y_i$ is becoming more and more informative about $x$ as $\alpha$ increases. Notably, instantiations of $p_S$ for different data modalities (continuous, discrete, and periodic) are different: For continuous variables, $p_S(y|x; \alpha) = \mathcal{N}(y|x,\alpha^{-1})$. For periodic variables proposed in this paper, $p_S(y|x;\alpha)=vM(y|x,\alpha)$.

How to sample $y_i$ sequence: As mentioned above, every $y_i$ is independently sampled from sender distribution $**y**_i \sim p_S(**y**_i|**x**;\alpha_i)$ with corresponding accuracy parameter $\alpha_i$ and instantiation according to the modality, e.g. Gaussian for continuous variable.

Is that based on Phi: As stated above, the process of sampling $y_i$ does not involve the neural network. $\alpha_i, i\in[1,n]$ are predefined as hyper-parameters which is similar to the forward process of diffusion models.

Reference:

[1] Gruver, N., Sriram, A., Madotto, A., Wilson, A. G., Zitnick, C. L., & Ulissi, Z. W. Fine-Tuned Language Models Generate Stable Inorganic Materials as Text. In The Twelfth International Conference on Learning Representations.

[2] Graves, A., Srivastava, R. K., Atkinson, T., & Gomez, F. (2023). Bayesian flow networks. arXiv preprint arXiv:2308.07037.

Dear Reviewer Yjva,

Thanks again for your insightful and thoughtful comments!

As the reviewer-author discussion period is closing soon (November 26 at 11:59 pm AoE), we would like to gently remind you that we are eagerly awaiting your feedback on our response.

We have improved the presentation of BFN, added a comparison to FlowMM, and reported the stability results of the crystal structure prediction task.

If you have any further questions about our work, please do not hesitate to let us know.

Thank you once again and we genuinely look forward to hearing from you!

Best regards,

Authors

We thank the reviewer for the detailed review comments and insightful suggestions. We humbly believe that we have addressed the reviewer's main concerns. Consequently, we kindly hope that the reviewer could increase the rating score of our paper. The responses are as follows:

W1&Q3: comparison with DiffCSP++ and MatterGen-MP

Thanks for pointing out two related works! However, DiffCSP++ differs from CrysBFN in the evaluation pipeline, so we conducted an additional experiment to demonstrate the effectiveness of CrysBFN within this pipeline. Regarding MatterGen-MP, there are currently no available resources to facilitate a fair comparison between MatterGen and other methods. The details are given as follows:

MatterGen-MP: After carefully reviewing the available resources and communicating with the authors of MatterGen[1], we conclude that there is no open-sourced code or data to conduct fair comparisons between MatterGen and any method currently. The private Alex-MP-20 dataset used by MatterGen is curated from the MP and Alexandria databases, containing 607,684 DFT recalculated structures, which differs significantly from the public MP-20 dataset with only 45,231 structures. Additionally, the generated samples from MatterGen are refined using DFT, but the detailed settings for this refinement are not disclosed in their draft, making it impossible to align evaluation criteria without access to their code. Furthermore, the primary evaluation metrics of MatterGen, such as S.U.N., require a reference dataset for calculation. Without access to their private training data, reproducing or comparing these results becomes infeasible. These factors collectively limit the possibility of a direct comparison at this moment.

DiffCSP++: It should be noted that the CSP results reported for DiffCSP++ were obtained using a different setting compared to ours. Specifically, DiffCSP++ utilizes CSPML [2] to retrieve strong initial structures with high quality, which are then refined using diffusion, whereas CrysBFN generates structures entirely from scratch. To ensure a fair comparison for CSP tasks, we implemented a retrieval-refinement version of CrysBFN (CrysBFN+CSPML) based on CSPML, similar to DiffCSP++. We provide comparisons with DiffCSP++ for CSP task in the following table:

The results demonstrate that CrysBFN consistently outperforms DiffCSP++ in this setting. Besides, the key component introduced in DiffCSP++, i.e., the space group constraint, is orthogonal to our approach and could naturally be integrated into CrysBFN to potentially enhance its performance further.

W2&Q4: evaluation of uniqueness, novelty, and stability

Thank you for your suggestion and we agree that such evaluations should be included. We compute the uniqueness and novelty following FlowMM's settings while using the method in Gruver, et al[3] to calculate the stability because FlowMM does not provide its DFT code. The results are listed as follows:

It could be found that CrysBFN could outperform FlowMM and DiffCSP on stability and also on the metric S.U.N. which considers the uniqueness, novelty, and stability jointly.

W3&Q2: how methods are evaluated in NFE experiment

Sorry for causing confusion. In our experiments, the step sizes $\Delta t$ are adjusted for all models by varying the global number of steps as you mentioned, i.e. $\Delta t \times N_{\text{steps}} = 1$. Specifically, keeping all hyperparameters unchanged except for the number of steps, $N_{\text{steps}}$, we conduct the training process from random model parameters, utilize saved checkpoints to sample the complete trajectory from $t = 1$ to $t = N_{\text{steps}}$, and evaluate the generated crystals for each value of $N_{\text{steps}}$.

W3&Q3: sampling efficiency comparison with FlowMM or ODE sampling of DiffCSP

We initially did not include FlowMM in Figure 4 due to its larger parameter size, which makes a comparison based on NFE less fair. To make our evaluation more comprehensive, we additionally include FlowMM due to its superior performance compared to DiffCSP in Figure 7 (revised version) as you suggested. It could be observed that FlowMM performs poorly in extremely low NFE settings (e.g., 20 steps), achieving only a 16.18% match rate. In contrast, CrysBFN achieves a significantly higher match rate of 60.02% with just half that NFE (10 steps) and consistently delivers the best sampling quality across all NFEs. This highlights that entropy-based samplers used in BFN can achieve high sampling efficiency and even outperform ODE-based samplers.

W4: about the presentation of BFN, a small toy example, and introducing von Mises distribution parameters

Thank you for your suggestion!

About the presentation of BFN: As suggested, we have refined the presentation of BFN in the revised version and hope it brings greater clarity and a better understanding.

A small toy example: We have added two .ipynb notebooks to illustrate how Gaussian and von Mises based BFN work with minimal code under the toy_examples folder in our anonymous code repository. We kindly hope they could offer helpful insights.

Introduction to the parameters of von Mises distribution: We have added the explanation of how $m$ and $c$ influence the distribution shape in Appendix A (revised version) and updated Figure 5 with different mean direction parameters $m$.

Q1: SE(3) or E(3) equivariance

Thanks a lot for pointing it out! Note that the backbones of CrysBFN directly followed previous works of DiffCSP[1] for fair comparison, which uses an EGNN-like CSPNet and it is E(3) equivariant. And this could make the learned density function is E(3) invariance for CrysBFN which includes the invariance of reflections as the reviewer mentioned. We also believe that it could be better to consider the SE(3) invariance for distinguishing the chirality in the crystals, which could be done by simply changing the backbone architecture of CrysBFN to SE(3) equivariance. We have included the discussions as you suggested to help the community better clarify the concepts.

Q5: training efficiency comparison

That's a great question and we agree that evaluating the training efficiency of generative models is an important aspect. Below, we provide a comprehensive comparison of the number of trained batches per GPU hour and the training time until convergence for different methods, utilizing a single 24GB NVIDIA RTX 3090 GPU in our experimental results (details in Appendix C):

We observe that: 1) FlowMM requires significantly more training time compared to DiffCSP and CrysBFN, primarily due to its larger parameter count; 2) CrysBFN shows comparable training time with DiffCSP on simpler datasets and enjoys better training efficiency on more complex dataset, such as MPTS-52. According to the above statistics, the von Mises Bayesian update incurs almost no additional training cost. This is attributed to the proposed fast sampling algorithm, which eliminates the need for auto-regressive Bayesian updates and instead enables parallelized tensor operations.

References:

[1] Jiao, R., Huang, W., Lin, P., Han, J., Chen, P., Lu, Y., & Liu, Y. (2024). Crystal structure prediction by joint equivariant diffusion. Advances in Neural Information Processing Systems, 36.

[2] Kusaba, M., Liu, C., & Yoshida, R. (2022). Crystal structure prediction with machine learning-based element substitution. Computational Materials Science, 211, 111496.

[3] Gruver, N., Sriram, A., Madotto, A., Wilson, A. G., Zitnick, C. L., & Ulissi, Z. W. Fine-Tuned Language Models Generate Stable Inorganic Materials as Text. In The Twelfth International Conference on Learning Representations.

Dear Reviewer VEwm,

Thanks again for your insightful and thoughtful comments!

As the reviewer-author discussion period is closing soon (November 26 at 11:59 pm AoE), we would like to gently remind you that we are eagerly awaiting your feedback on our response.

We have added a comparison with DiffCSP++ and MatterGen-MP, an evaluation on uniqueness, novelty, and stability, and clarification of NFE Experiments.

If you have any further questions about our work, please do not hesitate to let us know.

Thank you once again and we genuinely look forward to hearing from you!

Best regards,

Authors

Dear Reviewer VEwm,

Thank you once again for your valuable comments and insights!

As the reviewer-author discussion period is nearing its end (December 2nd), we wanted to kindly remind you that we are eagerly awaiting your feedback on our response to your comments.

In our response, we have provided a detailed rebuttal towards the concerns you raised, including the addition of comparison with related works, uniqueness, novelty, and stability metrics.

If you have any further questions or require clarification on any aspect of our work, please do not hesitate to let us know.

Thank you for your time and effort, and we truly look forward to hearing from you!

Best regards,

Authors

Thanks very much for your response. Most of my concerns are addressed and I have updated my score.