Generating GFlowNets as You Wish with Diffusion Process

Dear Reviewer h5Ma, Thank you very much for your feedback. We greatly appreciate your comments and have learned a lot from them.

Reviewer 4

Weakness

W1: The benefits of parameter generation compared to defining a forward policy conditioned on environmental information.

The core motivation behind our approach lies in the flexibility and efficiency of parameter generation. While defining a forward policy is feasible, it can become computationally expensive in scenarios with high-dimensional state spaces and complex environmental structures. Our method leverages parameterized probabilistic samplers (as discussed in Section 3.2) to generalize across environments effectively. We explain the advantage of our method over the mentioned as follows:

• Reduces the computational burden by decoupling policy training from environment-specific configurations.
• Provides adaptability to unseen environments by learning transferable parameters, which is critical for applications such as molecular design and structural optimization.
• Builds upon GFlowNet principles by enabling scalable and structured exploration across diverse environments, as outlined in Bengio et al. (2021).

By adopting parameter generation, we focus on generalization, a significant advantage when scaling to environments with varying dimensions and configurations.

W2: The reviewer suggests that incorporating more complex components of GFlowNets, such as diverse reward functions and action spaces, may present challenges.

Our method currently focuses on demonstrating the feasibility of parameter generation within a well-defined scope (e.g., hyper-grid environments). The generalization can be a proof of the different reward functions, especially in different tasks:

• Reward Functions: The GFlowNet formulation in our approach can accommodate task-specific reward functions (e.g., fragment-based, reaction-based). For instance, in molecular design, we highlight the use of tailored conditional embeddings in Section 3.4 to adapt to specific reward structures.
• Action Spaces: The latent diffusion model utilized for parameter generation (Section 2.3) supports diverse action representations. For example, actions in molecular synthesis (e.g., fragment addition) can be encoded within the same probabilistic framework.

We acknowledge that scaling to more complex spaces, such as hierarchical or dynamic state-action pairs, is an avenue for future work, as noted in the discussion. Thank you for your valuable suggestion again.

W3: Experiments: the experiments are limited to relatively simple tasks (e.g., hyper-grids and structural variations) and request more advanced use cases.

Thank you for your suggestions. We agree that more complex tasks can provide additional insights. However, the chosen synthetic tasks allow us to:

• Precisely evaluate generalization across structural variations in a controlled setting, as illustrated in Table 2 and Figure 1.
• Validate the adaptability of our method across distinct environments, which is critical for parameterized probabilistic samplers.
• Demonstrate the robustness of GFlowNet-inspired sampling in diverse configurations without confounding factors from real-world noise.

Additionally, we included a molecular generation task (Section 3.4) to illustrate real-world applicability. We aim to expand on this in future iterations by incorporating tasks such as RNA sequence generation or combinatorial optimization. I in our experiment means the iteration number in a training process, we updated this in our revision.

W4: The performance improvements in Tables 4 and 5 appear minor and requests clarification regarding the similarity in generalization between 𝑁=2 and 𝑁=5.

Thank you for your detailed observation. The performance improvements reported in Tables 4 and 5 reflect the consistent generalization capability of our approach across environments. The similarity in results for 𝑁=2 and 𝑁=5 arises from the intrinsic efficiency of the parameter generation framework, which effectively captures shared patterns in the GFlowNet structures (Section 3.3). This highlights the scalability of the method to different environmental dimensions.

Questions

Q1: What do I mean in the experiment?

I in our experiment means the iteration number in a training process, we updated this in our revision.

Reviewer 3

Dear Reviewer JPtD,

Thank you so much for your thoughtful feedback and the time you dedicated to reviewing our work. To save time and provide clarity, we summarize our responses below.

---

Weaknesses

W1: Experiments conducted solely on synthetic data and connection with Figure 1

The use of synthetic data was a deliberate choice to allow precise control over evaluating the core capabilities of our method. However, we also conducted experiments on molecular generation with real-world datasets and extended our analysis. For future revisions, we plan to add more real-world datasets to further enhance the work.

Regarding Figure 1:

• Efficiency: Figure 1 demonstrates GenFlowNet's time usage advantage, which is corroborated by the computational cost reductions shown in Tables 1 and 2.
• Accuracy and Low Divergence: Empirical L1 loss, JS divergence, and KL divergence in Tables 1 and 2 validate that the generated parameters closely match ground-truth probability distributions.
• Diversity: Diversity is showcased in Figure 5, highlighting the variability in parameter matrices that support downstream applications.
• Generalization: GenFlowNet's ability to generalize to unseen structures is validated in Table 2 and Figures 4c and 4d, illustrating its adaptability across tasks.

W2: Lack of clarity on GFlowNet training struggles with increasing trajectory length

We address this by evaluating performance across different trajectory lengths in:

• Hypergrid Task: Complexity increases with factors such as state dimension and size, leading to longer trajectories.
• Molecular Generation: This task involves deeper GFlowNet structures and more intricate challenges. Both examples demonstrate how increasing complexity naturally results in longer trajectories and highlight the method's robustness.

W3: Limited novelty

The novelty of our method lies in introducing parameter generation to the probabilistic sampler, offering a fresh perspective in this domain. Beyond using a latent diffusion model, we:

• Developed tailored conditional embeddings to align with task-specific needs.
• Validated the method’s generalization ability across multiple tasks, showing its effectiveness in a variety of settings.

---

Questions

Q1: Specific distinctions from existing parameter generation methods

Our approach has several key distinctions tailored to GFlowNet settings:

• Structured State-Action Spaces: GFlowNets require parameterization for trajectory-based generation processes while satisfying flow-matching constraints, which traditional generative models are not designed for.
• Environmental Variability and Generalization: Addressing challenges highlighted in prior work (Bengio et al., 2021; Jain et al., 2022), our method uses task-specific embeddings to dynamically adapt to varying environments efficiently.
• Scalability: By incorporating tailored conditional embeddings, we align parameterization with task geometry, making our method scalable for high-dimensional inputs, as supported by recent studies (Peebles et al., 2022; Erkoc ̧ et al., 2023).

Q2: Dataset preparation for the AE component

We clarify the details of dataset preparation:

• Number of Models: 200 models were used for AE training.
• Number of Samples: Each GFlowNet structure contains 200 models, resulting in 800 samples across 4 structures.
• Dataset: The hypergrid task was used to train and evaluate the AE component. Further details are included in the appendix (lines 789–794).

Q3: Inclusion of real-world datasets

While this work includes results on molecular generation tasks (Section 4.4), we aim to expand experiments in future iterations to cover more real-world applications such as RNA sequence generation and combinatorial optimization.

---

We appreciate your valuable feedback and are happy to address any further questions. Your support helps improve our work, and we thank you again for your time and effort.

Best regards,
Authors

Dear Reviewer Wwiu, Thanks so much again for the time and effort in our work. Considering the limited time available and to save the reviewer's time, we summarize our responses here.

W1: Scalability may be limited

Thank you for your comment. We address scalability in the following aspects:

• Scalability in Current Design: GenFlowNet’s scalability is demonstrated through diverse tasks, such as generalizing across unseen GFlowNet structures (Section 3.3, hypergrid tasks) and tasks (e.g., molecule generation in Section 3.4). By adopting a diffusion-based parameter generation approach, GenFlowNet circumvents the iterative training process of conventional GFlowNets, which scales poorly for large datasets. Empirical results in Table 9 further validate its efficiency in adapting to increasing trajectory lengths and state dimensions.
• Future Directions for Scalability: We acknowledge that further enhancements in scalability are necessary for larger datasets and more complex structures. Future work will incorporate distributed and parallel inference mechanisms into the framework to support real-world, large-scale applications.

---

W2: Necessity of our method

Thank you for raising this point. The necessity of GenFlowNet is underpinned by the challenges of existing GFlowNet training paradigms and the advantages provided by our approach:

1. Challenges in Traditional GFlowNet Training:

   • High Computational Costs: Iterative training is resource-intensive, especially for tasks with long trajectories or high-dimensional states (Bengio et al., 2021; Malkin et al., 2022).
   • Scalability Issues: Training becomes prohibitive with larger state spaces or complex reward functions (Deleu et al., 2022).

2. Benefits of GenFlowNet:

   • Efficient Parameter Generation: GenFlowNet generates ready-to-use parameters, reducing computational overhead (Section 3.2, Figure 4).
   • Adaptability Across Tasks: Supports rapid adaptation to diverse GFlowNet structures and tasks (e.g., hypergrid tasks and molecule generation).
   • Time Efficiency: Tables 1, 3, and 9 highlight significant reductions in computational time without compromising accuracy, making it ideal for tasks requiring rapid iteration.

GenFlowNet bridges the gap between theoretical advancements in GFlowNet sampling and practical application by offering a computationally efficient, training-free alternative. This is especially valuable in fields like molecular discovery and combinatorial optimization, where rapid prototyping is critical.

---

W3: Lack of real-world performance

Thank you for your concern. While our experiments are primarily synthetic, we address real-world applicability in two ways:

1. Simulating Real-World Conditions: The hypergrid task is parameterized to introduce diverse state dimensions and trajectory lengths (Section 3.3), mimicking real-world complexities.
2. Application to Real-World Data: The molecule generation task leverages real-world chemical datasets, demonstrating the framework’s potential in practical domains.

---

Q1: Measure uncertainty from a Bayesian perspective?

Thank you for this insightful question. GenFlowNet shares similarities with hypernetworks, which output model weights conditional on input features. This structured generative process facilitates uncertainty quantification in domains such as AutoML and model exploration.

• Bayesian Context: GenFlowNet introduces stochasticity in its parameter generation process, enabling uncertainty measurement in both sampling distributions and model outputs. This is particularly relevant for tasks where model uncertainty is critical.
• Practical Utility: This stochastic modeling approach aligns with Bayesian principles, providing a robust framework for uncertainty estimation in complex tasks.

---

Q2: Clearer motivation

Thank you for the suggestion. The motivation for GenFlowNet lies in addressing critical inefficiencies of traditional GFlowNets:

1. Reducing Training Overhead: Eliminates iterative optimization, enabling deployment in time-sensitive tasks.
2. Expanding Applications: Facilitates GFlowNet use in domains like AutoML and molecular discovery, which require rapid prototyping.
3. Scalability and Efficiency: Retains accuracy in high-dimensional tasks while significantly reducing computational costs (Section 3.3, Figure 4).

---

Q3: Applicability to LLMs or text-to-image tasks?

While GenFlowNet demonstrates robustness for complex GFlowNet structures, extending its scalability to large-scale models like LLMs or text-to-image tasks may require additional adaptations. This remains an exciting direction for future research.

---

We sincerely appreciate the reviewers’ valuable feedback and will continue refining our work based on these insights. Thank you for your time and consideration.

Dear Reviewer iPE3, Thanks so much again for the time and effort in our work. Considering the limited time available and to save the reviewer's time, we summarize our responses here.

Weakness

W1: Fixed reward functions in training GenFlowNet

We appreciate this insightful comment. In the current paper, we use a fixed reward set $R_0$, $R_1$, $R_2$, $H$ across all GFlowNets to evaluate the consistency and efficiency of parameter generation. This design isolates the evaluation of the parameter generation component by eliminating noise from variable reward functions. However, we agree that generalization to unseen rewards is critical.

To address this, we will include an experiment in the revised manuscript where the auto-encoder is trained on varying reward functions to further validate the flexibility and generalizability of our method.

W2: Slim experimental suite compared to recent GFlowNet works

Thank you for highlighting this. Our experimental suite prioritizes a proof of concept by focusing on two challenging and diverse tasks:

• Structured Synthetic Data: Hypergrid tasks are valuable for testing trajectory balance and parameter optimization under controlled conditions.
• Real-World Applicability: Molecule generation tasks illustrate GenFlowNet's relevance for real-world applications, including drug discovery.

We plan to expand this suite to include additional tasks such as:

• Protein structure prediction
• Combinatorial optimization
• Large-scale multi-agent simulations

These extensions will demonstrate the scalability and versatility of GenFlowNet.

W3: Varying rewards and forward policies

Thank you for your suggestion.

Varying Rewards: Our experiments already evaluate varying rewards across tasks. However, exploring varying reward functions within the same task is beyond the current scope. We will address this in future work.

Non-MLP Policies: While we focus on MLP-based policies due to their prevalence in GFlowNet literature, we acknowledge the importance of exploring non-MLP architectures. Future work will consider convolutional and graph-based policy models to extend GenFlowNet's applicability to spatial and graph-structured tasks.

W4: Lack of error bars or standard deviation

Thank you for pointing this out. We conducted evaluations with 10 repetitions and reported the averages in the manuscript. Below are additional results with best, average, and median performance for the hypergrid task:

---

Questions

Q1: What is the unit for "time usage"?

The "Time usage" in Table 1 is measured in seconds. This clarification has been added to the updated manuscript.

Q2: Highlighting "superior performance" and uncertainty quantification

We acknowledge that the claim of "superior performance" is primarily based on time usage reduction, as shown in Table 1. While improvements in sampling accuracy (e.g., KL divergence and L1 loss) are modest, the significant efficiency gains in computational time justify emphasizing this aspect.

Uncertainty quantification is reflected in the best, average, and median results provided above. Additional results are included in the supplementary material (lines 855–857).

Q3: Initialization in experiments

In Figure 4, the parameters generated by GenFlowNet serve as initializations for GFlowNet models without fine-tuning. This training-free approach generates high-quality parameters, reducing the need for iterative training and enabling faster deployment.

Q4: Why does the training-free method outperform trained ones?

The training-free method benefits from the diverse and generalizable training dataset, which encompasses a wide range of GFlowNet structures. This enables GenFlowNet to learn representations that generalize well to unseen tasks. As a result, GenFlowNet-initialized models often start with better parameters, reducing the need for extensive optimization and achieving competitive or superior performance.

Q5: Details about Section 3.3 (unknown structures in hypergrid tasks)

In Section 3.3, GenFlowNet generates parameters for previously unseen structures in the hypergrid task. This demonstrates its ability to generalize across tasks and structures not encountered during training, showcasing its adaptability to diverse applications.