Symmetry-Aware GFlowNets

Thank you for your helpful feedback and for recognizing our work as a significant contribution to the community!

---

Time Comparison with the Positional Embedding (PE)

Since PE must be computed at every transition, its computational cost scales poorly with the trajectory horizon. To evaluate how each method scales with the number of transitions, we sampled random graphs with varying transition requirement. We measured the time spent on only the major components of each method (e.g., computing automorphisms for Reward Scaling). 8-dimensional PEs were computed for the PE method. The table below shows the additional computational cost incurred by each method.

Plot link: https://drive.google.com/file/d/1hhB0TVyHKmclFJ65O__BlX74fRUvNtrS/view?usp=sharing

The experiments clearly show that our methods (Orbit Correction, Reward Scaling) scales much better by significant margins. Note that the reported time is required for each trajectory and accumulates over the course of training.

This indicates that exponential complexity of our method is not the current bottleneck in GFlowNets. We can compute the size of automorphisms in just a few milliseconds, even for graphs with thousands of nodes—far beyond the scale of current applications, which involve fewer than a hundred nodes.

---

Clarification of the Theorem 4.4

Your understanding of the theorem is correct: the flow constraints must be satisfied for all actions and states. Thank you for giving us the opportunity to clarify this, and we will further revise the manuscript in the updated version.

---

On Atom-based Generation Task

Your conjecture suggests that symmetries are uncommon in large, real-world molecular graphs, explaining the limited improvement in the atom-based task. We observed that the average symmetries in the synthetic environment is 2.6, whereas molecules in the QM9 training dataset have an average symmetry of 1.3. As you pointed out, this partly explains the limited improvement in the atom-based task. However, a few important points are worth noting:

• Our method does not inherently guarantee improved sample diversity, as this depends on the reward landscape.
• The average symmetry of molecules in the ZINC250k dataset is 2.44, despite containing larger molecules than QM9 (38 atoms vs. 9 atoms). Additionally, the Spearman correlation between the number of atoms and symmetry is 0.08 in ZINC250k, indicating that real-world molecular graphs do not necessarily exhibit fewer symmetries as they grow larger.
• For larger graphs, where only fragment-based generation is practical, what matters is the symmetry of fragments, as demonstrated in our paper.

---

On the Performance of Reward Scaling in Figure 3(c)

You are correct in pointing out that reward scaling only corrects the terminal probabilities, while the intermediate probabilities remain incorrect. In contrast, both transition and orbit correction methods utilize the correct intermediate probabilities. This explains the performance gap between Reward Scaling and Orbit Correction with detailed balance, as it depends on intermediate probabilities. As mentioned in the paper, this provides additional signal, leading to faster convergence.

---

On Outliers in Figure 3(a)

We are happy to elaborate on this. It turns out that the previous experiment stopped training too early. We resumed training from the last checkpoint with a larger batch size and a lower learning rate, obtaining an almost straight line!

See the plot in link: https://drive.google.com/file/d/1IHdLn5Ht4zlZ_frqeEOk31X-F-rrIKzC/view?usp=sharing

In addition, outliers in the Fig. 3(a) corresponds to to graphs that are challenging to represent using GNNs; compare Figure 3(a) with Figure 10. As mentioned in the paper, the expressive power of the GNN is another source of incorrect learning, as it can give the same representation for actions in different orbits.

Your suggestion to use the off-diagonal elements of the random walk matrix is insightful. However, in this simple environment, shortest-path lengths provide sufficient information to distinguish different actions. The main limitation was that the information is processed by a simple MLP with only one hidden layer. We observed similar learning dynamics with the random walk matrix. While other approaches could address this issue, the current setup already serves its illustrative purpose.

We hope this clarifies our points and addresses your concerns.

We sincerely appreciate your thoughtful feedback and your recognition of our work's potential impact in this area!

---

Impact of Incorrect Automorphism Counting

While we are uncertain about the conditions under which the bliss algorithm might fail, we believe it produces exact outputs given the current scale of GFlowNet applications (with fewer than 100 nodes). Moreover, reliance on bliss is not essential, as popular alternatives such as nauty are available.

That said, incorrect automorphism calculations can result in slow convergence or an inaccurate policy, with the exact impact depending on the reward landscape. We consider two specific cases separately.

Random Counting: We consider a scenario where the software randomly miscounts the size of the automorphism group in 10% of cases—doubling it 5% of the time and halving it 5% of the time. Since the GFlowNet objectives are constructed in the log space, this amounts to the random noise of $\log 2$, which incurs no bias.

Systematic Miscounting: We implemented an approximate automorphism counter using the Weisfeiler-Lehman (WL) algorithm. If the final coloring has $k$ distinct color classes, and class $i$ has size $s_i$, then an approximation is computed as $\prod_{i=1}^k s_i$. This assumes that nodes within each class can be permuted freely, which is an upper bound of $|\mathrm{Aut}(G)|$.

Experimental Results

We conducted a simple experiment in relatively small environment ($|\mathcal X| = 301$) using the TB objective with Reward Scaling. As shown in the plot linked below, Random Counting converges to the exact method but does so slowly. The approximation using the WL algorithm proved too crude for this environment. A more accurate approximation would better align with our method.

https://drive.google.com/file/d/12m4uqiZdPxcpsaUYowD7ZWUh3wYCGz99/view?usp=sharing

Additionally, we implemented the PE method of Ma et al. (2024), which gives an approximate solution to the problem. The experiments were conducted on two synthetic environments: Cliques and Cycles. While it underperformed compared to ours, it proved to be far better than the Vanilla GFlowNets, which highlights the importance of removing bias.

We obtained similar results in the fragment-based generation task (reported in page 8), where we proposed an approximate solution.

Overall, miscalculating automorphisms can degrade performance. However, as long as we can approximate them well, we should always apply the necessary corrections to eliminate bias.

---

Scalability Comparison

Our method scale far better than other correction methods. We evaluated each method, varying the number of transitions per trajectory. We sampled 100 random graphs for each horizon length category. We measured the time spent on only the major components of each method (e.g., computing automorphisms for Reward Scaling). The table below shows the additional computational cost (compared to Vanilla GFlowNets) incurred by each method per trajectory.

Plot link: https://drive.google.com/file/d/1hhB0TVyHKmclFJ65O__BlX74fRUvNtrS/view?usp=sharing

While the cost increases for all methods as the number of transitions grows, our method clearly scales better. It is important to note that the time differences accumulate over the entire training duration. Our implementation of positional encoding is adapted from torch_geometric.transforms.AddRandomWalkPE.

---

Training Time Comparison

We report the total training time for each method, including positional encoding (PE) method of Ma et al. (2024). The results for each training step are summarized in the table below.

While PE is faster than Transition Correction (which performs several isomorphism tests), our methods (Reward Scaling, Orbit) are faster. We used one processor with one GPU (24GB TITAN RTX GPU, Intel Xeon Silver 4216 CPU).

Overall, our method has lower computational cost compared to other methods and scales better.

We appreciate your thoughtful feedback, and recognizing our work as important.

---

Comparison with Ma et al. (2024)

Implementation of Positional Encoding (PE) method

To better position our work, we implemented the PE method to compare with Ma et al. (2024) and made our best effort to reproduce their approach. We computed 8-dimensional PEs for each node, with edge-level PEs obtained by summing the corresponding node-level PEs. Finally, actions sampled from the policy were compared with other forward/backward actions based on their PEs to identify equivalent actions.

Performance Comparison

Since PE is an approximate solution, it underperforms compared to our method in terms of $L_1$ error. We validated this in two synthetic environments: the Cliques and Cycles environment. The table below presents the results for the Trajectory Balance objective.

For the Detailed Balance objective, however, it is difficult to predict whether Reward Scaling will outperform, as the PE method corrects intermediate probabilities similarly to Orbit Correction. In our experiments, PE performed on par with Orbit Correction.

However, the PE method, as proposed in Ma et al. (2024), requires task-specific tuning and is not applicable to fragment-based generation. Therefore, we do not recommend using PE. Orbit Correction offers clear advantages over PE: exact correction, faster computation, broader applicability, and ease of implementation.

Scalability Comparison

Since PE must be computed at every transition, its computational cost scales poorly with the trajectory horizon. We evaluated each method, varying the number of transitions per trajectory. We sampled 100 random graphs for each horizon length category. The table below shows the additional computational cost (compared to Vanilla GFlowNets) incurred by each method per trajectory.

Plot link: https://drive.google.com/file/d/1hhB0TVyHKmclFJ65O__BlX74fRUvNtrS/view?usp=sharing

While the cost increases for all methods as the number of transitions grows, our method clearly scales better. It is important to note that the time differences accumulate over the entire training duration.

Training Time Comparison

While PE is faster than Transition Correction (which performs several isomorphism tests), our methods (Orbit Correction, Reward Scaling) are faster. We present the actual running time of each method on the synthetic environment presented in the paper.

We used one processor with one GPU (24GB TITAN RTX GPU, Intel Xeon Silver 4216 CPU).

---

Measure of Goodness-of-Fit

We emphasize that the purpose of the molecule generation task is to assess whether the correction methods can improve the generation of high-reward samples in realistic reward settings, rather than to demonstrate the correctness of our method. Therefore, we included the correlation plot only for fragment-based generation (Fig. 4). We obtained improved results with correction for atom-based generation as well, which will be incorporated into the revised version.

We used correlation as a metric because it is a widely adopted measure of goodness-of-fit in several previous works (Malkin et al., 2022; Malkin et al., 2023; Madan et al., 2023). While we considered your suggestion to use the recently proposed FCS score (Silva et al., 2025), we concluded that further investigation is needed to determine the best practice for its application.

---

On Notations

While we summarized our notations in Appendix A, we acknowledge that some of our notation can be confusing. We are carefully reviewing our notation and theoretical explanations to make the paper more readable in the revised version.

We hope this addresses your concerns regarding the weaknesses of our paper.

We sincerely appreciate your thoughtful comments and valuable feedback. In particular, we sincerely hope that our response helps evaluation recognizing our contributions, especially in providing a rigorous theoretical foundation and conducting extensive experiments on the problem.

---

Clarification on the Differences from Ma et al. (2024)

Our proposed approach is methodologically different from that of Ma et al. (2024). Their method explicitly identifies actions that lead to the same next state to compute exact transition probabilities, using positional encoding as an approximate way to find such actions. In contrast, our approach is based on Theorem 4.6 and does not rely on identifying equivalent actions. This brings clear operational advantages over the approach of Ma et al. (2024).

Exactness: While our method provides an exact bias correction, the PE method can only approximate the target distribution.

Faster Computation: Our method is orders of magnitude faster than the PE method, as PE should be computed for every transition (see Appendix I and the comparison below).

Broader Applicability: Our method is applicable to any graph type and can be extended to fragment-based generation with minimal modifications. In contrast, the PE method as proposed in Ma et al. (2024) cannot be applied to graphs with edge types or fragment-based generation.

Straightforward Implementation: Our method requires only scaling the final reward, making it straightforward to implement. In contrast, the PE method must identify both forward and backward equivalent actions and sum their probabilities, which may require substantial modifications to existing codebases.

One of our key theoretical findings is that graph automorphism is central to this problem, and in this sense, all solutions may appear fundamentally similar in hindsight. However, our approach offers clear theoretical and practical advantages, providing exact bias correction while maintaining broad applicability across different graph types. We further validate these arguments in the following comments.

---

Experimental Comparison with Ma et al. (2024)

To better position our work, we implemented the PE method to compare with Ma et al. (2024) and made our best effort to reproduce their approach. Implementation details, as well as plots for additional experiments, can be viewed through the following link: https://docs.google.com/document/d/1GhQgcaxQ32uXM7s6xfteGWsBovdm-JvQNW_m3hAG30Q/edit?usp=sharing

Performance Comparison

Since PE is an approximate solution, it underperforms compared to ours in terms of $L_1$ error. We validated this in two synthetic environments: Cliques and Cycles. The results are presented using the Trajectory Balance objective, while the results for the Detailed Balance objective are provided in the linked document.

Scalability Comparison

Since PE must be computed at every transition, its computational cost scales poorly with the trajectory horizon. We evaluated each method, varying the number of transitions per trajectory. We sampled 100 random graphs for each horizon length category. We measured the time spent on only the major components of each method (e.g., computing automorphisms for Reward Scaling). The table below shows the additional computational cost incurred by each method per trajectory.

While the cost increases for all methods as the number of transitions grows, our method clearly scales better. It is important to note that the time differences accumulate over the entire training duration.

Training Time Comparison

We measured the actual training time with the same setting used for our paper.

Our method is faster than the PE method in terms of actual training time.

---

Appendix C: We appreciate the feedback. We will revise it in the revision.

---

Conclusion

Overall, our method offers clear advantages over Ma et al. (2024), which are validated through additional experiments. This demonstrates the novelty of our solution. We compared our method with Ma et al. (2024) to better position our work, and we will incorporate these results into the revised version.