Graph Diffusion that can Insert and Delete

We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.

• Computational Overhead: GRIDDD's insertion and deletion operations introduce significant computational overhead. The paper lacks concrete optimizations (e.g., sparse graph updates, mixed-precision training) or quantification of additional computational costs (training time, memory usage, FLOPs). Have you explored optimization techniques (e.g., sparse updates, gradient checkpointing) to address computational overhead?

We have discussed the model's training time in the appendix (Section C.4), where we estimate that GrIDDD to be approx. 33% slower to train with respect to DiGress, denoting a typical tradeoff between expressivity/flexibility and computational overhead. Interestingly enough, we also observed that GrIDDD can, under some specific conditions, be more computationally efficient to sample than the alternatives. In particular, since our model is capable of starting the sampling process from a latent featuring any number of nodes, the reverse process can be much faster than DIGRESS if the initial latent is very small. As an example, GrIDDD takes approx. 42 seconds to sample 128 molecules on ZINC250k starting from latents with 2 nodes each, and approx. 64 when starting with 24 nodes (the most frequent size in the dataset). In contrast, DIGRESS, which needs to be initialized with a node matrix of size close to 36 to accommodate for the largest graph size in the dataset, requires approx. 75 seconds to run the same sampling task. As regards the number of parameters, GrIDDD uses a slightly larger but still comparable number with respect to FREEGRESS and DIGRESS (5M vs. 4.6M on QM9, 15.3 vs. 13.5 on ZINC250k). Overall, the tradeoff could be considered positive, since GrIDDD is more flexible than FREGRESS and DIGRESS and can be used for molecular optimization tasks. We commit to complement Section C.4 of the revised paper with these numbers and observations. As suggested by the reviewer, the slightly superior computational burden of GrIDDD could be mitigated by using appropriate optimization techniques. These would ultimately affirm GrIDDD as an comparable alternative to DIGRESS and FREEGRESS even in lower resource scenarios. However, we did not specifically employ any at this stage of development.

• Split Molecules: GRIDDD occasionally generates more split molecules due to late-stage node insertions. Further exploration to address this issue through additional constraints or algorithmic modifications is needed. Could you elaborate on the causes behind split molecule generation? Have you considered additional constraints to mitigate this?

Earlier in the development of GrIDDD, we analyzed where, in the reverse process, split molecules appeared, plotting the number of occurrences as a function of the denoising timestep. Overall, the statistics perfectly matched the shape of the $\zeta’(t)$ function which modulates the deletion schedule. As a result, we conjecture that split molecules are mostly caused by nodes inserted too late during the reverse process, which cannot be re-attached in time to the main graph, as the noise scheduler does not allow for too many changes in the graph’s structure during the last phases of the process. As regards implementing additional constraints, during our preliminary tests, we have found that GrIDDD generates less split molecules when given, as extra features, different powers of the adjacency matrix. This can be interpreted as a sufficiently accurate approximation of the number of connected components in the graph (as opposed to the computationally demanding task of computing the eigenvalues of the adjacency matrix's Laplacian). However, we have ultimately decided to proceed without this feature to ensure a fair comparison with the baselines. We will clarify these additional aspects regarding the issue of split molecules in the revised version of the paper.

• Limited Generalization to Larger Graphs: GRIDDD’s effectiveness decreases significantly when generating molecules larger than its training distribution. Enhanced methods for improving generalization capabilities to larger graph sizes would be valuable. How might GRIDDD be adapted to effectively handle larger molecules beyond current limitations?

First of all, we would like to state that according to our results, GrIDDD already takes a convincing step forward towards a more reliable out-of-distribution sampling with respect to existing approaches, as shown in Section 5.2 of the paper (which shows an improved capacity of generating valid molecules out-of-distribution with respect to the baselines). Clearly, we acknowledge that the problem is not completely solved and more research is needed, as generative models are trained to imitate the dataset distribution and not something they did not see during training. One possible direction to take in this regard is to parameterize both insertions and deletions (i.e., letting the neural network decide which kind of node to add, rather than relying on sampling from the marginal distribution of the nodetypes). In this sense, GrIDDD presents excellent opportunities to be extended and improved, and we plan to address the out-of-distribution sampling problem with innovative approaches in future studies. We will improve the conclusions section with this discussion.

• Impact of Hyperparameters: Could you provide sensitivity analyses regarding hyperparameters such as noise schedules and insertion/deletion probability distributions?

We have performed some experiments, tweaking the values of $p_{min}$ (we observed how $p_{max}$ can be kept equal to one, and that editing $p_{min}$ is enough), $D$, $w$, and the $\lambda$ associated with the insertion time's loss. We have trained several GrIDDD variants on QM9 while monitoring the behaviour of the training and validation losses, sampling 100 molecules from the trained models. The results are summarized in the table below. Overall, we observed that setting $D$ too small (which corresponds to starting the operations of insertion and deletion too early in the forward process) causes the model to generate higher rates of split molecules, since the model has little time to connect them back into the main graph while denoising. For values of $D$ higher than 0.5, we did not observe meaningful differences. We observed a similar behavior when setting large $w$ values, which was expected since this makes the "slope" of the $\zeta$ function less steep and, as a result, allows late reinsertions during the reverse process. Finally, we find the model to be relatively robust to different values of $p_{min}$, as well as the $\lambda$ associated with the insertion time loss (unless it is too high).

• val: validity (higher=better)
• avg. nc: average number of isolated connected components (lower=better)
• nc max: maximum number of isolated connected components (lower=better)
• n conn: number of valid connected graphs generated (higher=better)

We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.

• Assumption of Mutual Independence: The reverse process factorizes probabilities assuming mutual independence of nodes and edges, which might be a simplification given the highly interconnected nature of molecules.

The reviewer is correct in principle. However, all discrete graph diffusion models make the same assumption of mutual independence. As far as we know, there are no discrete diffusion models for graphs that do not make such assumption, as it significantly simplifies the underlying math. At the same time, we point out that the neural network tasked to compute the posterior $q(x_i^0|G^t)$ does not make use of that assumption, as it receives the entire graph in input to predict the categories of every node (and edge) in the graph. To summarize, we agree with the reviewer that this is an important limitation, although it is shared by all known discrete denoising diffusion models for graphs. We will extend the limitations section accordingly.

• Failure Analysis: For cases where GRIDDD shows slightly lower validity (e.g., QM9, $\mu$ target), what are the common types of invalid molecules generated? Understanding these failure modes could guide future improvements.

One very interesting insight we have gathered is that, unlike FreeGress, the experiments targeting $\mu$ tend to fail most frequently when targeting μs close to zero. The few molecules generated tend to be small, split graphs. We have investigated the phenomenon and found out that the molecule with SMILES string "CC" (ethane) is known to have a dipole moment equal to zero. We hypothesize that GrIDDD tries to generate ethane or similar small molecules to minimize the dipole moment, with higher failure rates than usual since these small molecules are under-represented in the data. Remarkably, DiGress and FreeGress are unable to pursue this minimization, since they almost inevitably start the reverse process from latents with a larger graph size and cannot adapt it to produce smaller molecules later on. Overall, this is an interesting behaviour which shows that GrIDDD successfully can learn the properties of molecules with infrequent sizes as well. We will extend the revised version to include a discussion of a failure analysis.

• Sensitivity to Hyperparameters: How sensitive is GRIDDD's performance to the choice of hyperparameters, particularly $p_{min}$, $p_{max}$, $D$, $w$, and the $\lambda$ weights in the loss function? Are there general guidelines for tuning these for new datasets?

We have performed some experiments, tweaking the values of $p_{min}$ (we observed how $p_{max}$ can be kept equal to one, and that editing $p_{min}$ is enough), $D$, $w$, and the $\lambda$ associated with the insertion time's loss. We have trained several GrIDDD variants on QM9 while monitoring the behaviour of the training and validation losses, sampling 100 molecules from the trained models. The results are summarized in the table below. Overall, we observed that setting $D$ too small (which corresponds to starting the operations of insertion and deletion too early in the forward process) causes the model to generate higher rates of split molecules, since the model has little time to connect them back into the main graph while denoising. For values of $D$ higher than 0.5, we did not observe meaningful differences. We observed a similar behavior when setting large $w$ values, which was expected since this makes the "slope" of the $\zeta$ function less steep and, as a result, allows late reinsertions during the reverse process. Finally, we find the model to be relatively robust to different values of $p_{min}$, as well as the $\lambda$ associated with the insertion time loss (unless it is too high).

• val: validity (higher=better)
• avg. nc: average number of isolated connected components (lower=better)
• nc max: maximum number of isolated connected components (lower=better)
• n conn: number of valid connected graphs generated (higher=better)

Greetings,

As we are on the last day before the end of the discussion period, we kindly request the reviewer to provide us with feedback on our reply.

We extend our gratitude for understanding.

Regards

We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.

• Empirical performance for property targeting is not remarkable

We strongly believe that our model specifically addresses a known limitation of all current discrete denoising diffusion models for graphs (including FREEGRESS) in a principled and elegant way. To our knowledge, this is the first method that can learn the graph size and adapt it to the generative task without resorting to estimates, heuristics, or auxiliary models. At the same time, it does so while performing on par or better than the existing baselines, meaning that its capability to reach and even surpass the same performance is actually a signal of better generalization capabilities than the former method. Moreover, our method is more widely applicable than both FREEGRESS and DIGRESS (which lack the ability to adapt the graph size needed for optimization tasks).

• The paper does not compare to graph continuous diffusion models

We thank the reviewer for the reference. From our understanding, the main difference between jump diffusion and our solution is that the former can only reduce the number of entries in the data point during the forward process (in the case of molecules, these would be the atoms), while our model can also increase them. This implies that graph continuous diffusion cannot be adapted for property optimization tasks, since it lacks the capability of deleting additional nodes during the reverse process besides the ones deleted during the forward process. Overall, jump diffusion is better suited to be applied to continuous data such as videos and, more in general, to data with positional dependencies. Nonetheless, we will certainly add the reference to the revised paper and extend the related works section to discuss the differences between graph continuous diffusion models and GrIDDD.

• An analysis of the computational overhead is required

We have discussed the overall estimated overhead in the appendix, in section C.4. Clearly, it is unavoidable that a model that does more than a standard discrete diffusion will feature some sort of overhead. However, we take the opportunity to point out that GrIDDD can, under some specific conditions, be faster than the alternatives. In particular, since our model is capable of starting the sampling process from a latent featuring any number of nodes, the reverse process can be much faster than DIGRESS if the initial latent is very small. As an example, GrIDDD takes approx. 42 seconds to sample 128 molecules on ZINC250k starting from latents with 2 nodes each, and approx. 64 when starting with 24 nodes (the most frequent size in the dataset). In contrast, DIGRESS, which needs to be initialized with a node matrix of size close to 36 to accommodate for the largest graph size in the dataset, requires approx. 75 seconds to run the same sampling task.

• Concerns regarding the pertinence of GrIDDD for property targeting

We believe our solution is also well suited for property targeting. Clearly, the distinctive trait of GrIDDD is that it can flexibly adapt the graph size during the forward and reverse processes. Therefore, its best usage is to target properties that directly relate to graph size. However, in drug design, several properties (such as molecular weight, polar surface, and synthetic accessibility, and, to a minor extent, solubility) significantly correlate with the number of atoms. At the same time, although GrIDDD has been applied only on molecular datasets due to the abundance of available baselines and datasets, it can target or optimize arbitrary graphs, meaning that it can be employed for a variety of tasks beyond molecular generation.

• Formatting of Table 1 and Figure 3

We thank the reviewer for spotting these mistakes. We will fix them in the revised version of the paper.

• Discussion on hyperparameter selection

We thank the reviewer for the opportunity to clarify this point further. Overall, the choice of the hyperparameters was the result of a series of preliminary tests. We have chosen the values for $w$ as 0.05 and $D$ as 0.5 because they gave us some of the best results in terms of validity and number of connected components using a validation set. Setting $D$ any lower causes the model to generate many split molecules. The $\lambda$ associated with the insertion time of the nodes does not seem to affect training much, although setting it higher than 0.5 causes the model to underperform. Same thing for $p_{min}$ and $p_{max}$ (although it seems like setting $p_{min}$ between 0.2 and 0.4 bears slightly better results). We will add this discussion in the revised version of the paper to improve clarity and reproducibility. We believe this answer should resolve the reviewer’s concern; however, we remain open to provide the reviewer with more in-depth comparative results if needed.

• Possible discontinuities in the edge diffusion processes. Any relevant observations?

Removing edges during the forward process is not an issue, as each node and edge is corrupted independently without considering the rest of the graph. We have no relevant observations in this merit; we remain, however, curious to read whether the reviewer has any hypothesis in this sense that we could test.

• Clarification on the DEL* node type (l. 150-152)

We apologize for the lack of clarity, and we will correct the sentence in the final version of our paper. Specifically, we were referring to the reverse process. During the reverse step, in a setup with only one DEL type, a node with such a category can either switch to a "proper" class (the expected behavior) or remain a DEL. As a result, we defined the DEL* type, which is assigned to nodes marked for deletion. With this addition, we can guarantee that the nodes re-inserted during the reverse process will immediately switch back to a proper class, as the transition matrices do not allow for switching from DEL to DEL* during the forward process (and, hence, from DEL* to DEL while denoising, or DEL* to DEL* for that matter).

• Motivations behind sampling from the marginal distribution of nodes and edges

The reviewer is correct, this is the exact motivation behind this design choice. During our preliminary studies, we tried different options for completeness, such as always inserting the least frequent node/edge class. However, using the sample’s marginal distribution consistently gave us better performance, aligning to our expectations. We will update the revised text to include this motivation to better clarify our intent.

• Comparison against DIGRESS baselines with DEL/DEL* node/edge types

If we understand the reviewer’s intent correctly, we argue that it is not possible to treat the edges connected to non-existing nodes exactly as we treat the ones connected to DEL* nodes. In fact, GrIDDD removes edges altogether while the suggested variant has fixed graph size, and the edges would have to switch back and forth between DEL and “proper” types. As a result, DEL nodes could appear in the final latent $X^T$, and the associated edges would have to be set as DEL as well. This, in turn, would imply that the noisy graph $X^T$ is not sampled from an actual stationary distribution. GrIDDD solves this problem with a forward process that does not allow the presence of DEL nodes at the final corruption timestep $T$. Another minor issue would be caused by mismatches between the node and edge types (due to, for example, an edge to which non-DEL nodes are connected switching independently to DEL). A naive solution to these shortcomings would be to simply lift the constraint proposed by the reviewer everywhere but at timestep 0. By doing so, we can train the model to predict, at time step zero, that the edges associated to a DEL node have to be set to a specific category (be it a DEL or a “proper” edge class). We have run a comparison experiment to answer the reviewer’s inquiry, by implementing two baseline variants on QM9. One is essentially DIGRESS with an additional DEL atom type, while the other adds a further edge type EDEL to mark any bond whose endpoint is of type DEL. The results of the comparison are shown below.

• val: validity (higher=better)
• avg. nc: average number of isolated connected components (lower=better)
• nc max: maximum number of isolated connected components (lower=better)
• n conn: number of valid connected graphs generated (higher=better)
• TV: total variation between generated distribution and the dataset’s (lower=better)
• CE: Cross-Entropy (lower=better)

As can be seen, both baselines perform poorly when compared to GrIDDD, especially in terms of connected components, indicating that the model has difficulties in generating molecules with less than nine nodes. The validity rates appear higher, but a close inspection of the generated molecules shows that they are inflated by the presence of isolated carbon atoms which were not correctly attached to the main component.

Please engage in the discussion with the authors. The discussion period will end in a few days.

Thank you for your response.

Discontinuity of the edge diffusion process: My concerns here are addressed. I believe the discussion included in the rebuttal, as well as the proposal to investigate smoother edge diffusion as future work, should be incorporated into the final version of the paper.

DEL* node type: My concern on this point still remains. The diffusion model should be, by design, capable of learning which nodes should remain in the DEL state at $t = 0$. While I understand the motivation behind introducing the DEL* token to enforce monotonicity in the reverse process, this design choice introduces two main limitations from my perspective: it complicates the formulation, and it restricts the potential for remasking/self-correction, which has been shown beneficial on masked diffusion models (as GrIDDD can be seen, with DEL as the Mask state). That said, I do not view either of these limitations as critical for the paper’s core message. In fact, they point toward promising directions for future work.

As an additional outcome of the rebuttal process, I would also recommend that the final version of the paper includes: a discussion on continuous diffusion models and dimension-varying approaches, a clear explanation of the sampling time overhead, improved formatting of Table 1 and Figure 3, justification of the newly introduced hyperparameters and their selection procedure, clearer motivation for sampling from the marginal distribution of nodes and edges, and comparison with DiGress baselines that include DEL / DEL* node and edge types.

Overall, I am positive about the paper and will raise my score. However, the extent of this improvement is limited by the number of revisions that are still necessary.

A particularly important point that shifted positively my perspective was the discussion around the broader utility of the method beyond property optimization. I now see its potential for property targetting as well, especially for properties that depend on the size of the graph, not just in molecular domains but in graph settings more generally. The flexibility of dynamically adjusting the number of nodes during the denoising process is indeed valuable. The results in Tables 1 and 2 support this view: while GrIDDD does not outperform existing methods for tasks where property dependence on graph size is minimal, it excels in cases where that dependency is more direct. This intuition is hinted in the text, but I would encourage the authors to elaborate further and make it more explicit in the final version.

We thank the reviewer for the insightful review. Below, we respond to the reviewer’s concerns in order.

• First of all, the limitation that graph size has to be specified beforehand is not as critical as it seems. One can simply start from multiple graph sizes and sample all of them, deciding for the best graph afterwards. This is more time consuming than changing graph size during denoising, though. Another possibility is to learn a predictor that estimates an optimal graph size for the given problem. This is done by FREEGRESS and the evaluation shows that the GRIDDD method performs more or less the same for most problems.

The reviewer is correct to point out that existing methods resort to pre-hoc (auxiliary models, estimation) or post-hoc (multiple sampling) heuristics to accommodate for different graph sizes. In contrast, GrIDDD is an ad-hoc strategy that directly learns the graph size according to the conditional generative task, while being as effective as, or better, than the baselines. This also brings advantages in terms of its broader applicability: in fact, while GrIDDD can be used for molecular optimization, FREEGRESS cannot (since it cannot adapt the graph size dynamically to optimize the property).

• Only for molecular weight, the GRIDDD method outperforms FREEGRESS clearly. However, this is a property that completely depends on graph size and might be somewhat arbitrary and, in practice, rather uninteresting.

We agree with the reviewer that molecular weight directly depends on graph size. We, however, respectfully disagree on the fact that it is not critical, somewhat arbitrary, and not interesting. A relevant number of molecular properties are directly relatable to molecular weight, and thus, to the graph size. For example, among the 5 Lipinski rules for drug development [1], the first one is indeed a constraint on molecular weight, and the others (number of rotatable bonds, number of hydrogen donors/acceptors, LogP to a minor extent) are related to the graph size. Other key properties that relate to graph size include polar surface or synthetic accessibility, which are crucial in drug design. Therefore, we argue that methods to adapt the graph size during the generative process are worth studying, researching, and improving.

[1] Lipinski et al. (1997). Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced Drug Delivery Reviews, 23(1–3), 3–25.

• I would argue that the most interesting problems for generative models are sampling of similar molecules that optimize certain properties (...). For such tasks the size of the optimized/generated molecule is often close to the input molecule anyways.

Again, the reviewer’s statement is correct. Nonetheless, the size of the molecule is likely to change during the optimization process (e.g., consider removing a toxicophore group from a compound to improve its safety, or adding carbon chains to improve hydrophobicity). Contrary to existing methods, GrIDDD is capable by design of adapting to different graph sizes without workarounds such as training larger adjacency matrices to accommodate the out-of-distribution graph sizes.

• The interesting question here would be rather: can I start from the input molecule and only add a little bit of noise to ensure that the generated molecule is close. Here, a method like GRIDDD could be advantageous, as it allows, in theory, to add new nodes rather late in the diffusion process. I'm not sure if the "Property optimization" section goes in this direction, as the description does not mention if they start from a random latent or from a input molecule

We thank the reviewer for spotting this shortcoming and for giving us the opportunity to clarify this point. We described this exact setup in Section 4.3 (Out-of-distribution sampling) in the paper, but we forgot to add it to the experimental setup of Section 4.2 (Property optimization) as well. Indeed, as guessed by the reviewer, for property optimization, we start from an input molecule, we corrupt it for $t=100$ timesteps, and then perform the denoising process as usual. The example suggested by the reviewer (adding new nodes late in the diffusion process) is exactly why GrIDDD is so advantageous and effective in the molecular optimization task.

• The decision to use monotonic insertions and deletions make the underlying markov chain time irreversible. My feeling is that this might introduces a lot of problems and it also makes the underlying math very complicated.

If the reviewer is referring to our use of absorbing states (DEL), there is already literature discussing similar strategies [2], which shows that diffusion models work regardless of whether the underlying Markov chain is irreversible or not. At the same time, we point out how our reverse process only deals with the reversible portion of the Markov chain, as it only deals with DEL* nodes which are not absorbing nodes (rather, a gateway between proper classes and the absorbing DEL state, but still reversible). The irreversible part of the process is dealt implicitly by just predicting when one or more DEL nodes should switch back to DEL*. Therefore, we do not consider this an issue or a limitation.

[2] Austin et al. (2021). Structured Denoising Diffusion Models in Discrete State-Spaces. 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

• I wonder if there is any advantage in the way GRIDDD introduces insertions and deletions. The original DIGRESS model keeps the number of nodes fixed, but it already allows for arbitrary insertions and deletions of edges. (...) In my opinion the GRIDDD method should evaluate against this very simple baseline.

As the reviewer already noticed, fixing the graph size to a large number of nodes to account for out-of-distribution graph sizes would be extremely resource-inefficient, with slower training and sampling times especially for datasets with larger molecules such as ZINC250k. With that being said, we have implemented two baselines as requested by the reviewer and tested them on QM9. One is essentially DIGRESS with an additional DEL atom type, while the other adds a further edge type EDEL to mark any bond whose endpoint is of type DEL. The results of the comparison are shown below.

• val: validity (higher=better)
• avg. nc: average number of isolated connected components (lower=better)
• nc max: maximum number of isolated connected components (lower=better)
• n conn: number of valid connected graphs generated (higher=better)
• TV: total variation between generated distribution and the dataset’s (lower=better)
• CE: Cross-Entropy (lower=better)

As can be seen, both baselines perform poorly when compared to GrIDDD, especially in terms of connected components, indicating that the model has difficulties in generating molecules with less than nine nodes. The validity rates of the baselines appear higher, but a close inspection of the generated molecules shows that they are inflated by the presence of isolated carbon atoms which were not correctly attached to the main component.

• Finally, I found several small errors in the manuscript (...)

We thank the reviewer for pointing the small errors out. We confirm that we used $w$=0.05. The slope is not too steep for our needs (unfortunately, we can not post pictures). Equation 1 is indeed the derivative (with respect to $t$) of $\zeta(t)=1/(1+exp(-(D - t) / w))$, the function used for the transition matrices, but we compute the absolute value of the gradient to turn $\zeta’(t)$ into a bell-shaped function (and then normalize). All errors will be revised whenever we will be allowed to edit the original submission.

• (...) I do not really understand the experimental setup of the "property optimization" section

We believe we have answered to this concern above. Nonetheless, we are open to further discussion or clarification, if needed.

• (...) it's relatively trivial to implement node insertions/deletions in DIGRESS (...) I would still be interested in how well such a baseline method performs in your evaluations.

We believe we have answered to this concern above. Nonetheless, we are open to further discussion or clarification, if needed.

• Finally, I just do not understand why you need two different DEL states. Can you explain this in more detail please?

The DEL* state is the first state that a deleted node enters during the forward process. Within the GrIDDD formulation, it is necessary since the model is taught to learn the exact timestep where a node gets deleted. This is because, when denoising the sample at step t, we force a newly inserted node to switch immediately back to a "proper" node category at step $t-1$. The most intuitive category to assign at re-insertion would be DEL. However, the semantics of the DEL state is two-fold: it may either turn back into a proper node category, or it could stay in the absorbing state DEL (in the reverse process). Therefore, we determined that the best course of action is to treat the first timestep after a forward deletion as its own class (which we called DEL*). By doing so, the reverse process will always switch to a "proper" class at step $t-1$. We hope that our answer resolved the reviewer’s doubts, and we remain open to discussing further clarifications.

We thank the reviewer for the time and effort in evaluating our rebuttal. We are glad that we both agree on the utility of adapting the graph size for molecular optimization.

Comment on molecular weight (MW)

While we acknowledge that MW may not be the most suitable property to be targeted in drug design, in this paper it serves two specific purposes:

• Since it strongly correlates with the number of nodes of the molecular graph, it provides a natural and intuitive testbed to assess the ability to flexibly adapt the graph size, which is precisely what GrIDDD was designed for.
• It enables a fair and comprehensive comparison with FreeGress, which first introduced this target in its experimental setting. Note that FreeGress is our direct competitor, achieving state-of-the-art performance in property targeting while also including a mechanism to control the graph size.

We commit to better explain the rationale behind choosing MW in the revised version.

On a more general note, we would also like to emphasize that GrIDDD has been designed to dynamically control the graph size, and not to optimize any molecular property in particular. Therefore, our contribution is primarily methodological and intended for the broader task of targeted graph generation, although we were constrained to apply it in the molecular domain due to the abundance of baselines. From this broader perspective, we see GrIDDD as a step forward in enhancing the flexibility and generality of discrete diffusion models for graphs, opening the door to applications beyond drug design (e.g., [1,2]).

Comment on the complexity of the model

We agree that GrIDDD introduces additional components when compared to seminal models like DiGress. However, we believe these additions are minimal and principled. Aside from the additional scheduler, the only modifications involve two extra rows and columns in the $A$ and $B$ matrices, and two sparse matrices $C$ and $D$, with at most one nonzero entry per row. Techniques such as insertions and deletions have been widely studied [3], although not for graphs. The same applies for the matrices $\overline{Q}_{t|s}$ [4] and absorbing diffusion [5,6]. As the reviewer correctly noted, the ability to adapt the graph size can offer a clear advantage in terms of flexibility (since our model is the only one capable of adapting the graph size without rough workarounds) and effectiveness (as shown by the empirical results and the ablations). In this light, we view the added components not as a detour from elegance, but as a natural extension motivated by necessity and well-justified by the empirical improvements.

Comment on the baseline

We will include the experiments with the baselines in the main paper, and we thank the reviewer for pointing us to this useful ablation that has significantly strengthened our contribution.

Final request

We would like to ask whether our rebuttal and this discussion have sufficiently addressed the reviewer’s concerns. We remain open to further discussion if needed.

References:

[1] Cheng et al. (2024). Parallel Vertex Diffusion for Unified Visual Grounding. Proceedings of the 38th AAAI Conference on Artificial Intelligence.

[2] Shabani et al. (2022). HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising. The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3] Johnson et al. (2021). Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Models. Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models (ICML 2021).

[4] Zhao et al. (2024). Unify Discrete Denoising Diffusion. Preprint.

[5] Austin et al. (2021). Structured Denoising Diffusion Models in Discrete State-Spaces. 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

[6] Kong et al. (2023). Autoregressive Diffusion Model for Graph Generation. Proceedings of the 40th International Conference on Machine Learning.