Thanks a lot for the review. We appreciate the comments and the criticism. However we believe there has been some misunderstandings, and we hope these issues will be clarified with our revised manuscript and the following answers. We address here the weaknesses pointed out by the reviewer, in the same order.

Weakness 1:

While indeed people have worked on generating molecules by fragments, it has not been solved, this is clearly highlighted in the conclusions of the review articles the reviewer points out [1], from their conclusions:

“Looking forward, we identify these research directions as promising for the field: Transparency and Uncertainty Quantification. Understanding how models decide on specific fragment is crucial for critical areas such as drug design (Deng et al., 2021).Integrating uncertainty quantification into models can identify reliable molecules with promising properties, thereby streamlining the drug design process (Hu et al., 2023). Focus on Realistic Properties. While many studies focus on computational metrics such as logP or QED11, there is a need for experimental validation of lead molecules. Future research could shift towards the generation of molecules optimized for properties that are realistically testable and applicable (Bilodeau et al., 2022). Advancing 3D Molecule Generation. The transition to 3D molecular representations is critical for structure-specific tasks. The application of fragment-based methods to 3D molecule generation provides an opportunity to address validity challenges in this area. Despite initial studies (Qiang et al., 2023), further exploration in this area is needed”

In light of this review paper [1] the reviewer points out, our model is clearly novel. In view of the highlighted points in the conclusion from [1] we can clearly see how our model addresses points 2.) and 3.) directly:

• We use properties like redox-potential, log-solubility and synthetic accessibility which are real world properties and key for the design of electroactive organic molecules for aqueous redox flow batteries, a promising alternative for energy storage. Moreover we evaluate our model on a significantly harder dataset than QM9, since QM9 contains molecules up to 9 heavy atoms, while the dataset we used contains molecules of up to 54 heavy atoms (mean of 24). This constitutes a significantly more realistic scenario. And we optimize the three properties simultaneously.
• We propose a viable solution for integrating 3D coordinates into an autoregressive model. Directly addressing a notable challenge of this type of models.

In view of the evident connection of our contributions to the needs and ‘research directions’ outlined in the review paper [1] pointed out by the reviewer we are more confident than our contributions are significant. We have now added the review paper to further contextualize the importance and contributions of our work.

Weakness 2

The evalution is very weak. This paper only demonstrates MolMiner's application in only three properties, specifically, SAScore, solubility, and redox potential. Evaluting the model solely on them is not a strong evidence of praticality of the proposed models, as these scores are somehow week and simple.

Note: We have now performed a benchmark comparison against a well established autoregressive model of the same characteristics, this is now added to the main test. We believe this further strengths our evaluation.

Demonstrating simultaneous multi property conditional generation using three properties, and in particular our choice of realistic and useful properties for the design of organic molecules for aqueous redox flow batteries is not a minor challenge in the least. We completely disagree that Redox potential or Solubility are ‘weak’ and ‘simple’ properties to do conditional inverse design. These are properties that are hard to predict, even with traditional simulations like DFT, which is among the reasons why finding suitable organic compounds for this type of batteries remains a challenging problem.

Weakness 3

The Introduction section mentioned "Incorporating 3D information". However, I did not notice how 3D information was significantly incorporated in both method formulation and evaluation.

Lines 255-258 introduce how the 3D information is computed at every step of a story, lines 296-299, 303-304 introduce how the 3D information is fed into the attention mechanism of the Transformer. Figure 3, on the right, Inside the architecture of the transformer, the specific point at which the geometry is introduced into the network is colored in red, the process is also described in the caption of figure 3. In figure 4 the pairwise-distance matrix is highlighted in red, with the caption ‘Pairwise distances between fragments. Then Apendix “A.4 The role of the geometry” (lines 795-841) discuss the effect of the geometry and point into future work regarding the role inclusion of geometry in ML models.

We would like to thanks the reviewer for its valuable work. We would like to use this comments to address some of the issues raised by the reviewer (most notably we have now benchmarked our model against its predecessor, to further highlight our contribution) and also clarify some points we may disagree on.

Weakness 1

Limited Empirical Scope: The experimental evaluation is restricted to a narrow set of target properties (log-solubility, redox potential, synthetic accessibility). Broader validation, including additional properties (e.g., pharmacokinetics or biological activity) and diverse datasets (such as QM9 or ZINC), would substantiate the versatility of MolMiner.

We disagree that the evaluation on simultaneous multi-property conditional generation using 3 properties constitutes a narrow set, most research done focuses on optimizing one property at a time. In the calibration experiment we perform 3 different experiments, on all of them the 3 properties are simultaneously being used as targets, but on each of the experiments we fix 2 of the properties to the mean while we vary only one at a time. Its important to note, that while we are varying only one, we are optimizing for the three. This is incredibly challenging and we disagree its narrow. The reason why we vary one property at a time, is because otherwise we could not visualize the calibration of simultaneously varying all three properties (that would require a 6-dimensional plot). It must be noted that very often when other work do conditional generation, single or multi-property, they look only at one set of values of the properties, and not to how calibrated the whole model is for all possible values of the single, or multiple properties. In other words, their evaluation, is on a single slice at constant value of the x-axis of our calibration experiments. We evaluate on the whole range of values possible for that target variable.

The set of properties chosen: Synthetic accessibility, redox potential and log-solubility are arguably the key properties to design organic molecules for aqueous redox flow batteries, which is the type of compounds the dataset was initially created for. We had this application in mind because we believe its a promising alternative for energy storage. We didnt include properties like pharmacokinetics or biological activity because the dataset we used did not use those properties in their annotation of molecules, because they are not the most important for the the design of the compounds the dataset is interested in. We showed it works on three arguably challenging properties simultaneously, so we have no reason to believe it would not work on other properties like those mentioned, its just that for the purpose of the dataset, those properties are not a priority.

Finally we did not use QM9 because we wanted to test our model on a more complex, and interesting set of compounds.. QM9 contains molecules with up to 9 heavy atoms, whereas the dataset we used the mean number of heavy atoms in a molecule was 24, with the maximum number of heavy atoms being 54. This constitutes a noticeably harder challenge.

To summarize, we have now benchmarked an existing model against MolMiner on a noticeably challenging task: multi-property simultaneously conditional generation of molecules, training both models in a real-world dataset. We believe this evaluation is challenging and so demonstrates the validity of our approach.

Weakness 2

Lack of Interpretability Quantification: While the model's interpretability is highlighted, the benefits are not explicitly demonstrated or quantified. User studies involving domain experts or real-world case studies would provide practical evidence of the interpretability advantages claimed by the authors.

We believe the approach is naturally more interpretable because the generation process is transparent. As can be seen from figures 5d, then figures 10-31 from the appendix, the generation process is decomposed in a sequence of steps. Furthermore, thanks to this, a chemist can intervene at every step of the generation and force a particular choice, then let the model complete the rest. Also one can give the model a partially grown molecule, and tell the model to complete it to satisfy some desired criteria. These possibilities are why we think our model is more transparent or interpretable, than others.

Weakness 3

Reviewer: W3. Novelty of method (modeling molecules via generating fragments) seems low and I would have liked to have seen the differences to these approaches better studied or discussed. (hence why low "contribution"/"presentation" score).

Its not the use of fragments that is novel with our work, but the way they are used, hence why how we treat the fragments to incorporate symmetries takes an important role on our paper (sections 2.1, 2.2 in the main text, sections A.1,A.2 in the appendix). To our knowledge this processing of fragments is novel, and crucial because otherwise the vocabulary of fragments could contain duplicates.

And thats exactly what happens in models like HierVae (Jin et al, 2020), from their original implementation https://github.com/wengong-jin/hgraph2graph/blob/master/data/logp04/vocab.txt#L30C1-L40C42 in their extracted vocabulary, lines 30-40 correspond to the vocabulary of attachments of the fragment ‘C1=CC=[NH+]C=C1’. Within all these different attachments of the fragment, there are duplicates. For example, a): 'C1=C[CH:1]=CC=[NH+]1' and b): 'C1=[NH+]C=C[CH:1]=C1' are the same attachment configuration (note this are indicated by a number next to the atom e.g C:1), but its only because their model does not take into account the symmetries that this is possible. Now, if during training, the model predicts a.) but the ‘correct answer’ is b) that would prevent the model to learn correctly. So the vocabulary is unnecessarily larger (because it contains duplicates), and the presence of duplicates harms the learning process. Our model deals with this by incorporating the symmetries. This we believe is a novel contribution on its own and we also incorporate geometry during the generation which has remained a notable challengel. To address the related work the reviewer points out:

• Marco Podda, Davide Bacciu, Alessio Micheli (2020) 'A Deep Generative Model for Fragment-Based Molecule Generation' AISTATS 2020.

This is a generative model that operates on SMILES strings. The objective of this work was to increase the chemical validity of the generated structures and increase their diversity. From their Introduction:

‘Our experimental evaluation shows that our model is able to perform on par with state-of-the-art graph- based methods, despite using an inherently least expressive representation of the molecular graph. Moreover, we show that generated compounds display similar structural and chemical features to those in the training sample, even without the support of explicit supervision on such properties.’

The authors mention the limited expressiveness of using SMILES strings as the modeling approach, although they are capable of leveraging this representation to increase the chemical validity of the generated molecules. But perhaps more important, from their conclusion:

‘... we show that our contributions can increase validity and uniqueness rates of LM-based models up to the state of the art, even though an inherently less expressive representation of the molecule is used. As regards future works, we aim at extending this model for task like molecular optimization. This will require the design of novel strategies to maintain high uniqueness rates, while preserving smoothness in latent space. In addition, we would like to adapt the fragment-based paradigm to graph-based molecular generators’ So the authors want, in future work, to adapt the approach to a more expressive representation of molecules, based on graphs.

Therefore one can argue that this further highlights the novelty of our work, MolMiner, because it achieves chemical validity while using a more expressive representation for the molecule.

• Bengio, E., Jain, M., Korablyov, M., Precup, D. and Bengio, Y. (2021) ‘Flow network based generative models for non-iterative diverse candidate generation’, NeurIPS 2021
• Xie, Y., Shi, C., Zhou, H., Yang, Y., Zhang, W., Yu, Y. and Li, L. (2021) ‘MARS: Markov Molecular Sampling for Multi-objective Drug Discovery’, ICLR 2021.
• Maziarz, K., Jackson-Flux, H., Cameron, P., Sirockin, F., Schneider, N., Stiefl, N., Segler, M. and Brockschmidt, M. (2022) ‘Learning to Extend Molecular Scaffolds with Structural Motifs’, ICLR 2022.

While these works help contextualize our model within current research in the field, they do not hinder the novelty of our approach. None of this methods include geometry in the generation process. We want to thank the reviewer for pointing this important work to us. We have now added them to our revised version to contextualize our contributions.

We would like to thank the reviewer for its detailed comments. We hope we will clarify some points with our comments here, and we have addressed some of the raised issues in a revised version.

Weakness 1

Reviewer: "The experiments do not consider any baselines, such as the alternative fragment-based approaches discussed (such as Jin et al., 2019, 2020) or non-fragment-based approaches (such as Gómez-Bombarelli et al., 2018). It is therefore hard to gauge how useful MolMiner would be and what advantages it offers over existing, already established approaches."

We have now benchmarked our model against HierVAE (Jin et al 2020), which shows that our model can generate significantly more novel molecules subject to desired conditions than its predecessor, HierVAE.

Reviewer: Moreover, I also have a hard time distinguishing the importance of the design choices made, e.g., the incorporation of 3D information (lines 256-258) or the semi-order agnostic serialization method proposed (line 263). (Information on 3D information is shown in Table 2 of the appendix, but it seems to only be in terms of accuracy, rather than the effect on optimization, and the effect seems fairly minor).

Incorporating the 3D geometry is shown to contribute to better reconstruction accuracy during training, as shown in Table 2.Its effect is minor, but we suspect this is part of larger trend, for example, if one looks at the leaderboard of MatBench a benchmark for ML methods in predicting materials properties, one can see that for 6 out of 13 tasks models that only use composition (without 3d structure) lead. And even when they do not lead the task they perform surprisingly well. Therefore the effect of 3D structure there is also smaller than one might think. With this we mean this is a general phenomena in the field, and that is what the authors are working currently on explaining (in a different work), but it falls outside the scope of this paper, in this paper we show that 3d geometry improves the accuracy of the model.

The empirical justification of using a semi-order-agnostic choice can be seen in the new benchmarking against HierVae (Jin et al 2020). They don't use a semi-order-agnostic, so every time they train on the same sequence, and so the model does not learn to flexibly construct molecules which results in a significant lower novelty score (although this is a consequence as well of our way of working with symmetries of fragments).

Weakness 2:

Reviewer: The method for optimizing molecules (based on picking an initial fragment according to how well it is modeled to affect a particular property score) seems like it is unlikely to scale to more interesting optimization tasks. My current understanding of the optimization method (lines 380-386) is that it involves learning a feed forward neural network to pick the initial fragment, but that the rest of the fragment generation process happens as usual. For harder optimization tasks, I worry that other parts of the molecule will also be important and so this method is unlikely to scale.

We disagree, this simple model is the best and simplest way to initialize the autoregressive process. This model’s task is to predict the probability of any of the fragments in the vocabulary of fragments appearing in a molecule, given a set of the molecules properties. The output of the model is then, for each of the fragments in the vocabulary, the probability of finding that fragment anywhere in a molecule with those properties. Then, if we take top-k fragments, that is, the fragments that the simple models is most confident they will appear anywhere in the molecule, then we can pass one of them to the autoregressive model, and because this models will decode the whole molecule from any starting point, in any order, then as long as the initial fragment is in the original molecule then the autoregressive model will be capable of reconstructing the molecule from it. In other words, the simple model does not need to guess all the fragments that will be in a molecule given some properties, as long as it gets 1 right, then the autoregressive model, given it can work with any order and any starting point, will take it from there. Its the simplest and more scalable way one can think of initializing an autoregressive model.

Furthermore, the autoregressive process, what the reviewer refers to as ‘rest of fragment generation’ does not ‘happen as usual’, the full process is condition dependent, at every decision in this process the choice is subjected to the target conditions.We believe this is a misunderstanding. One can see how the conditions are embedded upon every step of the generation process from figures 3,4, (in purple) labeled as “Conditions”. Also it is discussed in lines 297-299 and 311-313 in the revised text (it was present also in the previous version).

Weakness 3:

We thank the reviewer for pointing this work out, we were unfamiliar with it. It proposes a different approach to multi-property conditional generation. We have now added it to the revised version to contextualize our contributions.

(continuation to weakness 2)

• Our model does not operate on a graph: While the chemical graph of a molecule is retained to perform chemical validity checks, the model architecture does NOT use the chemical graph. Its a transformer model, which can be viewed as a fully connected (dense) graph, so every fragment is connected to every other fragment (as opposed to what graph-based approaches for molecules often do, they follow only the chemical graph) and on top of that we embed the 3d geometry of the molecule in a way that makes it invariant to rotations and translations (the pairwise distance matrix). Therefore this is not a widely used graph based model.

• The embedding of the geometry and its role: As the reviewer points out, this is a clearly novel part of our approach, but its not the only novel thing. We also note in appendix subsection “THE ROLE OF INCORPORATING GEOMETRY” that the contribution of the 3D geometry, while positive, its not as great as one could have thought it would be. And we point out that future work will need to be done to explain this surprising result. However, and now we include it in that same section in the revised paper, this is not an Isolated phenomena. The unreasonable effectiveness of poor materials representations is a widely reported issue, including work by [Tian, S. I. P., Walsh, A., Ren, Z., Li, Q., & Buonassisi, T. (2022). What Information is Necessary and Sufficient to Predict Materials Properties using Machine Learning? https://arxiv.org/abs/2206.04968]. This widespread issue in Machine Learning for Materials and molecules is currently being studied, but in a global context of the field, which is out of the scope of this paper. In this paper we show that 3D geometry helps. Furthermore, if one wants to contextualize this one may take a look at matbench (https://matbench.materialsproject.org/) a benchmark of machine learning models for predicting materials properties. There, out of 13 tasks for general purpose algorithms, in 6 of them, the leading models do not use 3D geometry, and even in those where 3D geometry-aware models lead, composition restricted models seem to perform remarkably well, therefore and following the same argument the reviewer states, why would we use 3D geometry at all?. This we believe is the same thing we are reporting in our work. It requires further study, and we are actively working on it, but its global phenomena and so the scope its out of this work.

On the evaluation and baseline comparison in experiment section:

Note: New benchmarking done, it has been added to the revised version

In this new benchmarking done one can see how our model is capable of retaining comparable performance on conditional multi-property generation while significantly increasing the novelty ratio. We believe this result highlights the importance of our contributions.

On the fragment extraction, difference with JT-VAE and HierDiff. :

Note: This discussion has been now included in the appendix section: “Comparison on fragmentation approaches”

Our approach of decomposing the fragments within a molecule is most similar with that of HierVAE, itself based on previous work by the same authors, JT-VAE. HierDiff is based on JT-VAE, so in summary our approach in MolMiner, HierVae, JT-VAE and HierDiff are closely related.

Ours is the simplest among the others. It breaks a molecule into its smallest set of small rings (SSSR), and all bonds not within a ring and in the main text we provide the motivation to use such a simple approach. This also means that our fragments tend to be smaller than those of the rest of the approaches, but we found no issue with this (e.g the total number of fragments for the dataset used is 34). For example, two fused rings could constitute a single Fragment in HierVae, HierDiff or JT-VAE, whereas in MolMiner it would always be two fragments, one per ring. This is particularly useful because then, all fragments extracted by MolMiner are Single Cyclic graphs, which is what allows us to deal with Fragments symmetries in a way that neither HierVAE, HierDiff or JT-VAE do.

As now shown in the new subsection of the appendix “COMPARISON ON FRAGMENTATION APPROACHES” if one does not take into account the symmetries, fragments contain duplicates, which harms the training process and unnecessarily increases the size of the vocabulary. We highlight this issue, which has to the best of our knowledge not been reported, on an example within the original implementation of HierVAE. We hope this further highlights the importance of the contributions and novelty of our work.

On the novelty from the network structure parts.

Note this discussion on the role of 3D geometry is now added to the appendix subsection “THE ROLE OF INCORPORATING GEOMETRY"

• The training objective is not widely used: Autoregressive models are not trained on different rollouts at every epoch, for every molecule. Moreover our rollouts are done in the most general way we could think of from lines 241-242 in the original text:

‘A story can start from any fragment and can be grown in any particular order, only bound by having to dock new fragments to existing ones’.

This is the most general formulation the problem of autoregressive generation can take. And it's not a widely used formulation. We believe our contribution in this aspect is to generalize the autoregressive approach to its minimum constraint which allows us to formulate the problem as a semi-order-agnostic process and use the theory of order-agnostic processes.

(to be continued in the next comment)

(Continuation on Weakness 1)

From [2] in their conclusion:

‘While the non-autoregressive approach has the advantage of fast and computationally efficient generation of molecular graphs without any iterative procedure, the existing methods suffer from low performance. To overcome this limitation, we proposed an efficient learning method to build a graph VAE that generates molecular graphs in a non-autoregressive manner. In order to improve the generation performance, we introduced approximate graph matching, reinforcement learning, and auxiliary property prediction into the training of the model. ….. One downside of the proposed method its high complexity with regard to space and time. Both the space and time complexity increase with the size of the graphs, owing to the use of graph neural networks. Thus, the proposed method would be impractical for learning and generating large molecular graphs, e.g., hundreds of heavy atoms in a molecule.’

Once again the authors highlight the fast and efficient advantage, but no claim on global modelling advantage.

From [3] : In the Conclusion

‘Lastly, it will be promising to explore alternative generative architectures within the MolGAN framework, such as recurrent graph-based generative models (Johnson, 2017; Li et al., 2018b; You et al., 2018), as our current one-shot prediction of the adjacency tensor is most likely feasible only for graphs of small size.’

Again there are no claims of better global modeling, and as a matter of fact, the authors seem to be pointing at autoregressive models (they refer to them as ‘recurrent’) as a promising direction to be able to deal with larger molecules.

From [4] its a Normalizing Flow based generative model. There is no claim of improved global modeling over size-autoregressive models, instead the argument focuses on the equivariant nature of the approach. Similarly, [5] is a Diffusion based model. We could again not find any claims here about the advantage in global modeling of non-autoregressive models over autoregressive. Moreover, the only mentions of size-autoregressive models in [5] section Related Work:

‘Tangentially related, other methods generate molecules in graph representation. Some examples are autoregressive methods such as (Liu et al., 2018; You et al., 2018; Liao et al., 2019), and one-shot approaches such as (Simonovsky & Komodakis, 2018; De Cao & Kipf, 2018; Bresson & Laurent, 2019; Kosiorek et al., 2020; Krawczuk et al., 2021). However such methods do not provide conformer information which is useful for many downstream tasks.’

Point out that some of the autoregressive models do not provide conformer information, which is something that MolMiner does (incorporating 3D spatial information into the attention mechanism). And in the conclusion

‘We presented EDM, an E(3) equivariant diffusion model for molecule generation in 3D. While previous non-autoregressive models mostly focused on very small molecules with up to 9 atoms, our model scales better and can generate valid conformations while explicitly modeling hydrogen atom’

They compare the better scalability to larger molecules of their approach to other non-autoregressive approaches. This might well be because autoregressive approaches scale better to larger molecules, as also pointed out in the conclusion of [3], and so the comparison they do is only within non-autoregressive models. In any case, not any mention of the advantages in global modeling.

In summary, On the flexible node numbers during generation: This constitutes an improvement over methods that do fix the size of the molecule upon generation, because constraining the size of the molecule is unjustified and the way the molecule size is constrained in conditional generation (e.g in [1]) is problematic.

On the motivation of non-autoregressive generation over autoregressive generation in the sense of global modeling: The claims from [1], used to rebut our proposal, have been investigated and shown to be unfounded. We do not claim better global modeling over non-autoregressive approaches because we can't prove it, but neither does [1] prove the opposite.

[1] Coarse-to-Fine: a Hierarchical Diffusion Model for Molecule Generation in 3D

[2] Kwon, Y., Yoo, J., Choi, Y.-S., Son, W.-J., Lee, D., and Kang, S. Efficient learning of non-autoregressive graph variational autoencoders for molecular graph generation. Journal of Cheminformatics, 11(1):1–10, 2019.

[3] De Cao, N. and Kipf, T. Molgan: An implicit generative model for small molecular graphs. arXiv preprint arXiv:1805.11973, 2018.

[4] Satorras, V. G., Hoogeboom, E., Fuchs, F. B., Posner, I., and Welling, M. E(n) equivariant normalizing flows, 2022

[5] Hoogeboom, E., Satorras, V. G., Vignac, C., and Welling, M. Equivariant diffusion for molecule generation in 3d, 2022.

Thanks a lot for the review. We value your input a lot. We believe there have been some missunderstandings, and so we will address here all the weaknesses and questions raised, together with a revised manuscript where we emphasize this discussion.

Weakness 1

On the flexible node numbers during generation:

Note: The importance of enabling flexible node size during generation is now discussed in a new subsection in the appendix under the title “On the importance of variable-size autoregressive generation of molecules”

Enabling flexible node numbers during generation signifies the removal of an arbitrary constraint. Why should any model predefine the size (in terms of atoms or in terms of fragments) of the molecule? We believe this is a result of adapting generative approaches from Vision tasks (image generation, where the image has a fixed size) to materials modeling without taking into account the unique challenges of materials.

Given a set of target properties, the space of possible molecules that satisfy those properties do not necessarily have the same size, therefore, by constraining the space of possible solutions, to those with some predetermined size constrains the solution. Given that this is an unnecessary constraint (other than convenience), the justification should be demanded from models that impose it, and not of models that remove it.

Finally and to further highlight the error that fixing the size of the molecule upon generation constitutes, we can go deeper into how the size of the molecule is actually chosen upon initialization in models that pre-define it. From [1] section C. Additional Experiments subsection C.1. Experimental configuration.:

‘In all non-autoregressive methods, the number of nodes used for sampling is drawn from the size distribution histogram calculated on the training set.’.

So they sample the size of the molecule from the size distribution of the dataset. If they do so for non-conditional generation this imposes an unjustified constraint as discussed before. However if this is done for conditional generation (section C.6. Conditional generation) then this is altogether wrong. Because they are sampling from the distribution of sizes when they should be sampling from the conditional distribution of size given the property. In other words, in general, given a set of target properties the distribution of sizes of molecules satisfying those properties IS NOT the general distribution of sizes. We hope this highlights the fact that enabling the model to choose the size of the molecule during generation is the right approach.

On the motivation of non-autoregressive generation over autoregressive generation in the sense of global modeling:

We disagree on the claim of global modeling stated in [1] and pointed out by the reviewer. From [1] they state

‘Compared with the autoregressive approach, non-autoregressive generative models are more promising for 3D molecule generation, due to their natural advantages of global modeling ability’.

It is unclear what they mean by global modeling. An autoregressive model can perfectly do global modeling, and in general it does. For example our model, MolMiner, does global modeling, in every decision the model takes as input the entire molecule grown up to that point, not just its local environment. But lets examine the citations [1] use to support the aforementioned statement [2][3][4][5]:

From [2] in the introduction:

‘The non-autoregressive approach successfully generates small molecular graphs in a very fast and efficient manner but suffers from difficulty in model training and a low validity ratio when generating larger graphs. Owing to such limitations, the autoregressive approach, which aims at sequentially generating a molecular graph node by node using an autoregressive distribution, has been a main research direction…. These methods successfully generate molecular graphs with a high validity ratio, while involves an iterative procedure for each generation which makes them less efficient.This work focuses on the non-autoregressive approach for fast and efficient generation of molecular graphs. Here we propose an efficient learning method to train a VAE that generates molecular graphs in a non-autoregressive manner.’

From this extract we can not see supportive information for the claim of improved global modelling. The advantages of non-autorregressive models discussed in this extract come from the fact they are faster and more efficient according to the authors.

(to be continued in the next comment)

[1] Coarse-to-Fine: a Hierarchical Diffusion Model for Molecule Generation in 3D

[2] Kwon, Y., Yoo, J., Choi, Y.-S., Son, W.-J., Lee, D., and Kang, S. Efficient learning of non-autoregressive graph variational autoencoders for molecular graph generation. Journal of Cheminformatics, 11(1):1–10, 2019.