Major 1: Ranking comparison in Figure 1

Results are updated to show ~$2\times$ improvement

Thank you for your suggestion. We have created a new figure (attached in the rebuttal PDF) based on your feedback. We first identify the maximum ranking position among the three single-conditional generation sets for each multi-condition generated polymer. Then, we calculate the median of these maximum ranking positions, which is 16—approximately $2\times$ better than single-conditional generation, which has a median value greater than 30.

Major 2: Data imbalance issue

The datasets are balanced, and the accuracy is reasonable

Thank you for your comment. Please refer to Line 200:

For drug design, we create three class-balanced datasets from MoleculeNet [45]: HIV, BBBP, and BACE...

The positive-to-negative ratio is 1:1, making accuracy a reasonable metric in this case.

Major 4: How avg. rank calculated

It is calculated by averaging the ranking positions of model performance for each condition (property).

Major 5: The issue with $q(\mathbf{X}_G^{t} | \mathbf{X}_G^{t-1})$

We correct this issue

Thank you for pointing this out! The term $\mathbf{X}_G^{t-1}\mathbf{Q}_G^{t}$ results in unnormalized probabilities. In the implementation, we need to split it into node and edge components and normalize them separately. We have corrected this issue and revised the description of Eq. (6) as follows:

We introduce a new diffusion noise model. At each forward diffusion step $t$, noise is applied to $\mathbf{X}_G^{t-1}$, resulting in an unnormalized probability $\mathbf{\tilde{p}}= \mathbf{X}_G^{t-1} \mathbf{Q}_G^t$. We first separate and normalize the first $F_V$ columns of $\mathbf{\tilde{p}}$ to obtain the noisy node states $\mathbf{X}_V^t$. We then reshape and normalize the remaining $N \cdot E$ dimensions to obtain the edge states $\mathbf{X}_E^{t}$. These components are combined to construct $\mathbf{X}_G^{t}$.

Minor 1: Can the proposed method achieve no condition during training

Yes, here are two strategies:

1. Replace the condition encoder with a molecular encoder that uses a self-supervised task during large-scale pretraining. In this case, the generation conditions should be the molecular structure rather than labels. For property conditions, we can retrieve molecular structures with similar labels.
2. Learn null embeddings of the conditions during pretraining. Although fine-tuning the condition encoder for specific tasks will still be necessary, pretraining can help reduce costs and label requirements.

All these strategies are promising directions for extending Graph DiT.

W1, W2: Contribution, limitation, and paper presentation

Thank you for your comment. We can highlight lines 41-43 and 62-64 using italics to emphasize the contributions and organize them into bullet points. Any further suggestions or discussion are highly appreciated.

We discussed limitations in Section 4.4 (Lines 281, 288-289) regarding generation diversity and Oracle functions used for evaluation. We revise the main paper to acknowledge these again in the Conclusion.

We have corrected Figure (4) colors and revised Eq. (6) for clarity. Any further suggestions or discussions are welcome.

W2: Lack of metrics (Novelty & Uniqueness)

Results on required metrics

We show that Graph DiT achieves good values for novelty and uniqueness.

Flaws in uniqueness and novelty

We did not choose uniqueness and novelty as major metrics because recent research shows they can be flawed [1]: randomly adding carbons to existing molecules can yield almost 100% novelty and uniqueness. Instead, we use internal diversity to better reflect generation diversity.

Current evaluation has 9 metrics, enhanced by expertise

We defend our nine metrics for evaluating molecular quality, practical utility, and diversity. These metrics cover chemical validity, distribution matching (diversity, distance, similarity), and multi-condition controllability.

Additionally, Section 4.3 includes case studies with domain experts, providing valuable assessments often lacking in previous studies.

Based on the above points, we respectfully request a reconsideration of the assessment if the reviewer finds these metrics sufficient. We welcome further discussion on this matter.

W3: Results in Table 2

From 0.6 to 0.9, Graph DiT significantly outperforms the baselines

Thank you for your comment. Condition control is a core goal in inverse molecular design, as we need to design drugs/materials that meet human requirements. As shown in Table 2, Graph DiT effectively improves existing baselines from 0.6 to 0.9 in this regard.

Based on the above points, we respectfully request a reconsideration of the assessment that "... results in Table 2 did not demonstrate superior performance..."

More thoughts on effectiveness and practical utility

We value the reviewer's opinion that an ideal model should excel across all metrics. However, we respectfully argue that defining ground-truth diversity and distribution distance is still debated with many choices (e.g. uniqueness or internal diversity). Therefore, we use up to 9 metrics to comprehensively evaluate model performance from diverse perspectives. Comparing very similar numbers, such as a distance metric of 6.7 vs. 7.0, is less compelling for demonstrating a model's ability to generate practically useful molecules. Significant improvement in multi-conditional controllability is more indicative of practical utility. Additionally, in Section 4.3, we involve domain experts to verify the generated results in real applications.

We greatly appreciate the reviewer’s comments and invite further discussion on the matter for any remaining concerns.

Q2: Diverse scales of properties

Lines 217-218 align with the reviewer’s question

Thank you for the excellent question. As detailed in Lines 217-218, our implementation for molecular optimization baselines aligns with your question. Results in Tables 1 and 2 demonstrate that Graph DiT effectively outperforms these baselines.

Q3 LCC meanings

Lines 185-188: LCC denotes largest connected component

Thank you for your question. Please see Lines 185-188 for more:

A common way of converting generated graphs to molecules selects only the largest connected component [42], denoted as Graph DiT-LCC in our model. For Graph DiT, we connect all components by randomly selecting atoms. It minimally alters the generated structure to more accurately reflect model performance than Graph DiT-LCC.

Q4: Visualization method in Figure 6

We use PCA (Principal Component Analysis) to reduce the dimensionality of Morgan Fingerprints [2] to two dimensions for visualization.

Reference

[1] Genetic Algorithms are Strong Baselines for Molecule Generation. 2023.

[2] Extended-Connectivity Fingerprints. JCIM 2010.

Thank you for the authors' responses. The authors' rebuttal does not convince me.

Firstly, I disagree with the authors' claim that "randomly adding carbons to existing molecules can yield almost 100% novelty and uniqueness. Instead, we use internal diversity to better reflect generation diversity." Additionally, I could not find evidence of the alleged deficiencies in uniqueness and novelty as mentioned in reference [1]. Could the authors provide evidence supporting the above claims? On the contrary, simply adding carbon atoms, such as carbon chains, tends to result in nearly 100% validity. Uniqueness is an effective measure to assess the model's distribution learning capability (i.e., whether it only generates simple carbon chains), while novelty serves as an indicator of whether the model is experiencing overfitting.

Based on the additional experimental results, the proposed DiT model's uniqueness is not high, at only 0.89. Furthermore, the validity (second to last) and novelty (third to last) do not outperform the baseline models. This suggests that the model's ability to learn molecular graphs (validity), distribution learning (uniqueness), and resistance to overfitting (novelty) are relatively trivial.

W1: Limited Demonstration of Multi-Condition Capability

We evaluate models on up to four conditions, not two

Thank you for your comment. In Table 1, we evaluate all models on up to four conditions: $O_2$, $N_2$, and $CO_2$ permeability.

Based on the above points, we respectfully request a reconsideration of the assessment that "Limited Demonstration of Multi-Condition Capability".

W2: Text condition

Text condition is out of the scope of the paper

We appreciate your comment. Property conditions align with our research objectives in inverse molecular design.

The DiT work [1] did not explore text conditions

The original DiT work [1] used classes from ImageNet rather than text conditions, leaving text conditioning for future work.

Based on the above points, we respectfully request a reconsideration of the assessment that "it is imperative to assess how it performs under texture condition" and "adherence to the principles derived from DiT".

W3: Incomplete uniqueness, novelty, and FCD

FCD was used in the paper

Line 208, Tables 1 and 2, and Figure 4 presented FCD as the distance metric.

Uniqueness and novelty are provided

Here are results complementing Table 1:

Graph DiT achieves good results. However, we note these metrics alone don't necessarily indicate practical utility.

Uniqueness and novelty may be flawed metrics

Recent research has shown uniqueness and novelty can be easily flawed [2]: Randomly adding carbons to existing molecules can yield almost 100% novelty and uniqueness. We have opted to use internal diversity, which offers a more nuanced reflection of generation diversity.

Tables 1 and 2 present up to 9 comprehensive metrics

The nine metrics (Lines 204-211) are comprehensive, evaluating chemical validity, distribution matching, and multi-condition controllability. They assess diverse and useful molecule generation. Section 4.3's expert case studies provide additional valuable model assessment.

W4: Ambiguity in graph-dependent noise

Figure 4$(c)$ shows ~2x improvement on controllability

Thank you for your feedback. Figure 4$(c)$ shows the non-dependent noise model has only 49-55% of the graph-dependent model's controllability for gas permeability, demonstrating a significant improvement. We respectfully request a re-examination of the results.

Q1: Ranking and K value

Details are in Lines 35-37, 56-57

For single-condition constraints (Lines 35-37):

we check whether a shared polymer structure that meets multi-property constraints can be identified across different condition sets. If we find the polymer, its rank K (where K is between 1 and 30) indicates how high it appears on the lists, considering all condition sets. If not, we set K as 30.

For multi-condition generated graphs (Lines 56-57)

The Oracle determines the rank of this graph among 30 single-conditionally generated graphs for each condition.

Q2: How oracle rank the molecular graphs

Details are in Appendix B.3 (Lines 520-522)

For how Oracle ranks polymers (Lines 520-522):

We rank these polymers based on the mean absolute error between the generated properties (evaluated by a random forest model trained on all the data to simulate the Oracle function) and the conditional property.

We use ranking positions to measure closeness to target conditions. For multi-conditional generated polymers, their median ranks are 4, 9, and 11 for Synth., $O_2$, and $N_2$ permeability (Lines 57-58, Figure 1).

Q3: Does Graph-dependent noise model enhance validity?

Yes, it improves validity from 0.4946 to 0.8245

Q4: Novelty

Good similarity and distance do not imply bad novelty

Novelty is provided in response to W3.

In Line 204, the reference set consists of left-out test cases unknown during training. Therefore, good similarity and distance to the reference set do not indicate poor novelty, which is typically defined on the training set.

Q5: Learnable dropping embeddings

It is widely used [1,3,4] by DiT [1] and DALL-E 2 [4]. It enhances flexibility for handling missing values and improves training stability by learning representations for null embeddings.

Q6: Have you explored three or more conditions?

Yes, we explored and reported results for up to four conditions in Table 1

The HIV and BBBP datasets contain only one overlapping molecule. We may use learnable dropping embeddings to handle missing condition values. But we cannot obtain enough test cases for multi-conditional evaluations.

Q7: True polymer in Figure 3

The SMILES string is:

NC1=C(*)C=CC(=C1)C1=CC(N)=C(C=C1)N1C(=O)C2=CC=C(C=C2C1=O)C(C1=CC=C2C(=O)N(*)C(=O)C2=C1)(C(F)(F)F)C(F)(F)F

The figure is in the rebuttal PDF and will be updated in the paper.

Reference

[1] Scalable Diffusion Models with Transformers. ICCV 2023.

[2] Genetic Algorithms are Strong Baselines for Molecule Generation. 2023.

[3] Classifier-Free Diffusion Guidance. NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.

[4] Hierarchical Text-Conditional Image Generation with CLIP Latents. 2022.

Thank you for your response. The feedback addressed several of my concerns; however, I still have a few remaining issues. Regarding W1, the conditions $O_2$, $N_2$, and $CO_2$ represent similar conditions without any competitive dynamics. Consequently, I would appreciate further clarification on the performance of the proposed method in relation to GSK3β+JNK3+QED+SA. For W3, Uniqueness and Novely may be flawed. However, a high VUN value does not necessarily indicate high performance of generation, whereas a low VUN value indicates underperformance in the generative process.

W1: Conditions on Gas Permeability

Thank you for your feedback. We believe that conditional generation based on multiple gas permeability properties is a challenging task.

First, the condition space for gas permeability is vast, as noted in Line 26, where gas permeability varies widely, exceeding 10,000 Barrier units. This variability presents significant technical challenges in controlling polymer generation across such a broad condition space.

Second, the task relatedness in multiconditions does not reduce the difficulty but rather increases it. As detailed in Section 4.3, polymers often require high permeability for one gas and low permeability for another, making it challenging for the generation model to capture the crucial differences between various gas permeability properties. The GSK3β and JNK3 properties mentioned by the reviewer also have relatedness, as both are serine/threonine protein kinases.

Finally, we have provided additional results to further clarify our model's performance (best results are highlighted for distribution learning and condition control).

Due to time constraints, we sampled a subset of 600 data points from the kinase dataset provided by the MARS paper and split them into training, validation, and test sets in the same manner as described in the paper. We then generated 1,000 examples for evaluation. The results show that Graph DiT significantly outperforms other methods on the GSK3β and JNK3 properties, with strong distribution matching to the reference set.

W3: VUN Metrics

Our model demonstrates reasonable values on the VUN metrics. We believe that a Validity of 0.9760, Novelty of 0.9702, and Uniqueness of 0.8919 illustrate Graph DiT's ability to successfully model the data distribution of complex molecules.

We agree with that reviewer that Novelty and Uniqueness are important perspectives in evaluating molecular generation models. We would also like to share additional thoughts on drug and material discovery. This field may focus on individual instances that have significant real-world impacts. A model capable of suggesting a single valid, novel, and unique drug or material that satisfies diverse property requirements, even if it produces many invalid ones (resulting in lower VUN metrics), may be promising as well compared to other models that generates numerous valid, novel, and unique suggestions but fails to meet specific condition requirements.

We appreciate the reviewer's follow-up questions and welcome further discussion on any concerns the reviewer feels have not yet been fully addressed.

W1: Scalability of graph dependent noise

Subgraph-level noise model is possible for larger graphs

Thank you for your comment. The transition matrix is manageable for molecules, as we only need to model heavy atoms, which usually number less than 50 for a molecule [1].

For larger graphs, we could explore building the matrix at the subgraph level in the future, treating subgraphs as nodes and maintaining only inter-subgraph edges. This approach could handle larger-scale graphs more efficiently.

W2: Color shades

Thank you for your comment. The discrepancy is due to different alpha values for transparency. We have updated Figure 4 with consistent alpha values and provided it in the rebuttal PDF.

Q1: How graph structure is modeled

Thank you for your question. Graph DiT differs from vision and language Transformers by using graph tokens. Given a node on the graph, our new graph token concatenates node features with all related edge features, preserving node connectivity.

Reference

[1] Molecular sets (MOSES): a benchmarking platform for molecular generation models. Frontiers in Pharmacology. 2020.