Dynamic and Chemical Constraints to Enhance the Molecular Masked Graph Autoencoders

Thank you very much for your valuable feedback and suggestions. We carefully address each point below.

---

Q1. _"SimSGT-DyCC surpasses the 'no pretrain' model by 10.4%." The average difference between the two models is actually 7.1% according to Table 1...

Thank you for pointing this out. The reported 10.4% refers to the relative improvement in the average ROC-AUC between SimSGT-DyCC and the "no pretrain" model:

$\frac{74.1-67.0}{67.0} \approx 10.4 \%$

Here, 74.1 ("SimSGT-DyCC") and 67.0 ("no pretrain") are the average ROC-AUC scores across the eight MoleculeNet classification datasets (Table 1, L257-L263).
Note: dataset-level scores are rounded to two decimal places in the table, but the relative difference was computed using unrounded values.

---

Q2. "Substantial performance variations were observed among the original AttrMask, MoleBert, and SimSGT models due to differences in tokenizers.": why is a difference of approx 2% (according to Table 1) considered substantial?

Thank you for raising this important question. While a ~2% absolute performance gap may seem modest at first glance, we consider it substantial in the context of molecular representation learning for the following reasons:

1. Small gains are meaningful on saturated benchmarks.
   The MoleculeNet benchmarks we evaluate on (e.g., Tox21, HIV, BBBP) are well-established and highly competitive. Recent state-of-the-art (SOTA) works often highlight consistent improvements of 1–2% ROC-AUC as meaningful. For example, SOTA models evolved from ~71% (AttrMask, GraphCL, 2020) to ~74% (MoleBert, 2023) and 75% (SimSGT/StructureMAE, 2024). Our best model, SimSGT-DyCC (GraphTrans), further reaches 75.5% (Appendix Table 6).

2. Broad improvements across diverse tasks.
   Beyond the eight MoleculeNet classification datasets (Table 1), DyCC consistently improves performance on 4 regression tasks, 2 drug–target affinity prediction tasks, and 3 quantum-mechanical property tasks (Appendix Tables 5 & 7, L532–L544). These consistent gains demonstrate DyCC’s generality rather than isolated improvements.

3. Statistical significance and consistency.
   We conducted five independent runs for each model configuration. Even though the absolute improvements range from ~1.5% to ~4.3%, all differences are statistically significant and consistent across datasets and tasks.

   Five-run results:

   Model	Run1	Run2	Run3	Run4	Run5
   AttrMask	70.8	70.1	70.9	70.9	70.7
   AttrMask-DyCC	73.7	73.7	73.8	73.6	73.8
   Mole-BERT	72.2	72.1	72.1	72.4	72.3
   Mole-BERT-DyCC	73.8	73.8	73.9	73.7	73.8
   SimSGT	73.0	73.2	72.9	73.0	73.1
   SimSGT-DyCC	74.1	74.3	74.2	74.2	74.0

   Statistical analysis:

   Comparison	p-value	Mean Baseline	Mean DyCC
   AttrMask vs AttrMask-DyCC	0.000038	70.68	73.72
   Mole-BERT vs Mole-BERT-DyCC	0.000052	72.22	73.80
   SimSGT vs SimSGT-DyCC	0.000072	73.04	74.16

4. Real-world impact in virtual screening.
   In practical applications such as virtual screening, even a 1–2% ROC-AUC improvement can translate into significantly higher hit rates and substantial reductions in experimental cost.

---

Summary:
Although a ~2% absolute difference may seem small in isolation, in the context of mature molecular property prediction benchmarks, such improvements are systematic, statistically significant, and practically important. This fully justifies our use of the term “substantial” and further motivates the tokenizer-agnostic DyCC framework.

---

Q3 The success rate of GIBMS was 74%, demonstrating the effectiveness of the GIBMS module.": this is a promising result, but can you compare it to other baselines (in identifying relevant components of a graph) to get a sense of its importance?

Thank you for this suggestion. Most prior subgraph identification methods are supervised and thus cannot be directly compared to our unsupervised setting. The method most similar to ours is RGCL [1], which uses Rationale-aware Graph Contrastive Learning to identify important nodes in a pretraining dataset, also in an unsupervised manner.

For reference, we also report the performance of PGExplainer, a post-hoc explanation model for supervised GNNs, on the same task.

These results indicate that, under the unsupervised setting, our proposed GIBMS more effectively identifies important nodes compared to RGCL. Its performance is lower than the supervised PGExplainer, which is expected because GIBMS is applied during pretraining on task-agnostic datasets rather than task-specific fine-tuning.

[1] Li, Sihang, et al. "Let invariant rationale discovery inspire graph contrastive learning." International conference on machine learning. PMLR, 2022.

Q4: Presentation: I found the way you motivated your approach confusing, especially in the introduction. If I understand your text correctly...

Thank you very much for your insightful feedback. We acknowledge that the motivation in the introduction could have been presented more clearly. In the original version, we decomposed the limitations into three chemically motivated constraints (CC1–CC3) to emphasize specific aspects often violated by existing approaches. However, as you correctly pointed out, these ultimately boil down to two key challenges in masked graph modeling for molecules:

1. Lack of adaptivity in masking strategies across molecules (corresponding to CC1 & CC2);

2. Ambiguity in reconstruction targets due to inherent chemical variability (corresponding to CC3).

Our intention for presenting CC1–CC3 was to analyze the limitations from a chemistry-centric perspective, which might resonate better with chemical researchers. From a model design perspective, however, summarizing them as the two principles above is indeed clearer and easier to follow.

We appreciate your suggestion and will revise the Introduction to explicitly map each chemical constraint to the corresponding design component:

• CC1 & CC2 → Lack of adaptivity → GIBMS

• CC3 → Ambiguity in reconstruction targets → SLG

This restructuring will make the intuition–solution mapping clearer and ensure that readers from different research backgrounds can quickly grasp the motivation and contributions of our work.

Q5 Pointing out errors in the text

Thank you very much for your careful reading of our paper. We appreciate you pointing out the typos and providing helpful suggestions for corrections. These improvements are valuable and will certainly enhance the overall quality of the manuscript.

Q6 Can you report the training time as well in Appendix C.1?

Our experiments were conducted on an NVIDIA DGX A100 server. The GIBMS module, which is a reusable component, requires 683 minutes for training. For AttrMask, integrating the SLG module increases the training time from 512 minutes (baseline) to 561 minutes. We have included these details in Appendix C.1 in the revised version.

Q7 In figure 3, you mention that SGL yields a more uniform distribution of tokens on the ZINC dataset, when the data has 95% of atoms with the 3 most common labels. Is a uniform distribution desired here?

Thank you for this thoughtful question. As noted in the paper, the ZINC dataset exhibits a highly skewed atom distribution, with 95% of atoms belonging to the three most frequent types (e.g., C, N, O). This extreme imbalance can make the reconstruction task overly simple and limit the model’s ability to learn to distinguish rare atom types.

Our goal in introducing the Soft Label Generator (SLG) is not to enforce a perfectly uniform distribution, but rather to mitigate class imbalance, thereby improving the model’s generalization and robustness. Specifically, SLG:

• Maps each reconstruction target from a single hard label to a soft-label distribution over learnable prototypes, increasing the complexity of the reconstruction task and improving context modeling;

• Reduces the tendency to overfit dominant classes and encourages the model to better recognize long-tail (rare) classes;

• Provides a more diverse expression space at the prototype level (as illustrated in Figure 3(c)), which compensates for the limited expressiveness of a small token vocabulary.

This diversity manifests as high prototype perplexity in the early training stages (i.e., more uniform prototype usage), which gradually converges as the model learns effective representations.

Therefore, the “more uniform token usage” brought by SLG should be understood as a mechanism to dynamically adjust reconstruction difficulty and increase representational diversity, rather than an attempt to impose a strictly balanced distribution. We have further clarified this point in the revised manuscript.

---

We hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review.

Thank you very much for your valuable feedback and suggestions. We carefully address each point below.

---

Q1. I have some concerns about the performance improvements reported in Table 1. Given the magnitude of the variance, can it be confidently stated that the proposed method yields a statistically significant "boost" in performance?

Thank you for this question. We have repeated all experiments five times and report the results below:

We also conducted a paired t‑test between baseline and DyCC‑integrated models:

As shown above, the performance improvements brought by DyCC are statistically significant (p < 0.001), even though the absolute improvements range from ~1.5% to ~4.3%. This confirms that the observed gains are consistent and unlikely to be caused by random fluctuations.

---

Q2 In Appendix Table 5 (DTA regression experiments), it appears that models such as Mole‑BERT and AttrMask were not evaluated using the proposed method. Was there any practical difficulty in applying the method to these models? If so, this could constitute a limitation of the proposed approach and should be mentioned explicitly.

Thank you for pointing this out. We have now evaluated Mole‑BERT‑DyCC and AttrMask‑DyCC on the DTA regression tasks and present the results below:

The results demonstrate that, after incorporating DyCC, both Mole‑BERT‑DyCC and AttrMask‑DyCC consistently achieve better performance. This further validates the generality of our approach and shows that DyCC can be applied to existing masked graph autoencoder (MGAE) methods without practical difficulty.

---

Q3 One of the key advantages of the proposed method is its reduced reliance on arbitrary mask selection. Conversely, has there been any experimental verification of how much performance fluctuates in existing methods when masks are chosen arbitrarily? If such analysis exists, it would strengthen the motivation for your approach and help quantify its benefits more precisely.

We appreciate this insightful suggestion. Quantifying the sensitivity of existing MGAE methods to arbitrary mask choices indeed strengthens the motivation for DyCC.

As shown in Fig. 1(a) of the paper, we varied the mask ratio for standard MGAEs (AttrMask, Mole‑BERT, GraphMAE) and observed a non‑monotonic trend: performance first improves and then degrades as the ratio increases. This confirms that fixed, arbitrarily chosen mask settings can significantly affect result

For instance, Mole‑BERT (MAM) reports the performance under different masking ratios, as shown below. The results clearly indicate that the choice of masking ratio has a non‑trivial impact on performance, with 0.20 yielding the best result:

Similarly, Fig. 1(b) varies the tokenizer difficulty (K‑hop). The results indicate that “harder” targets do not uniformly help, implying that performance fluctuations stem from the proxy‑task specification rather than the model capacity itself.

These findings highlight the issue our dynamic masking mechanism (GIBMS) aims to resolve: it adaptively adjusts mask selection and target generation during training, eliminating instability and suboptimality caused by arbitrary fixed settings.

---

We hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review.

Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.

---

Q1. Weakness 1. The major weakness is that the baseline methods are weak and outdated. The proposed method is compared to quite a lot self-supervised graph pretraining models, but the results are from MoleBert [41], which is published in 2023. After the publication of [41], self-supervised pretraining for molecular representation learning has advanced rapidly and quite a lot new methods have been proposed. A search of papers citing [41] can find some of them. Here I list several papers that the authors may consider.
[1] Hongxin Xiang, et al. IJCAI 2024
[2] Liang Wang, et al. ICLR 2025
[3] Yue Wan, et al. Nature Communications 2025

Thank you for pointing this out and for listing several recent works. We agree that the baseline set can be further strengthened and have updated the experiments accordingly.

1. Expanded baseline set. In addition to the MGAE‑style methods originally reported (AttrMask [2020], GraphMVP [2021], Mole‑BERT [2023], and SimSGT [2024]), we have added the recent models you suggested (IEM‑GraphMVP, IEM‑Mole‑BERT, StructMAE‑P/L). The updated table below reports their numbers on the same scaffold‑split evaluation protocol as used in our paper:

The strongest average result among the new baselines is 75.1 (StructMAE‑L), which is still lower than our SimSGT‑DyCC‑GraphTrans (75.5) (see Table 6).

2. Scope clarification. Our primary goal is not only to chase SOTA, but to diagnose limitations of existing molecular masked pre‑training paradigms (e.g., fixed masking ratio, rigid reconstruction targets) and propose a general‑purpose solution (DyCC) that can be plugged into a variety of MGAEs.

3. References and discussion. We will add the recent works you suggested (IEM‑GraphMVP, IEM‑Mole‑BERT, StructMAE) and others published after 2023 into the related‑work section and discuss how DyCC differs conceptually.

In summary, the updated results show that our approach remains competitive or superior to recent methods, while being a model‑agnostic plug‑in rather than a full‑stack redesign.

---

Q2. This paper lacks a review of self-supervised molecular pre-training techniques. A comprehensive review of this field can place the current research in the correct context.

Thank you for this constructive suggestion. We agree that a more comprehensive overview will improve the clarity and positioning of our contribution.

In Section 2, we currently review two main paradigms of self‑supervised molecular graph pretraining:

• Graph contrastive learning, e.g., GraphCL, JOAO, GraphLOG, RGCL, SimGRACE

• Graph generative (masked) learning, e.g., AttrMask, GraphMAE, Mole‑BERT, SimSGT

These representative methods are directly relevant to our proposed DyCC framework. In addition, Appendix C.3 provides detailed descriptions of all baselines used in Table 1.

That said, we acknowledge that the field has been evolving rapidly, and several recent works were not explicitly covered. In the revised manuscript, we will expand Section 2 into a broader taxonomy of self‑supervised molecular pretraining, including multi‑view/multi‑modal methods, contrastive learning, 3D‑aware and SE(3)‑equivariant models, and generative objectives.

---

Q3. The proposed method is only evaluated on one dataset.

We respectfully clarify that our method is evaluated on multiple datasets and task types, not just a single dataset:

• In Section 4 (Table 1), we report results on eight classification tasks from MoleculeNet.

• In Appendix Table 5, we further evaluate on four molecular property regression tasks and two drug–target affinity (DTA) regression tasks.

• In Appendix Table 7, we also report results on quantum chemistry property prediction tasks.

These evaluations span different domains (classification, regression, and quantum property prediction), demonstrating the robustness and generality of the proposed method.

---

Q4. There is no comparison to conventional masking strategies with proper hyper-parameter selection.

We would like to clarify that for AttrMask, Mole‑BERT, and SimSGT, the masking strategy is primarily controlled by the masking ratio. The authors of these methods have already conducted extensive hyper‑parameter studies in their original papers, and we follow their default settings to ensure fair and consistent comparison.

For example, Mole‑BERT (MAM) reports results across multiple masking ratios in Table 7 of the original paper (reproduced below), showing that 0.20 is near optimal:

Our AttrMask‑DyCC, Mole‑BERT‑DyCC, and SimSGT‑DyCC experiments keep the same experimental setup and hyper‑parameter configurations as the corresponding baselines. We did not conduct additional hyper‑parameter searches to avoid introducing bias and to ensure a fair comparison.

---

Q5. Does the proposed method bring high time cost to pretraining?

We appreciate the reviewer’s concern.

Theoretical analysis: Our method introduces two lightweight modules—GIBMS and SLG.

• GIBMS is trained once and reused across all runs, so its cost is amortized.

• SLG only adds two matrix multiplications (Eq. 14 and Eq. 15: $\mathbf{H}^y \cdot \mathbf{Q}$ and $\mathbf{H}^p \cdot \mathbf{Q}$), which introduce negligible computational overhead.

Empirical results: All experiments were conducted on an NVIDIA DGX A100 server.

• GIBMS training takes 683 minutes (one‑time).

• Adding SLG to AttrMask increases pretraining time from 512 min to 561 min, a modest 8.7% increase.

Thus, the proposed method achieves performance gains with only a small and manageable time cost.

---

We hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review.

Thank you very much for your valuable feedback and suggestions. Below, we address each point in turn.

---

Q1. I remember a similar paper by Liu et al. called "Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders". ...

We sincerely thank the reviewer for this careful observation. We clarify the issue and its impact in detail below:

1. Why the differences exist. The baseline results reported in our Table 1 (e.g., for AttrMask) were taken from the reproduction experiments of Mole‑BERT for consistency. As you correctly observed, these numbers differ slightly from those reported in StructMAE (e.g., 70.8 vs. 71.1 for AttrMask). Similar discrepancies can also be seen for other baselines such as GraphMAE. Unfortunately, neither Mole‑BERT nor StructMAE provides full baseline reproduction code, which makes it difficult to align every implementation detail (e.g., random seeds, preprocessing, training hyperparameters). In future updates, we will re‑evaluate all baselines using the official codes of these works (where available) to further minimize such inconsistencies.

2. Impact on our conclusions. We would like to emphasize that these differences do not affect the validity of our conclusions:

• Regardless of the baseline number (AttrMask = 70.8 in Mole‑BERT vs. 71.1 in StructMAE), our DyCC‑enhanced model AttrMask‑DyCC achieves 73.7, consistently outperforming the baseline.
• Across all baselines cited in these works, the best performance reported by StructMAE is 74.0 (Mole‑BERT). This is still lower than the best result achieved by our framework: 75.5 using SimSGT‑DyCC‑GraphTrans (Appendix Table 6).
• The core focus of this paper is to demonstrate that DyCC is a plug‑and‑play framework that improves a range of existing MGAE models. We integrated DyCC into AttrMask, Mole‑BERT, and SimSGT, and in all cases it significantly improved the underlying baseline model.

3. Clarifying Mole‑BERT and SimSGT settings in Table 1. We apologize for any confusion caused by differences in the reported numbers and the original papers. In Table 1, we intentionally excluded auxiliary components unrelated to masked atom modeling in order to evaluate DyCC more fairly:

• Mole‑BERT includes two modules: MAM (Masked Atom Modeling with VQ‑VAE) and TMCL (Triplet Masked Contrastive Learning). Since TMCL is contrastive learning–based and outside the scope of our study, we only report the MAM results in the main text. Full results including TMCL are provided in Appendix Table 6.

• SimSGT uses a Graph Transformer backbone by default, while other baselines (e.g., AttrMask, Mole‑BERT) use GNN backbones. To avoid backbone effects, we report the GNN‑backbone variant of SimSGT (also reported in the original SimSGT paper) in the main text. Results with the original Graph Transformer backbone are reported in Appendix Table 5.

In the revised manuscript, we will update the naming of these variants to make this clearer.

4. Comparison with StructMAE StructMAE also focuses on node importance but differs fundamentally from our GIBMS module in its motivation and design:

Importance scoring: StructMAE computes a structural importance score for each node either using:

• Pre‑defined heuristics such as PageRank (StructMAE‑P), or
• A lightweight learnable scoring network trained jointly with the model (StructMAE‑L).

Principle: In contrast, GIBMS is derived from the information bottleneck principle, focusing explicitly on preserving semantic completeness in molecular graphs.

Training vs. reuse : StructMAE jointly learns masking scores during training for each model, whereas GIBMS is trained once and can be reused across different backbones, making it more modular.

Masking curriculum : StructMAE adopts an “easy‑to‑hard” curriculum (gradually masking more important nodes), which we do not use.

Additional modules: We further introduce the SLG module to mitigate label imbalance, which StructMAE does not consider.

Focuses: Our method focuses specifically on the molecular domain, emphasizes cross‑model generality, and can be integrated into multiple MGAE models, while StructMAE is designed as a stand‑alone general‑graph pretraining model.

Performance comparison: When integrated into SimSGT, DyCC achieves a mean ROC‑AUC of 75.5 on MoleculeNet, surpassing StructMAE’s best result (75.1).

To summarize:

• The baseline discrepancies arise from differences in sources (Mole‑BERT vs. StructMAE) and unavoidable implementation details.

• These differences do not alter our conclusions, as DyCC consistently outperforms the baselines regardless of the exact baseline number.

• Table 1 uses simplified settings (MAM‑only for Mole‑BERT, GNN‑backbone for SimSGT) to enable a fairer comparison.

• Compared to StructMAE, GIBMS has a stronger theoretical foundation (information bottleneck), is reusable, and is tailored for molecular graphs.

Q2. Can you evaluate whether the atom importance scores are stable across different pretraining runs, or does each run select a different set of “important” atoms?

We thank the reviewer for raising this point.

1. Stochastic sampling design.
   The GIBMS module learns an importance score for each atom, but the masking itself is sampled stochastically using a Gumbel–Sigmoid reparameterization:

$\begin{aligned} &\lambda=\operatorname{Sigmoid}\left(\frac{1}{t} \log \left[\frac{p}{(1-p)}\right]+\log \left[\frac{u}{(1-u)}\right]\right)\\ \end{aligned}$

where $t$ is the temperature hyper‑parameter. Thus, the learned scores are deterministic once the model is trained, but the actual mask selections during training are stochastic by design.

2. Stability evaluation.
   To directly assess whether the learned importance scores are stable across pretraining runs, we retrained GIBMS three times with different seeds and repeated the Section 4.3 MUTAG functional‑group identification test, using detection accuracy of the mutagenic groups (NH2_2/NO2_2) as the metric. The accuracies across runs were 0.74, 0.72, and 0.75, demonstrating a high degree of consistency.

3. Conclusion.
   This consistent performance suggests that GIBMS produces stable importance scores across different pretraining runs, enabling reliable identification of key functional groups, even though the masking process itself is stochastic.

Q3 The soft label approach somewhat reminds me on vector quantization (VQ), which is also often used to bridge regression methods and discrete tokens. Can you elaborate why your softlabel approach works better and what makes it different?

We sincerely thank the reviewer for this insightful question regarding the relation between our soft label approach and vector quantization (VQ). Indeed, Mole‑BERT adopts a VQ‑style method. While both approaches share the idea of using prototype‑based representations, there are fundamental differences in their motivation, mechanism, and application, as detailed below:

1. Purpose and Conceptual Distinction
   VQ is primarily designed to quantize continuous representations into discrete codes through a hard nearest‑neighbor assignment to a fixed codebook, often with commitment losses to stabilize training. This makes VQ well‑suited for bridging continuous and discrete representations (e.g., in VQ‑VAE). In contrast, our method does not quantize representations. Instead of selecting a single prototype, we retain the entire soft distribution over prototypes as the target. This preserves uncertainty and provides richer supervision via soft targets, which is particularly beneficial in masked atom modeling, where multiple reconstructions can be semantically plausible.

2. Soft Labels vs. Hard Assignments
   VQ’s hard assignments inherently discard uncertainty and can introduce gradient mismatch. Our approach deliberately avoids hard assignments by using a temperature‑controlled softmax over cosine similarities to prototypes.

3. Task‑Specific Motivation
   Masked atom modeling often involves chemically similar atoms as potential reconstructions. Hard targets (as in VQ or standard classification) ignore this nuance. Our soft label design explicitly leverages temperature‑scaled similarity to learnable prototypes, generating probabilistic targets that better capture chemical variability.

4. Prototype Utilization and Regularization
   To further enhance learning, we introduce Mean Entropy Maximization (ME‑MAX) regularization, which encourages balanced prototype utilization and prevents collapse. This regularization is not typically considered in VQ approaches.

5. Empirical Effectiveness
   As shown in our experiments, the proposed soft label method consistently outperforms hard label (e.g., AttrMask) and VQ‑style (e.g., Mole-Bert) baselines. We attribute this to its ability to encode uncertainty and chemical similarity through soft supervision.

Summary:
While both methods involve prototypes, our soft label approach fundamentally differs from VQ by eschewing quantization, embracing soft distributions as supervision, and explicitly modeling uncertainty. These differences yield richer training signals, more robust prototype usage, and ultimately better generalization, particularly in the context of molecular masked modeling.

---

Q4. I think the $\alpha H\left(s^{\bar{y}} p\right)$ in equation 17 should be $\alpha H\left(\bar{s}^{\bar{p}}\right)$ but I am not sure. The formula seems to have a typo, though.

We sincerely thank the reviewer for carefully checking our equations. You are correct that there is a typo in Equation 17. The correct formulation should be:

$$\alpha H\left(\bar{s}^{\bar{p}}\right)$$

instead of

$$\alpha H\left(s^{\bar{y}} p\right)$$

---

We hope these explanations and corrections satisfactorily address your concerns. Thank you again for your thoughtful review.