SDMG: Smoothing Your Diffusion Models for Powerful Graph Representation Learning

Thank you for your constructive and positive feedback; we appreciate your insights and have addressed your comments below.

Question 1 and Weakness 1: Clarification on Masking Strategy Motivation.

The masking strategy has two primary motivations: enhancing robustness and promoting learning discriminative global representations. Recent literature (Daras et al., 2022) on diffusion-based models highlights that carefully designed masking or corruption processes encourage the model to selectively prioritize the most meaningful structural information during reconstruction. Specifically, masking introduces controlled information degradation, forcing the model to reconstruct from limited context, which aligns naturally with our objective of focusing on smooth, globally relevant graph structures. This process prevents the model from simply memorizing trivial local or high-frequency noise features. Consequently, such a masking strategy guides the model to leverage global graph topology, improving performance on downstream tasks through more robust and semantically meaningful representations.

We have also demonstrated the performance of our model without masking; please refer to our response to Reviewer P6bv, Question 1, for detailed comparisons. Based on your suggestion, we will include further clarifications regarding this aspect in the final version of the manuscript.

Question 2: Clarification on Frequency Component Definition.

Thank you for the insightful question. Yes, in Section 3 (the Investigation Section), the "5% lowest frequency components" indeed refers specifically to the eigenvalues of the Laplacian matrix derived from the normalized adjacency matrix, which naturally captures the global structure of the graph.

We adopt the Laplacian spectrum because it reflects the smooth, global signals of the graph. Please note that we perform the graph Fourier transform only in the Investigation section to validate our hypothesis. In the Method section, we instead propose an approximate computation method for extracting low-frequency information, which is more efficient and scalable.

Question 3: How to extend SDMG beyond classification, such as social community detection or traffic flow prediction.

Extending our framework to tasks such as social community detection or traffic flow prediction would involve a few targeted modifications that build on our core approach of low-frequency emphasis and multi-scale smoothing. For social community detection, one could replace the classification head with a clustering-oriented output by incorporating a clustering loss (e.g., modularity maximization or a spectral clustering regularizer) to directly optimize the learned embeddings for community structure, or by using the generated representations as input to an external clustering algorithm. Additionally, integrating community-aware regularizers into the diffusion process could further enhance the capture of global structural patterns.

For traffic flow prediction, a regression task with significant temporal dynamics, the output layer would need to be restructured to produce continuous predictions (using, for instance, a mean squared error loss), and it would be beneficial to integrate temporal modeling components (such as recurrent layers, temporal convolutional networks, or attention mechanisms) into the denoising decoder. These modifications would allow our method to adapt its emphasis on smooth, global features and multi-scale consistency to effectively address the unique challenges of these different tasks.

Weakness 2: Clarification on Balancing MSS Loss and Low-Frequency Encoders.

Thank you for your insightful comment. While Table 3 shows a larger improvement from introducing the low-frequency encoders compared to the MSS loss alone, we believe these components have distinct yet complementary roles. The low-frequency encoders provide a robust initial approximation of the global, low-frequency signals from the graph topology and node features, which are critical for guiding the denoising process. In contrast, the MSS loss fine-tunes the representations by enforcing multi-scale consistency and preventing the reintroduction of high-frequency noise during training. Although its standalone impact may seem smaller, the MSS loss is essential for maintaining overall representation quality over time. Moreover, adding extra low-frequency encoding (e.g., additional GNN layers) to the final node representations would likely be redundant, as our ablation studies show that the best performance is achieved when both the low-frequency encoders and the MSS loss are used together.

Thank you again for your helpful comment. We will incorporate the necessary modifications in the final version.

We appreciate your valuable input and positive comments, and we have carefully addressed your comments in our responses below.

Question 1: Clarification on the Meaning of “–” Symbol on Table 1.

In Table 1, the “–” symbol indicates that the corresponding method either exceeded the available memory, had prohibitive computation time on that particular dataset, or that the original authors did not report performance on that dataset. We will clarify this notation in the revised manuscript to avoid any ambiguity.

Question 2: Clarification the U-Net Representations on Equations (11) and (12).

For Equations (11) and (12), we only concatenate the activations from the U-Net’s up-sampling layers. However, the information from the down-sampling layers is fully preserved through skip connections that directly link these layers to the up-sampling path. As a result, although our final representation is built solely from the up-sampling activations, it inherently incorporates the local features captured during down-sampling, ensuring that both local and global information is maintained without redundancy. Thank you for pointing this out, and we’ll revise the final version to make this clearer.

Question 3: Clarification on the Notation of Encoding “Z” and Node Representation “H”.

In Theorem 3.1, the encoding “Z” typically refers to the intermediate representation produced by our graph encoder. For example, mapping the input graph to a low-dimensional space. On the other hand, “H” denotes the final node representation extracted from the activations of the U-Net. In the context of our paper, these two representations can be considered equivalent. Based on your suggestion, in the revised version, we will harmonize our notation by clearly stating that H is derived from Z, ensuring consistency throughout the paper.

Weakness 1: Clarification on Gaussian Noise Intensity Used in Figure 1.

We appreciate your observation regarding the noise intensity. In our experiments for Figure 1, we sample Gaussian noise from a normal distribution, $\mathcal{N}(0, 5)$, to simulate realistic high-frequency perturbations. We agree that specifying this value explicitly would improve clarity, and we will include these details in the final version. Furthermore, please refer to our response to Reviewer Q4ro, Question 1, where we provide a detailed explanation of the motivation behind Figure 1.

Weakness 3: Clarification on Reconstructing Only Node Features.

Thank you for your insightful comment. As shown in Figures 3(b) and 3(d), we demonstrate that reconstructing the smooth aspects of the graph structure benefits downstream tasks. Accordingly, in our work we also introduce a low-frequency encoder for graph topology to extract global, smooth structural information. Thus, when reconstructing node features, our SDMG essentially leverages both topology and node feature information to capture the distribution of node features. Nonetheless, we agree that an alternating reconstruction strategy for node features and graph topology is an interesting idea, and we plan to explore this approach in future work.

Thank you for your positive feedback. We appreciate your time and have addressed your comments below.

Question 1: Clarification on noise addition in Figure 1.

The primary goal of Figure 1(a) is to illustrate the misalignment between generative reconstruction objectives and representation learning. As shown, reconstructing only a small fraction of information (e.g., the lowest 5% frequency components) is sufficient to achieve competitive downstream performance. Conversely, reconstructing the full frequency spectrum (100%), as most current methods do, results in performance degradation. We attribute this to diffusion models allocating their limited information capacity to unnecessary reconstruction of high-frequency noise.

To further confirm this hypothesis, we performed the experiment shown in Figure 1(b), where we explicitly added high-frequency Gaussian noise with variance $\sigma^2 = 5$, specifically sampling from $\mathcal{N}(0, 5)$. The rationale is that, if the diffusion model could effectively prioritize reconstructing task-relevant (low-frequency) information under limited capacity, its performance should remain robust despite the addition of irrelevant high-frequency noise.

Indeed, in Figure 1(b), we observe that models focusing solely on reconstructing a small fraction of low-frequency information remain robust, maintaining strong downstream performance even after noise injection. In contrast, reconstructing the full frequency spectrum leads to a significant performance drop under added noise conditions. This clearly indicates that under limited capacity, enforcing full reconstruction exacerbates the misalignment between generation and representation. Our theoretical analysis in Theorem 3.1 further supports and explains this phenomenon.

Thank you for your insightful comment. We will revise the final version accordingly.

Question 2: Clarification on Interpretability implications of focusing on low-frequency signals.

Low-frequency components summarize global structural information and thus can offer valuable interpretive insights in certain applications. For instance, in relatively homogeneous networks such as social networks, low-frequency signals help explain group memberships in clustering tasks by highlighting shared global characteristics among nodes. However, interpretability is inherently task- and context-dependent. In applications like biological anomaly detection or extreme event prediction, relying solely on low-frequency information may not provide sufficient interpretability, as critical insights could reside in higher-frequency or local features.

Question 3: Clarification on the Aggregation Mechanism in MSS loss.

The construction of $\mathbf{h}$ in the $R_{MSS}$ loss (Eq. 13) indeed employs an aggregation strategy similar to those used in standard GNN layers. Note that, we leverage this aggregation within our proposed MSS loss specifically to capture multi-scale reconstruction information across different neighborhood ranges. By explicitly comparing node representations at multiple neighborhood scales, our MSS loss uniquely encourages the model to prioritize smooth, global (low-frequency) signals during reconstruction, which differentiates it from conventional aggregation strategies that primarily aim for node-level feature updates.

Weakness: Clarification on Frequency selection and potential extensions.

We acknowledge the reviewer’s point regarding tasks that might benefit from high-frequency information. While our current approach intentionally emphasizes low-frequency signals due to their effectiveness in mitigating the misalignment between generation and representation, we do not exclude the potential value of mid- or high-frequency components. Instead, we view our method as providing a foundational perspective that future studies can adapt or extend based on specific task requirements. Future work may incorporate adaptive frequency selection mechanisms, which could better accommodate scenarios such as anomaly detection or extreme event prediction.

We thank you for taking the time to review our manuscript and for the suggestions that have helped clarify our original contributions. Due to its overarching importance, we would like to start with Weakness 2:

Novelty

Our work is not on GRL as such, but GRL for use in a downstream classification task. As highlighted by Reviewers 9RUT and DKTW we are the first to consider potential misalignment and possible solutions in this context. Specifically：

1. Identifying and analyzing the misalignment between “graph generation” and “graph representation”.

   • Existing diffusion-based GRL typically seeks to match the entire frequency spectrum, including irrelevant details. From both theoretical and empirical perspectives, we show that this reduces performance on downstream tasks.

2. Proposing a novel multi-scale smoothing (MSS) loss that mitigates misalignment between generation and discriminative representation.

   • Even if one applies known LF filters (e.g., by truncating high frequencies or using a GNN), relying on MSE for element-wise reconstruction reintroduces high-frequency noise. (see Figure 4 and Theorem 3.1). Our MSS loss overcomes this by enforcing stronger penalties on LF errors throughout the diffusion steps. This mechanism is new and, from a theoretical standpoint (Theorem 4.1), leads the reconstruction toward LF signals without entirely suppressing useful high-frequency components.

Thus, we emphasize that the key contribution is not about which LF encoder one employs (we use a mature and efficient GNN simply because it is practical). Rather, it is the combination of LF and our MSS that resolves the inherent misalignment between generation and representation.

We thank you for your comments and will revise our manuscript to reflect this more clearly.

Question 1 and Issue 1: Performance Without Masking

Below are our node/graph classification results without any masking. As shown, our method consistently improves over DDM on all datasets, with especially substantial gains in graph classification.

Node Classification

Graph Classification

Question 2 & Issue 2: Statistical results (p-value)

In our paper, we report the widely used mean and standard deviation as standard performance statistics. However, we are also happy to include more statistical tests (e.g., Wilcoxon Signed-Rank Test) with p-values. In general, p < 0.05 indicates statistically significant improvements; while some datasets (e.g., Cora) show p > 0.05, most meet p < 0.05.

Question 3 & Weakness 1: Why Diffusion Model Benefits

Thanks for your comment. We do not merely “insert” a low-frequency (LF) encoder and a multi-scale smoothing (MSS) module into generative models. Rather, we exploit the diffusion process’s iterative noise-injection–denoising scheme: at each timestep t, varying noise levels enable our MSS term to progressively emphasize coherent LF signals. This contrasts with single-step models (e.g., GraphMAE), which lack such iterative noise scheduling. While our MSS approach can be generalized elsewhere, the table below shows that it yields larger gains in our multi-step diffusion framework, aligning with our core viewpoint.

Question 4 & Issue 3: Runtime Efficiency
Our method does not suffer from excessive runtime overhead (training + inference)—in fact, it often converges faster. By prioritizing global smooth signals (e.g., via our MSS), SDMG quickly encodes downstream-relevant information and requires fewer total epochs. All experiments were run on a single NVIDIA H100.

Weakness 3: Notation Clarification

Thank you for the suggestion. As noted (page 6, line 290), $\mathcal{E}^x_{\theta}$ is a GAT encoder for LF node features. $f = \mathcal{E}^A_{\phi}$ is an MLP for topology. $\mathcal{A}$ is a learnable embedding matrix approximating the LF spectrum. $f$ can take various graph-structure inputs (e.g., adjacency), and we use random-walk positional encodings for preserving LF information. We will expand these details (including the mask strategy) in the final version.