Simple Graph Contrastive Learning via Fractional-order Neural Diffusion Networks

Thank you for the insightful comments and suggestions.

W1. Why FDE for GCL.

A core principle of GCL is to generate diverse views, with novelty in how they are constructed. FD-GCL uses neural diffusion-based encoders governed by FDEs, where fractional order $\alpha$ controls diffusion scale—enabling views with varying locality/globality. FDEs generalize ODEs ($\alpha=1$) but offer greater flexibility. Fixed $\alpha=1$ yields less diverse views, weakening contrastive effect. As a generalization, FDEs perform no worse than ODEs while enabling richer view generation, making them a better choice. See Appendix C and D for details on FDEs and ODEs.

Theoretically, Thm 2 (formal version of Thm 1) proves that embeddings generated with different $\alpha$ are provably distinct, with contrast increasing as differences in $\alpha$ grows. Empirical results (Table R2 for Reviewer ttop) further confirm FDE's advantage over ODE-based methods (GRAND, GraphCON) for all datasets.

Additionally, this is the first work to apply FDE-based diffusion to GCL, offering a novel and flexible mechanism for view generation via tunable diffusion dynamics, despite prior usages of FDEs in GNNs [R1,R2].

[R1] Kang. et al, Unleashing the potential of fractional calculus in graph neural networks with FROND, ICLR, 2024.

[R2] Zhao. et al, Distributed-order fractional graph operating network, Neuips, 2024.

W2. FD-GCL mitigates dimensional collapse

Dimension collapse refers to features being confined to a low-dimensional subspace, which may harm CL performance, though low-dimensional features are not inherently poor. Thus, to fairly compare CL models, model performance remains the primary comparison metric for different datasets. Nevertheless, we have provided evidence that FG-GCL can mitigate dimension collapse. Theoretically (Thm 2 cf. Appendix E), we have shown that for small $\alpha$, the generated features tend to belong to a space spanned by a relatively large amount vectors in the spectral domain, which is empirically supported by the PCA visualization (a set of collapsed features should resemble a delta function). Refer to Fig.(link) for the PCA visualization of PolyGCL (i.e., low-pass and high-pass spectral views), indicates that the number of significant PCA components for FD-GCL (Fig. 2 in our paper) is comparable to that of PolyGCL.

W3. The effect of different diffusion depths ($T$)

As rigorously discussed in Appendix E, increasing diffusion depth $T$ generally enhances view diversity. We empirically evaluate the effect of varying $T$ on both homophilic and heterophilic datasets in Table R5, showing that larger $T$ values consistently lead to improved performance. The values $T$ for each dataset are reported in Table 8 in Appendix F.3.

Table R5. Classification accuracy w.r.t $T$

W4. Mathematical analysis of FDEs

This claim is theoretically supported by Thm 2 (cf. Appendix E), which proves that embeddings generated by FDEs with different fractional orders are provably distinct, with the contrast between them increasing as the difference in $\alpha$ grows. The rigorous mathematical analysis can be found in Appendix E. The relation between Theorem 1 and the claim in the comment has been explained in Sec.4.2 (2nd paragraph of ''distinct views''). Intuitively, for large $\alpha$, we have shown that the spectral are more concentrated on low-frequency components. It is well-known in GSP that generally, a low-frequency signal lacks variation across the entire graph and thus it represents a ''global view''. In addition to the theoretical justification, this claim is also empirically validated. As shown in Fig. 1 and Appendix G.1, node features generated by two FDE encoders with different fractional orders exhibit clearly distinct characteristics. Specifically, a smaller fractional order (e.g., $\alpha=0.01$) leads to embeddings with a concentrated core, whereas a larger order (e.g., $\alpha=1$) yields features that are more evenly distributed across the space.

W5. Tuning of $\alpha_{1}$ and $\alpha_{2}$

See Reviewer ttop's W4.

W6. Lack comparisons with two CL works (e.g., Simgrace and BGRL)

Please note that the requested comparisons have already been included: SimGRACE results are in Table 10, and BGRL comparisons are in Table 1 and 2. Although SimGRACE is specifically designed for graph classification, FD-GCL achieves comparable results on this task. Moreover, FD-GCL surpasses BGRL on node classification across both homophilic and heterophilic datasets, with notable gains on heterophilic datasets (e.g., 28% on Squirrel/Wisconsin, 26% on Texas/Cornell, 5% on Actor/Roman/Arxiv-year).

Thank you for the insightful comments and suggestions.

W1. The novelty of FD-GCL

A general guiding principle for GCL is to generate views from diverse perspectives, with novelty lies in how these views are generated. For example, PolyGCL uses polynomial filters for low-pass and high-pass spectral views, while bandpass graph filters are well studied in GSP and GNN. Analogously, the core novelty of FD-GCL is not merely replacing components in existing augmentation-free or negative-free pipelines, but introducing a new perspective for encoder design: generating distinct views via diffusion dynamics, which aligns with GCL's core principle.

FD-GCL uses neural diffusion-based encoders governed by FDEs, where the fractional order $\alpha$ controls the locality/globality of features. By using different $\alpha$ values, FD-GCL produces views with varying diffusion scales. To our knowledge, this is the first work to apply diffusion dynamics via FDEs for contrastive learning, offering a novel and flexible mechanism for view generation through tunable diffusion rates.

Another technical novelty is the rigorous analysis in Thm 2 (the formal version of Thm 1), which proves that embeddings generated by FDEs with different fractional orders are provably distinct, and that the contrast between them increases as the difference in $\alpha$ values becomes larger. This is the first formal mathematical analysis of this property in GCL, despite the established use of FDEs in GNNs [R1,R2]. This theoretical insight is further supported by numerical evidence in Figure 1 and Appendix G.1, where we examine the FDEs under two widely separated fractional orders ($\alpha_{1}=0.01$ and $\alpha_{2}=1$). The results clearly show distinct feature distributions: a small $\alpha$ leads to embeddings with a highly concentrated core, while a large $\alpha$ produces more evenly spread features.

Moreover, unlike BGRL, CCA-SSG, GraphACL and PolyGCL, which rely on either data augmentation or negative sampling, FD-GCL requires neither, simplifying usage. AFGRL shares this simplicity but is limited to homophilic datasets. In contrast, FD-GCL is straightforward and effective on both homophilic and heterophilic settings (Table R3).

Table R3. Comparison with state-of-the-art GCL methods

[R1] Kang. et al, Unleashing the potential of fractional calculus in graph neural networks with FROND, ICLR 2024.

[R2] Zhao. et al, Distributed-order fractional graph operating network, NeurIPS 2024.

W2. Lack of theoretical analysis & Fig. 3 explanation

The core principle of GCL is to design encoders that generate distinct yet meaningful views. FD-GCL introduces a novel view-generation mechanism via neural diffusion governed by FDEs, where fractional order $\alpha$ controls feature locality versus globality across continuous scales. This mechanism is theoretically grounded, shown in Theorem 2 (cf. Appendix E). We refer to response to W1 for more details.

Fig. 3 shows the unsupervised clustering capability of FDE-based encoders. For each class $c$, we measure clustering quality using the discrimination ratio $r_c = d_c^{\mathrm{inter}}(\text{intra-class distance})/d_c^{\mathrm{intra}}(\text{inter-class distance})$. The higher the ratio, the better the clustering quality. Each curve (one per class) shows that $r_c$ increases and then stabilizes during training. This trend indicates that the encoder gradually enhances inter-class separation while maintaining intra-class cohesion, which is crucial for classification.

W3. Purpose of increasing feature dimensions

By [R3], increasing feature dimension enhances GCL expressiveness by capturing more complex patterns. We use the operator $Y_l = XW_l$ to project features to a higher dimension, and larger dimensions generally improve performance on all graph types (see Table R4).

[R3] Xiao et al, Simple and asymmetric graph contrastive learning without augmentations, Neuips, 2023.

Table R4. Classification accuracy w.r.t feature dimension d

W4. Tuning of $\alpha_{1}$ and $\alpha_{2}$

Refer to Reviewer ttop's W4.

W5. Lack of latest work (e.g., LOHA)

Please note that LOHA (AAAI 2025, Jan. 2025) is concurrent work per ICML 2025 policy; thus, direct comparison is not required. Also, the code for LOHA is not public, hindering timely reproduction.

Thank you for the insightful comments and suggestions.

W1. The novelty of FD-GCL

FD-GCL's novelty is not merely replacing components of existing augmentation-free or negative-free pipelines, but introducing a new perspective for encoder design: generating distinct views via diffusion dynamics, aligning with GCL's core principle. See W1 of Reviewer ZHMy on both architectural and theoretical novelty.

W2. Scalability & efficiency of FD-GCL

Scalability and efficiency are illustrated by training and running time comparisons on the large-scale Ogbn-arxiv dataset, shown in Table 4 in the paper and Table R1, respectively. The results confirm that FD-GCL maintains competitive training and running times, highlighting its practical efficiency. Additional evaluations on other large-scale Roman-empire and Arxiv-year datasets (Table 1 and 2) further support its scalability, with dataset sizes comparable to benchmarks like PolyGCL.

Table R1. Testing time (sec). OOM refers to out of memory on an NVIDIA RTX A5000 GPU (24GB) during training.

W3. How FD-GCL mitigates dimensional and feature collapse

Dimension collapse refers to embeddings being confined to a low-dimensional subspace. In Theorem 2 (cf. Appendix E) , we theoretically prove that using a small fractional order $\alpha_{1}$ mitigates this issue by reducing energy concentration in the spectral domain. Specifically, it shows that $\mathbf{Z}_{\alpha_1}(t)$ is less energy concentrated in the spectral domain,

i.e., it has a decomposition $\sum_{1\leq i\leq N} c_i\mathbf{u}_i$ with many large $|c_i|$, meaning the embeddings span a higher-dimensional subspace. This is further supported by PCA results (Fig. 2), where smaller $\alpha$ values preserve significantly more principal components.

On the other hand, view collapse means generated views converge to similar representations. We interpret the reviewer is referring to view collapse. While existing augmentation-free GCL methods typically rely on architectural changes or explicit regularization to address feature collapse, FD-GCL adopts a fundamentally different approach by leveraging fractional diffusion dynamics. We mitigate this collapse with a regularized cosmean loss (i.e., $\mathcal{L}(\mathbf{Z}_1,\mathbf{Z}_2)= \mathcal{L}_0(\mathbf{Z}_1,\mathbf{Z}_2)+\eta |\langle \mathbf{c}_1, \mathbf{c}_2 \rangle|$) that includes a penalty term on the angle between the dominant directions $\mathbf{c}_1$ and $\mathbf{c}_2$ of embeddings $\mathbf{Z}_1$ and $\mathbf{Z}_2$. This encourages diversity between the views without relying on negative samples. This approach is possible as we have observed that features from each view of FD-GCL tend to have a pronounced dominant component. As demonstrated in Fig. 5 and Fig. 6 in Appendix F.4, this regularization ensures stable performance across training epochs. Combined with our theoretical findings, these results provide strong evidence that FD-GCL effectively avoids feature collapse through its diffusion-based framework.

W4. Tuning of $\alpha_1$ and $\alpha_2$

We refer the reviewers to Appendix F.3. Both $\alpha_{1}$ and $\alpha_{2}$ are tunable within the range $(0,1]$. Motivated by the theoretical insights in Thm 1 (or Thm 2), which suggest that a larger difference between $\alpha_2$ and $\alpha_1$ enhances the contrast between views, we adopt a simple yet effective strategy: we fix $\alpha_{2}=1$ to maintain a consistent global view and tune $\alpha_{1}$ over the range $(0,1]$ using a grid search. This approach is guided by our analysis and provides a practical, computationally efficient solution for tuning $\alpha_l$. The corresponding values $\alpha_{1}$ and $\alpha_{2}$ for each dataset are reported in Table 8 in Appendix F.3. We recognize that more adaptive or data-driven methods for selecting fractional orders are promising. To this end, in our future GCL work, we may adopt variable-order fractional derivatives where the derivative orders depend on hidden features.

W5. Compare with alternative diffusion models.

Note that the fractional diffusion model in FD-GCL generalizes other graph diffusion techniques (e.g., GRAND (cf. (3) in Appendix C) and GraphCON (cf. (4) in Appendix C)) by setting $\alpha_{1}=\alpha_{2}=1$. Table R2 shows this fixed setting yields poorer results on both homophilic and heterophilic datasets, since it produces less distinct views, weakening the contrastive effect. This highlights the advantage of fractional-order diffusion.

Table R2. Classification accuracy on different graph diffusion models.