Adaptive Temperature Enhanced Dual-level Hypergraph Contrastive Learning

W1: The presentation of the paper requires to be improved heavily. Some examples are as follows:

(i) In the listed contributions, the statements of "Novelty", "Generalization", and "Effectiveness and Robustness" are improper. The authors should state the specific contributions of this work here, e.g., "Novelty"->"A novel adaptive temperature enhanced hypergraph contrastive learning framework".

(ii) In Preliminaries, the statement "The main idea of graph contrastive learning aims to maximize the agreements among positive and negative contrastive pairs over" is not clear. Using a more clear statement "The main idea of graph contrastive learning aims to maximize the similarity between positive pairs and meanwhile minimize the similarity between negative pairs" is better. Also, in Equation (2), sim($\cdot$) is not given a definition or explanation.

(iii) In Methodology, the graph augmentation process itself adds noise to the original graph, so the module with the name "noise-enhanced hypergraph augmentation" is strange. It may be better if the authors replace "noise-enhanced" with "feature distortion based". The symbols of $x_{i}$ and $\delta_{i}$ should be bolded as they represent vectors, not values. The definition of the concatenation operator is ambiguous: sometimes they are denoted as $[\cdot:\cdot]$ and sometimes they are denoted as $\oplus$. In Equation (5), $\eta$ has unclear statements and it represents the learning rate of the temperature adjustment.

(iv) Many contents in the main body are not self-contained, e.g., Table 3 should be moved into the main body.

W2: Lacking necessary analysis of the important loss function. The authors do not analyze the JSD loss in the paper, but why they can conclude that AdT-HyGCL is suitable for JSD loss (note that JSD loss is totally different from NT-Xent loss)?

W3: The presented experimental results have lots of errors. Some examples are as follows:

(i) In Table 1, for Macro-F1 results, the best result should be "64.98 ± 1.77" and the runner-up should be "64.65 ± 0.85" on Citeseer.

(ii) In Table 2, the best result should be "59.65 ± 1.62" on House (Nettack).

These errors make the whole experimental analysis not convincing.

W4: Lacking some important baseline comparison. The paper does not compare AdT-HyGCL with other temperature adjustment methods (e.g., the methods in [1], [2], and [3]), which limits the ability to assess the effectiveness of AdT-HyGCL relative to other methods.

[1] Chen, Jiawei, et al. "Adap-$\tau$: Adaptively Modulating Embedding Magnitude for Recommendation." WWW2023.

[2] Qiu, Zi-Hao, et al. "Not All Semantics are Created Equal: Contrastive Self-supervised Learning with AutomaticTemperature Individualization." ICML2023.

[3] Zhang, Oliver, et al. "Temperature as uncertainty in contrastive learning." arXiv preprint arXiv:2110.04403(2021).

1. The motivation of this work is not good enough. For example, the authors claim that "they (graphs) have limitations in captureing ... higher-order group-wise structures.'' What is the meaning of high-order group-wise structures in this paper? Why is the statement correct? Considering the higher-order structures (such as communities) are widely studied in the field of graph data analysis, it is better to showcase the viewpoints carefully. Furthermore, the authors claim that "current contrastive learning methods over hypergraphs still have limitations in modeling the high-order relationships ... within hypergraphs ...". Then, maybe hyper-hypergraphs, if any, cannot capture the high-order relationshi within hyper-hypergraphs. Is it really interesting? Should we propose another "dual-level contrast mechanism" for the hyper-hypergraphs?

2. The novelty of this study is not clear.

(1) The novelty of the concept of `noise-enhanced' should be clearly presented. The stategy of performing noises to node attribute features has been well utilized in the literature, such as [1]. The authors should carefully discussed them in the manuscript, especially when [1] has been cited as [Zhu, et al. 2021b] in the paper.

(2) The idea of using community structures in constrastive learning has been proposed in the literature, such as [2]. The authors should clearly present this point. In addition, it seems that the ``community'' in this study does not match the well-known community in the literature. Then, it is better to use another phrase in this work to aoivd the ambiguiity.

(3) The idea of the adaptive temperature-enhanced contrast optimization in pre-tained models (including GNNs) has been proposed in the literature, such as [3-6]. The authors should clearly discuss the differences between this work and the literature.

3. The methods of the paper should be carefully presented. Considering the adaptive temperature-enhanced contrast optimization in pre-tained models (including GNNs) has been proposed in the literature, such as [3-6], the authors should clearly present the relationship between the propostions as well as their proofs and the literature. In addition, the proofs of these propositions should be carefully presented rather than giving one or more examples. For instance, it is stated that ``the temperature descents'' in the paper. Why must the temperature descent during the training process for any loss? The authors should prove it seriously.

4. The experiments of the paper should be improved. As shown in Table 1 and Table 2, the experimental results of the proposed AdT-HyGCL are not always the best. Maybe the authors should analyze and present the reasons. Also, the parameter sensitivity experiments should be provided, such as $\tau_{low}$, $\eta$, $\rho$, $\lambda_1$ and $\lambda_2$. In addition, more recent studies on hypergraph contrastive learning shoud be considered as baselines, such as [7][8].

[1] Graph Contrastive Learning with Adaptive Augmentation. Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, L. Wang. WWW 2021.

[2] Graph Communal Contrastive Learning. B. Li, B. Jing, H. Tong. WWW 2022.

[3] Temperature as Uncertainty in Contrastive Learning. O. Zhang, M. Wu, J. Bayrooti, N. Goodman. arXiv:2110.04403. 2021.

[4] Understanding the Behaviour of Contrastive Loss. F. Wang, H. Liu. CVPR. 2021.

[5] Rethinking Temperature in Graph Contrastive Learning. Z. Liu, H. Feng, C. Wang. 2021.

[6] Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Data. A. Kukleva, M. Bohle, B. Schiele, H. Kuehne, C. Rupprecht. ICLR 2023.

[7] Cross-view graph contrastive learning with hypergraph. J. Zhu, W. Zeng, J. Zhang, J. Tang, X. Zhao. Information Fusion. 2023.

[8] Collaborative contrastive learning for hypergraph node classification. H. Wu, N. Li, J. Zhang, S. Chen, M. K. Ng, J. Long. Pattern Recognition. 2023.

Typo:, in Figure 1, Dual-Leve -> Dual-Level Hypergraph Contrastive Learning.

For the citeseer's Macro F1, it seems CHGNN performs slightly better.

And some common datasets like DBLP and Pubmed seems missing in the paper.

For House and walmart, most baseline models performs poorly, probably due to the heterophily intrinsic.

Only one data augmentation method is new, which is HyedgeR. I'm a little concern about the novelty since it's intuitive.

1. The novelty of the proposed framework in this paper is incremental. Both multi-level contrast and adaptive temperature are not novel ideas. The former has been explored in many previous works [1-3], while the latter has been studied in [4]. The authors didn't cite these works, which should be referred to, discussed, and compared.
2. The performance improvement seems limited in Table 1.
3. Table 2 misses several competing baselines in Table 1, like CHGNN.

[1] Duan, Jingcan, et al. "Graph anomaly detection via multi-scale contrastive learning networks with augmented view." AAAI 2023.

[2] Ju, Wei, et al. "Unsupervised graph-level representation learning with hierarchical contrasts." Neural Networks 2023.

[3] Liu, Yanbei, et al. "Multi-Scale Subgraph Contrastive Learning." IJCAI 2023.

[4] Zhang, Hongjun, et al. "Sleeppriorcl: Contrastive representation learning with prior knowledge-based positive mining and adaptive temperature for sleep staging." arXiv preprint arXiv:2110.09966 (2021).

The methodology section of this paper appears to be somewhat limited, particularly in section 4.2.2, which lacks a comprehensive theoretical explanation of 'COMMUNITY-LEVEL HYPERGRAPH CONTRASTIVE LEARNING'. Additionally, it may be beneficial to provide an exposition on how these two branches are integrated to obtain the final representations.