ELU-GCN: Effectively Label-Utilizing Graph Convolutional Network

• Novelty and motivation. Several works have studied the GNN with label propagation, e.g., GCN-LPA, which has also be adopted as baseline.
• Following the first comment, the paper overstates its contributions, as it is not the first to address GCN limitations in label information utilization.
• According to Definition 2.2, if "the prediction of GCN is consistent with the output of LPA," one might question why not directly use LPA predictions, given their importance. And this may also be the reason why this work does not provide superior performance (compared to the real SOTAs), as Definition 2.2 gives a false direction to GCN.
• Technique contribution is minor, generally speaking, this work is a supervised graph contrastive learning with LPA as data augmentation.
• So many baselines within the graph structure learning and graph contrastive learning domains are missing, weakening comparative analyses.
• Detailed experiment setup is missing, such as the GCN layer, the learning rate, the training epoch and so on.
• It is unclear how baseline results were obtained, particularly as the NodeFormer results are significantly lower than reported in its original paper.
• In Sec. 3.2.4, the definition of generalization gap is not a good metric for model generalization ability, as there is a trivial solution when the training loss and validating loss are the same without convergence. Small loss gap does not necessarily lead to good generalization ability.
• Actually, as shown in Figure 4, the validation loss of ELU-GCN is worse than GCN, which may even suggest that GCN is much better.
• No hyper-parameter analysis for $\beta$ in Eq.7|8|9, threshold in Line 256, and $\eta$ in Eq.11.

1. The abstract should be enriched with a brief description of the shortcomings of the past methods and the innovative nature of the proposed method.
2. In the introduction, LU-GCN is mainly divided into three categories, but it is also mentioned that the above two methods ignore the critical importance of neighboring label utilization in semi-supervised scenarios, which creates ambiguity.
3. The classification accuracy results in Fig. 2 are introduced abruptly, without explaining the experimental setup.
4. The parameter sensitivity of the objective function and the convergence analysis experiment can be added appropriately.
5. The visualization of Fig. 3 can be compared by adding some plots from the ablation experiments.
6. the evaluation indexes used in Table 1 and Table 2 are not reflected, and the evaluation indexes are relatively single.

See weaknesses.

Q1. Confirmation of symbols.

• "The first $m$ points $x\_i\, (i \leq m)$ are labeled as $Y\_l$": Does the subscript $l$ mean that the entire $Y$ has a label? Or is it an index for traversal? Because the first part writes each $\mathbf{x}\_i$ individually, and the second part writes the whole $\mathbf{Y}\_l$, it makes people think that it might be a typo of $\mathbf{Y}\_i$.
• "Given a classification function $f:\mathbf{X}\to\mathbb{R}^{n\times c}$, ..." : it looks like it should be changed to $f:\mathbb{R}^{n\times n}\to\mathbb{R}^{n\times c}$.
• $V\_l$, $f\_{ik}$ and $f\_{jk}$ in Eq. (1): What do these notations denote? Authors should introduce them.
• Eq. (10): Since it is a logarithmic form, it is recommended not to use the fractional form, which is indeed not conducive to understanding. In addition, according to line 188 in test-flan.py in the anonymous link, $e^{\overline{\mathbf{P}}^\top\overline{\mathbf{P}}+\tilde{\mathbf{P}}^{\top}\tilde{\mathbf{P}}}$ should be taken element by element instead of the matrix exponential. The author should modify the form or make special instructions.

intra_c = torch.exp(F.normalize(intra_c, p=2, dim=1)).sum()

Q2. Theorems and definitions are ambiguous.

• Theorem 2.1: Can this theorem be expressed in formula form?
• Definition 2.2: The GCN effectively utilizes label information if the prediction of GCN is consistent with the output of LPA, i.e., $V\_{\mathrm{ELU}} = \\{V |`LPA`(\mathcal{G}) = `GCN`(\mathcal{G})\\}.$ It seems that Def 2.2 is a definition of a GCN, but Eq. (2) seems confusing, is the GCN should be consistent with the output of LPA on any graph $\mathcal{G}$, or the $\mathcal{G}$ is a given one? Moreover, what does Eq. (2) mean? Does it mean a subset of the whole node set $V$ belonging to the given $\mathcal{G}(V,E,\mathbf{X},\mathbf{Y})$? The current definitions are too vague, and I would be very happy if the author could explain to me in detail what they really mean.

Q3: The derivation of Eq. (6) seems to lack some support.

• "..., the second term can make the subsequent matrix inversion more stable. " What is the subsequent matrix, it seems to refer to $(\mathbf{HH}^\top+\beta \mathbf{I})$ in Eq. (7)? Or the matrix $S$? Judging from the proof in the appendix, I tend to believe the former explanation. It seems that this is the form of Eq. (7) first and then Eq. (6) pieced together. Why choose this unnatural form in the narrative?
• Since the Frobenius norm has been adopted, why is the regularization term not simplified by the Frobenius norm (and the summation term loses the superscript)? As a result, there is a typo ($\beta\mathbf{S}$ should be $\beta\mathrm{Tr}(SS^\top)$) in the second line of Eq. (18).
• Eqs. (19-20) seem a bit strange. The result of Eq. (19) I calculated is $\frac{\partial \mathcal{L}}{\partial \mathbf{S}}=2(\mathbf{Q}^{(i)}-SH)(-\mathbf{H}^{\top})+2\beta \mathbf{S}=\textcolor{red}{-2\mathbf{Q}^{(i)}\mathbf{H}^{\top}}+2\mathbf{SHH}^{\top}+2\beta \mathbf{S},$ And the Eq. (20) should be $\mathbf{S}=\textcolor{red}{\mathbf{Q}^{(i)}\mathbf{H}^{\top}}(\mathbf{HH}^{\top}+\beta \mathbf{I})^{-1}.$ I don't know whether $\mathbf{Q}^{(i)}\mathbf{H}^{\top}$ is a symmetric matrix, so that it can return to the form used by the author. It seems that this proposition does not hold. This formula seems to involve the core of this article, so I was unable to continue reviewing subsequent derivations and experimental results. However, it is worth mentioning that since the matrix shapes are aligned, whether my or the author's derivation is wrong, the derivation of complexity seems to be correct.