IGNN-Solver: A Graph Neural Solver for Implicit Graph Neural Networks

1. Given that the IGNN algorithm can implicitly capture long-range dependencies, I had hoped that the IGNN-Solver would show a greater advantage over explicit GNNs on large datasets. However, on the ogbn-arxiv and ogbn-products datasets, the IGNN-Solver did not demonstrate a significant performance improvement over explicit GNNs. I would like to see not only the comparison of inference times between the IGNN-Solver and traditional IGNN, but also a comparison of the training and inference speeds of the IGNN-Solver with those of traditional explicit GNNs. I hope that the IGNN-Solver can significantly outperform explicit GNNs in terms of performance or efficiency, which would convince me of the practical significance of this work.

2. In Section 4.1.2 of the paper, "IMPROVED ANDERSON ITERATIONS WITH TINY-GNN" is mentioned. I would like to know more specifically the impact of having this module on the acceleration of inference. Additionally, we all know that for most graph datasets, there is a significant performance drop after reducing 50% of the edges. I would like to know the specific settings of the GNN Sparsification part in IGNN-Solver and the impact of this part's pruning rate on the overall effect.

3. The results in Figure 2 and Table 1 seem to have some inconsistencies. Is this due to different experimental settings? For example, in Figure 2(c), the best accuracy of the IGNN-Solver is around 0.716, while the accuracy given in Table 1 is 0.725. This is not a big issue; I just hope the authors can pay attention to such minor flaws.

4. The accuracy of the baselines in the ogbn-products dataset seems a bit low. According to the official LeaderBoards of OGB and my experimental experience, the accuracy of algorithms such as GCN/GAT/GraphSAGE on ogbn-products is generally between 0.76-0.8. These are all obviously higher than the corresponding values given in Table 1 and the accuracy of the IGNN-Solver.

In summary, if the authors can address my concerns, I would be open to increase my score.

1. Current background lacks coverage of recent advances, such as: Method [1], which models IGNN as a bilevel optimization problem, achieving significant speedups. Method [2], a scalable implicit model with higher accuracy on the ogbn-arxiv dataset. Given [2] is already cited, consider a comparison to highlight IGNN-Solver's distinct advantages in context.

2. Since efficiency is a core advantage of IGNN-Solver, it is crucial to benchmark its runtime against multiple existing methods beyond the basic IGNN. This will provide a clearer view of IGNN-Solver’s efficiency benefits.

3. To convincingly demonstrate superiority, include a straightforward baseline of IGNN using phantom gradients, a common efficiency enhancement. This comparison would clarify IGNN-Solver's performance against well-known alternatives.

4. Typo and Logic:

Line 093: "[34] ... Subsequently, [43] introduces ...". The chronology is incorrect, as [34] was published after [43]. Equation Formatting: In Equation 4, R^n -> R^{n\times n}

[1] Zhong Y, Vu H, Yang T, et al. Efficient and Effective Implicit Dynamic Graph Neural Network[C]//Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2024: 4595-4606. [2] Liu J, Hooi B, Kawaguchi K, et al. Scalable and Effective Implicit Graph Neural Networks on Large Graphs[C]//The Twelfth International Conference on Learning Representations. 2024.

1. The expreriment results in Table 1 cannot convince me. At least, the baselines for ogbn-arxiv and ogbn-products is severely sandbagged. For example, on ogb learderboard, GCNII is 72.74 for ogbn-arxiv, and GCN is 75.64 on ogbn-products. These gaps are more than 4% compared with the number reported in the paper.

2. There is no study on how accurate the alpha predicted by the tiny GNN model.

1. Lack of Theoretical Justifications: The paper does not provide sufficient theoretical underpinnings for the proposed fixed-point neural solver. Notably, it remains unclear if the fixed-point equation is well-posed, and there are no convergence guarantees for the solver.

2. Uncertain Effectiveness of the Learnable Initializer: Without guarantees of convergence, it is ambiguous whether the learnable initializer can indeed reduce the number of iterations required, thus raising questions about its potential to speed up the optimization process.

3. Limited Architectural Verification: The experimental validation of the proposed neural solver is restricted to the IGNN architecture. It is uncertain whether its benefits extend to other implicit GNN frameworks.

4. Absence of Ablation Studies: The paper does not include ablation studies that assess the solver’s ability to capture long-range dependencies, a key advantage associated with implicit GNNs.

5. Over-Smoothing Mitigation: The study does not address whether the proposed solver can mitigate the over-smoothing issue often encountered in GNNs.

1. More deeper analysis on why the proposed solver can be faster than others. In my view, the speedup comes from the less number of iterations required. I think that would be better if the authors can provided some theoretical analysis. If theoreical anlaysis is difficult to have, I would like to see some empirical evidences on how many iterations the proposed solver needs vs the traditional solver needs.
2. The high-level descriptions on the method (RPI-Graph) used for graph sparsification are not provided. I think it would be better if the authors can explain some high-level idea on this sparsification method. It can make the article more self-contained.
3. In my view, this work is mainly about a plug-in solver for Implicit GNNs, not specifically for that model named IGNNs. Therefore, I think that it would be better to apply the proposed solver to different implicit GNNs, such as MIGNN.