An Efficient Memory Module for Graph Few-Shot Class-Incremental Learning

Q1: The abbreviations of the proposed method are confusing.
A1: We appreciate your attention to our work and the valuable feedback, and sincerely apologize for the confusion caused by the use of abbreviations. According to your suggestions, we have re-examined the entire text and decided to standardize the abbreviation for "Memory Representation Adaptive Module" to ‘MRaM’, and the ‘’Memory Construction module’’ to ‘MeCs’ for minimizing the confusion. We have carefully reviewed our paper and ensured that all relevant abbreviations have been modified to avoid any potential confusion.

Q2: Fig. 1 is confusing; it doesn’t show the details and advantages of the SMU and MRaM structures.
A2: We appreciate your feedback regarding to Fig.1 and apologize for the confusion. The Structured Memory Unit (SMU) and the Memory Representation Adaptive Module (MRaM) are the key components of Mecoin, serving to learn representative class prototypes and interact with the Graph Neural Network (GNN) to retain past knowledge while updating new knowledge. In accordance with your advice, we have redesigned Fig.1 which include the structure and workflow of the SMU and MRaM. The redesigned Fig. 1 is included in the submitted PDF.

Q3: In Sec.3.1, Eq.6 lacks explanatory information. How does Eq.6 integrate edge information into the loss function?
A3: The first term in Eq.6 calculates the similarity between node features, thus reflects the strength of the connections between nodes[1,2]. This connection strength helps the SMU better distinguish the features of different classes of nodes, leading to more representative class prototypes.
[1]Confidence-based Graph Convolutional Networks for Semi-Supervised Learning.
[2]Geom-GCN: Geometric Graph Convolutional Networks.

Q4: The proof of Theorem 1 is too brief.
A4: Thank you for your suggestions.We included the proof outline and description of Theorem 1 in the appendix but not the detailed proof. We appreciate the reviewer's suggestion, as a detailed explanation would help readers better understand our theorem and conclusions.According to your advice, we have supplemented the proof, due to the rebuttal's word limit, we apologize for not being able to present the complete proof here, but give the outline. Similar to the proof of Thm3.1 in [16], we first decompose the generalization error $\mathcal{R}=\mathbb{E}[\ell(f(d_{\epsilon}(x)),y)]-\frac{1}{|\mathcal{X}_T|}\sum_i f(x_i,y_i)$ based on the probability $\mathrm{Pr}((x,y) \in \mathcal{S}_c)$ and the Total Probability Theorem. After some calculation and modification of the above decomposition, by using Lemma 1 in paper reference [7], we obtain Eq.11. For simplicity, we denote the last term of Eq.11 as $R’$, which has a tighter upper bound for other models according to Lemma 4 in paper ‘Combined scaling for open-vocabulary image classification’, and is not necessarily 0. While it equals 0 for Mecoin based on the definition of $\mathcal{S}_c,\mathcal{I}_c$ and the matching method(i.e. based on the smallest distance) in Meocin, since the term $\mathbb{E}_z[\ell(f(x),y)|z\in\mathcal{S}_c] = \frac{1}{|\mathcal{I}_c|}\sum_i\ell(f(x_i),y_i)$. Thus we conduct that Mecoin has a lower generalizaiton bound than other models. Furthermore, to clarify the theorem result and the proof, we have modified the symbols and statements, and have updated in the revised paper.

Q5: The definition of $N$ makes it challenging to explain how new knowledge is updated in Eq.9
A5: Thank you for pointing out the issues and errors. In Eq.9, we incorrectly used symbols. To distinguish between seen classes and unseen classes, we will use 𝑁_𝑠 and 𝑁_𝑢 to represent the sample sizes for seen and unseen classes, and replace 𝑁 in Eq.8,9 accordingly. When new class samples appear, Mecoin first calculates the class prototypes 𝑚61:100 for these new classes through the SMU module and identifies the corresponding 𝑝61:100, where 𝑝61:100 are randomly initialized and not updated. During the training process 𝑝61:100 are updated and integrated into MRaM via Eq.9, thus updating and storing new knowledge.

Q6: Explain the pre-training experimental setup. In Tab.3, the comparison method MAG-Meta is mentioned twice. Why does Mecoin appear among the comparison methods in Tabs.2, 3, and 4?
A6: Thank you for thorough examination and feedback. We used a standard graph node classification task in pre-training, where the specific samples and number of classes in each dataset are detailed in Tab.1. We apologize for the error in Tab.3. The second MAG-Meta should be Geometer and we have corrected it in the revised manuscript. Tab.2, 3, 4 show the comparison results of Mecoin with other methods across different datasets, which demonstrate the consistent performance and superiority of Mecoin under various experimental settings and across multiple benchmarks.

Q7: Sampling only 4 classes makes it difficult to demonstrate the ability of different settings to handle overlapping problems.
A7: Thank you very much for your suggestions. In Section 4.2, we used three commonly used graph-structured datasets: CoraFull, CS, and Computers. Our experiment aimed to demonstrate the impact of the SMU and related operations (Eq.2, Eq.3) on model performance and class prototype learning. We realized that sampling only four classes was insufficient to fully illustrate the handling of overlapping issues in different settings. Therefore, based on your suggestion, we increased the number of sampled classes to 10 and presented the results in the submitted PDF file. The experimental results with 10 classes were consistent with those obtained with four classes.

Q1: Lacks error bars, standard deviations.
A1: Thank you for the valuable feedback. In the initial version, we omitted detailed statistical information due to formatting and space constraints. However, these details are crucial for a comprehensive evaluation of model performance and result reliability. Based on the reviewer's suggestions, we have added error bars, standard deviations, and statistical significance analyses to the experimental results.

We apologize that due to word limits, we cannot include the results for all sessions for CoraFull and CS. We will include this detailed statistical information in the revised version of the paper to better demonstrate the reliability and significance of the experimental results.

W1：The paper needs further polish.
A1:Thank you for your thorough review and suggestions. We have carefully reviewed our paper and standardized the symbols and abbreviations. For example, we have unified MRaM and MRAM to MRaM. Additionally, we have conducted a comprehensive review and revision of the paper, focusing on sentence expression and structure to ensure clarity and accuracy.

W2:self-attention and MRaM is not clearly explained.
A2:Thank you for your thorough review and suggestions. In Eq.2, self-attention is used to enable the interaction between current task node features and seen class information. Specifically, the SMU stores class prototypes of seen classes, which are representative samples containing information about these classes. In continual learning on graphs, there are implicit relationships between graph structures of different tasks. Capturing these relationships helps in learning more representative class prototypes, thereby preventing forgetting[1]. Self-attention has been widely used to capture interactions between pieces of information. In Eq.3, attention is used to process the current task's node features, effectively capturing the relationships between nodes, which aids in learning more representative class prototypes. We designed MRaM to decouple the learning of class prototypes from the learning of class representations. Traditional graph continual learning methods further learn the probability distribution of classes (class representations) based on learned class prototypes for classification. This often leads to forgetting due to extensive parameter updates during prototype learning. MRaM separates the learning of class prototypes from class representations, avoiding this issue.
[1]Geometer:Graph few-shot class-incremental learning via prototype representation.

Q1: Explanation of meta-learning samples.
A1: Thank you for your valuable feedback. In the context of graph few-shot incremental learning, 'meta-learning samples' refers to the samples used in the meta-learning training process. We apologize for any confusion caused and have provided additional clarification on meta-learning samples in our paper.

Q2: Why using Gaussian random projection to reduce the dimensionality?
A2: This is an excellent question. We employ Gaussian random projection for dimensionality reduction for the following reasons:

1. Computational Efficiency: Compared to other dimensionality reduction methods, Gaussian random projection offers high computational efficiency. For instance, performing principal component analysis on a matrix of shape $(n, n)$ has a computational complexity of $\mathcal{O}(n^3)$, whereas Gaussian projection has a complexity of $\mathcal{O}(n^2)$.
2. Theoretical basis: According to the Johnson-Lindenstrauss lemma[2], Gaussian random projection can effectively preserve the distance relationships between high-dimensional sample points with a certain probability. It also exhibits good robustness for non-linearity, noise, or unevenly distributed data. Taking the scenario discussed in this paper as an example, let $z$ be an embedding vector and $v$ a perturbation of $z$. When dimensionality reduction is performed on $z+v$ using the Gaussian random projection matrix $R$, if $v$ lies in the null space of $R$, then $R(z+v)=Rz=Rz+Rv$, meaning the perturbation has no effect. Since $R$ is used for downsampling and has a large null space (downsampling matrices typically reduce data dimensions, projecting many original data directions to zero), perturbations like $v$ are more likely to be eliminated.
   [2]On variants of the Johnson–Lindenstrauss lemma.

Q3: Explanation of why concatenating $G_T$ and $H_T^p$.
A3: Information Integration: According to Eq.2, $H_T$ is obtained by using attention to integrate current task node features with past task class prototypes stored in the SMU. We then reduce the dimensionality of $H_T$ using a Gaussian random projection to obtain $H_T^p$, which helps reduce subsequent computational complexity. For $G_T$, Eq.3 further extracts the current task's graph structure information. By concatenating these two types of information, we obtain richer and more comprehensive node features, which better capture the complex relationships between the current and past tasks' graph structures.
Theoretical Support: Concatenation is a common and effective feature fusion method, validated in many studies on graph neural networks and self-attention mechanisms. For example, previous works like MixHop[3] concatenate current node information with neighbor node information to integrate neighbor features and capture richer structural information.
Experimental Support: The ablation studies in Fig.3 demonstrated the effectiveness of using $G_T$ or $H_T^p$ alone (no graphinfo, no inter) respectively, as well as with and without the concatenation method(with MeCo, no MeCo). And the concatenation method performs the best.
[3] MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing.

Q4: Difference between class prototype and class representation.
A4: Thank you for your meticulous review and valuable feedback. The class prototype $\mathbf{m}$ is a typical representation of the class. In our paper, it is obtained by integrating the information of seen classes and local graph structure with node features through MeCo, representing the class centroid. The class representation $\mathbf{p}$ contains probability information for each category and is used to determine the sample's category. In our paper, $\mathbf{p}$ is randomly initialized and updated through the distillation process in GKIM. When a sample finds its closest class prototype $\mathbf{m}$ via Euclidean distance, it further matches $\mathbf{p}$ for classification.