N2GON: Neural Networks for Graph-of-Net with Position Awareness

Q1. The Introduction's definition of Graph-of-Net (GON) is vague and needs a precise mathematical formulation of each node-as-graph (size, structure, features) and clarification of how it differs from traditional graphs in modeling hierarchical, multi-dimensional relationships.

R1: Thank you for your insightful comments. In our framework, GON is designed as a hierarchical graph where each node is not a single data point but an entire graph. Formally, let the top-level graph (network) be defined as $\mathcal{G}^N = ( \mathcal{V}_G, \mathcal{E}_G )$, where $\mathcal{V}_G$ is the set of graph nodes and $\mathcal{E}_G$ denotes the edge set. For each node $v_G \in \mathcal{V}_G$, we associate a graph: $\mathsf{G}_v = (V, E, X)$. Here, $V$ represents the nodes within the graph, $E$ is the set of edges among these nodes, and $X$ is the feature matrix corresponding to the graph nodes. The size of each graph is determined by $|V|$, which depends on the inherent structure or the domain-specific construction of the graph. The graph structure, represented by $E$, captures the internal relationships among the nodes in the graph. This structure may vary depending on the level of detail or domain-specific insights desired. Each graph is equipped with a feature representation $X$ that encodes the characteristics of the nodes in $V$. The formation of these features can be based on raw data attributes or results from a prior processing step.

Traditional graph models represent data as a general graph where each node corresponds to an atomic data point. In contrast, GON captures multi-dimensional relationships by explicitly modeling two levels of interaction. This dual-level representation allows GON to be particularly effective for complex hierarchical data, as it provides the capacity to model nested relationships and capture both local and global patterns within the data.

Q2. In some formulas (e.g., the PPR algorithm), the derivation process could benefit from more detailed explanations. Providing a more comprehensive derivation would aid in understanding.

R2: The Personalized PageRank algorithm provides a way to measure how “close” or “important” one node is relative to another within a network. Imagine a random walker who starts at a specific node in the network. Instead of wandering the network entirely at random, the walker follows a rule: at each step, they decide either to move to one of the neighboring nodes or to jump back to the starting node. This jump-back mechanism ensures that the influence of the starting node remains strong throughout the walk.

What makes PPR particularly useful is that it considers not only the direct connections between nodes but also the broader network structure. In simple terms, it captures the idea that even if two nodes share the same label or initial property, they can have varying levels of relatedness depending on how they are connected within the network.

By adopting this method, our approach goes beyond simply saying, "nodes with the same label are similar." Instead, we are able to gauge the subtle nuances in how closely nodes are related based on both their labels and their positions within the graph structure. This allows our model to better capture complex relationships and provides a more refined way of measuring similarity among node graphs.

Q3. It would be helpful to clarify how GON manages the hierarchical structures within the graphs of nodes.

R3: Thank you for the valuable feedback. In a Graph-of-Net (GON), each high-level node represents an entire graph that can have its own structure and detailed relationships. Instead of treating every graph as a monolithic object, we decompose the problem into two levels: 1) At the lower-level, we focus on extracting meaningful representations from the individual graphs. We utilize graph neural networks to process each graph, effectively summarizing its properties into a fixed-size embedding. 2) Once each graph has been transformed into a representation, these embeddings serve as the nodes for the higher-level graph. The interrelations among these nodes are then modeled, combining both the abstract representation of the graphs and the positional information within the larger network. The two-stage process helps us manage the inherent complexity of graphs that reside within nodes. We will add above description in subsequent revisions of the paper.

Q1. The term "Position Awareness" is not strictly defined in the paper. A formal definition could be added for clarity.

R1: Thank you for highlighting the need for a more formal definition of the term "Position Awareness." In our work, we use "Position Awareness" to refer to the model’s ability to capture and integrate the relative placement and connectivity of nodes within the overall network structure. Position Awareness is the property of a network model whereby each node's representation explicitly encodes its structural context within the graph. This encoding captures not only the node’s intrinsic features (e.g., labels or attributes) but also its topological characteristics—such as how it is connected to other nodes, and its overall influence in the network. In our framework, this is operationalized by leveraging a scoring mechanism (i.e., via the Personalized PageRank algorithm) that quantifies the relative importance or influence of nodes.

By assigning a score that measures the likelihood of reaching one node from another through random walks with restarts, our approach formalizes a node’s “position” within the network. This score effectively differentiates nodes that might share similar labels but occupy distinct roles depending on their connectivity pattern and centrality in the graph. These scores are then integrated into the node representations, ensuring that position-dependent information contributes to the final embedding. As a result, the model is better equipped to capture nuanced relationships that go beyond simple label similarity.

We will include this formal definition in the revised manuscript to enhance the clarity and rigor of our contribution.

Q2. The distinction between GON and existing hierarchical graph structures (e.g., Hierarchical Graph Networks, Hypergraphs) is not clearly quantified.

R2: Thank you for your comment. We distinguish our Graph-of-Nets (GON) framework from other hierarchical graph structures as follows: 1) Explicit Two-Level Representation: In GON, each node is a complete graph that retains its full internal structure. This is different from many hierarchical graph networks, which typically process standard graphs where the nodes are represented as vectors rather than graphs, potentially losing fine-grained structural details. 2) Preservation of Intra-Node Complexity: Instead of merging all details into a single representation, GON processes each graph independently (using specialized graph neural networks) and then integrates these detailed embeddings into the higher-level network. This two-stage approach effectively captures both local (intra-node) and global (inter-node) relationships.

We will add these distinctions in the revised manuscript.

Q3. In the description below Equation (1) in the algorithm section, "funcitons" should be corrected to "functions."

R3: Thank you for catching the typo. We appreciate your attention to detail, and we will correct in the revised manuscript.

Q4. The similarity function ( \text{sim}(\cdot, \cdot) ) in Equation (6) lacks a clear explanation of its basis (e.g., if using cosine similarity, the vector normalization step should be specified).

R4: Thank you for your comment. We clarify that in Equation (6), we directly compute the cosine similarity on the final representations without any additional normalization. We will update the manuscript to clearly state this.

Q5. "node graph" → It is recommended to standardize this term as "node-graph" (with a hyphen).

R5: Thank you for your comment. We appreciate your suggestion to standardize the term. We will update the manuscript to use "node-graph" consistently.

Q6. The paper would benefit from additional discussion on the future research directions. For example, investigating how GON could be integrated with other deep learning techniques (e.g., reinforcement learning, meta-learning) to adapt to new or evolving graphs could open up interesting lines of inquiry.

R6: Thank you for the insightful suggestion. We plan to explore several integration approaches with GON in our future work. For example, integrating reinforcement learning could allow for dynamic graph adaptation, where RL agents iteratively rewire inter-graph connectivity in applications like drug discovery and evolving recommender systems. Additionally, we are considering meta-learning approaches to pre-train GON encoders on diverse tasks—enabling rapid adaptation to new scenarios, especially when encountering limited labeled data. Moreover, we can extend GONs to handle temporal dynamics by incorporating architectures for gradually updating both intra- and inter-graph connections in evolving systems. We believe these strategies will further enhance GON’s flexibility and robustness across various challenging, real-world applications.

Q1. It is recommended to add runtime comparisons.

R1: Thank you for your comment. We conducted the runtime experiments and the results (the average elapsed time per epoch), summarized in the table below, indicate that on benchmark graphs, our runtime is comparable with that of SOTA baselines, while on biomedical datasets our method is significantly more efficient than traditional algorithms.

Table I. Running Time per epoch on Benchmark Graph Datasets (in seconds)

Table II. Running Time per epoch on Biological Datasets (in seconds)

Q2. More discussion on position-awareness in graph learning should be included.

R2: Thank you for the feedback. In our work, position-awareness is achieved by leveraging the Personalized PageRank algorithm, which computes the topological influence of node graphs on each other, implicitly revealing their relative positions in the network. In essence, once all pairwise similarities are determined, each node inherently carries information about its relative position within the similarity network structure.

Q3. Providing dataset details (e.g., node/edge counts) for clarity.

R3: Thank you for your valuable suggestion. We have now added detailed descriptions of benchmark dataset in the table below.

**Q4. Discuss more real-world applications (e.g., drug discovery) and potential limitations to better illustrate GON's impact.

R4: Thank you for your valuable suggestions. We will further expand the discussion of GON’s practical applications in the manuscript. For example, in drug discovery, GON can model drug molecules as graphs of atoms while representing proteins as residue graphs, capturing the binding patterns between them through a global network. In bioinformatics, GON enables multi-scale analysis of protein interaction networks, such as identifying functional modules at the residue level. For social networks, GON can model user communities (e.g., user-centric social subgraphs) and the relationships across communities, though real-world deployment must address privacy concerns and data sparsity issues. Common challenges include the cost of data construction (e.g., the need for expert annotations for molecular graphs) and computational overhead.

Q5. There are some grammatical errors and typos.

R5: Thank you for your detailed feedback. We will review the manuscript and corrected all the mentioned grammatical errors and typos.

Q6. Could you clarify if Algorithm 1 uses full-batch training or mini-batch sampling?

R6: Thank you for your valuable feedback. For the GON datasets, we use full-batch training—processing the entire data at once—since it is feasible to handle these datasets in one go. In the revised manuscript, we will clarify this point.

Q7. How is this coefficient set for the loss in the algorithm?

R7: Thank you for raising this important point. In our work, we deliberately did not introduce a weighting coefficient between the constraint loss ($L_{\text{con}}$​) and the NLL loss. We found that both loss components naturally operate on comparable scales, allowing us to simply sum them without additional tuning. This design choice not only simplifies the loss function but also reduces the number of hyperparameters, streamlining the training process.

Q1. The partitioning of GON —particularly in datasets like CiteSeer—appears questionable, where the partitioning is performed directly using KNN, resulting in computed outcomes that merely replicate the information propagation process of GNN.

R1: Thank you for your feedback. Below, we provide clarification as follows:

1. Rationale for $k$-hop Sampling Strategy. We would like to clarify that we adopt a $k$-hop sampling (instead of KNN, which only involves 1-hop) to construct a graph for each node. This method is motivated by the observation that in many real-world scenarios, a node’s full identity is not encapsulated solely by its features but also by the structural context provided by its neighboring nodes.

For example, 1) in Citation Networks such as CiteSeer, a paper’s identity is defined not only by its content but also by its references and cited-by relationships. By sampling $k$-hop neighbors, we explicitly model a node’s "extended identity," which includes its local academic context (e.g., related works). This aligns with the GON philosophy of representing nodes as hierarchical structures. 2) in Social Networks, a user’s social graph (friends, followers) is a natural extension of their identity, and $k$-hop sampling preserves this contextual information. Therefore, sampling the $k$-hop neighborhood effectively captures the local context.

The reviewer noted that, in the case of CiteSeer, direct partitioning by $k$-hop might induce outcomes very similar to standard GNN information propagation. We believe this observation is, in fact, supportive of our methodology. The successful aggregation of neighbor information by GNNs substantiates that a node’s neighborhood plays a crucial role in the learning process. Our method does not merely replicate the GNN propagation process but rather formalizes the node’s expanded representation through explicit graph construction. This ensures that our Graph-of-Net structure inherently reflects the multi-level composition of nodes—each node graph is a manifestation of both its own features and the collective representation of its neighbors.

2. Alternative Graph Construction Method. Beyond $k$-hop sampling, we explored constructing graphs from raw node data. We appreciate the reviewer's note on the challenge of accessing original data. In response, we made significant efforts and successfully obtained the raw textual data for the CiteSeer data through the GitHub project (https://github.com/sivaramanl/Information-Retrieval/tree/master/Text%20Processing/citeseer). This repository provided us with nearly all the title and abstract information for each paper. For this alternative graph construction, we processed each paper’s text as follows: 1) Text Preprocessing: We used NLTK to split the text of each paper into sentences, treating each sentence as a node in the paper’s graph. 2) Feature Extraction: We generated embeddings for each sentence using the popular sentence transformer model all-MiniLM-L6-v2, which provided the node attributes. For papers where the original information is missing, we default to representing the paper as a single node with an all-zero attribute vector. 3) Edge Construction: Edges were formed by computing the cosine similarity between sentence embeddings and applying a threshold of 0.7 to retain only strong connections.

We then conducted experiments on this newly constructed data. The generated data statistics and performance comparisons, as presented in the tables below. The comparable performance of $k$-hop and text-based GONs (81.27% vs. 81.82%) demonstrates that both methods validly capture hierarchical semantics. The $k$-hop approach is a pragmatic and effective proxy when raw data is unavailable, while text-based construction confirms GON($k$-hop)’s flexibility for domains with explicit substructures.

Table I: Resulting GON Statistics on Citeseer

Table II: Accuracy on data CiteSeer

Q2. The full name of GON is mentioned twice.

R2: Thank you for pointing that out. We will remove the extra name.