MPformer: Advancing Graph Modeling Through Heterophily Relationship-Based Position Encoding

Q2: no discussion on the running time and training cost.

A2: Thank you for your insightful feedback and thoughtful considerations regarding the computational expenses and training costs associated with the method employed in our research. We are cognizant of the computational expenses involved in training the model, and we recognize that optimizing computational resources is crucial. To address this concern:

1. Efficiency of Tree2Token. When we designed the algorithm, we took into account that the Tree2Token computation spends more money, and for this reason we put the Tree2Token computation before the model training. This makes it possible to use the Tree2Token to aggregate this k-hops information only during the training of the model.
2. Cost-Benefit Analysis. Despite the computational overhead of Tree2Token, its improvement in model performance over common information aggregation methods is significant. We compare Tree2Token with two common information aggregation methods, as well as did not use any information aggregation, and the experimental results show that Tree2Token's improvement in model performance is significant. Therefore, the computational spend of Tree2Token is worthwhile. | | Cora | CIteseer | Cornell | Textas | Wisconsin | Actor | Chameleon | Squirrel | |----------|----------|----------|----------|----------|----------|----------|----------|----------| | without MP | 68.31±2.53 | 55.85±6.52 | 74.32±5.83 | 71.35±7.66 | 75.29±4.66 | 32.89±3.28 | 33.64±2.89 | 23.37±2.29| | GCN | 66.66±7.59 | 63.64±9.71 | 58.65±3.64 | 59.46±4.98 | 46.47±6.80 | 26.30±1.11 | 50.92±5.11 | 43.46±3.13 | | Mixhop | 78.33±4.53 | 68.26±4.62 | 60.00±4.32 | 60.27±6.63 | 56.86±8.99 | 34.87±1.12 | 49.01±4.36 | 38.28±3.84 | | Hops2Token[1] | 15.73±1.33 | 14.46±2.31 | 11.48±1.23 | 12.57±3.21 | 12.75±4.21 | 7.41±0.82 | 11.16±1.33 | 6.65±0.89 | | Tree2Token | 86.42±3.84 | 68.28±5.04 | 76.49±8.97 | 78.38±6.84 | 75.10±8.73 | 36.14±1.10 | 64.28±3.46 | 52.39±2.92 |

In the table, 'without MP', 'GCN', 'Mixhop', and 'Hops2Token' respectively represent the performance without using any aggregation method, using the aggregation method in GCN, using the Mixhop aggregation method, and using the Hops2Token method. In this experiment, to verify the performance differences between aggregation methods, our experiments were conducted with fixed parameters and without using any positional encoding.

Q3: the effectiveness of Tree2Token was not analyzed.

A3: Thank you for highlighting the absence of experiments regarding Tree2Token's effectiveness in our paper. Based on your suggestion, we have included experiments to assess the impact of including or excluding the Tree2Token module in MPFORMER on the model's performance.

To validate the efficacy of Tree2Token, we conducted ablation experiments on it: keeping all parameters constant, we substituted Tree2Token's aggregation with Mixhop and GCN aggregations, as well as conducted experiments without any information aggregation. We compared these four scenarios with the approach using Tree2Token, and the results are presented in the table above. From the table， we can see that the model performance is improved the most when using Tree2Token, and Tree2Token is helpful in improving the performance with the model.

Q4: The Introduction Section is not well-organized.

A4: Thank you for your valuable feedback on our manuscript. We genuinely appreciate your insights, especially regarding the organization and length of the introduction.

We acknowledge that the introduction's structure could benefit from improvement. We are committed to addressing this concern by implementing the following strategies:

Paragraph Restructuring: We will restructure the introduction by breaking down lengthy paragraphs into more concise sections. This will facilitate a clearer flow of ideas and improve readability.

Focused Content: We recognize the importance of maintaining focus. Our revisions will ensure that each paragraph contributes directly and purposefully to establishing the context and significance of the research, avoiding unnecessary verbosity.

We will make changes to the introductory section based on your suggestions and would like to thank you for your suggestions.

1. NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphs

2. Advantages of Tree2Token.
   1. Tree2Token preserves some of the high-frequency information in the original graph signals. According to the theory of graph signal process (GSP), the GNN is a low-pass filter, and according to papers such as [1],[2], it has been shown that high-frequency signals help to help the GNN improve its performance on heterogeneous graphs. Let the eigenvalue of $\mathbf{A}$ be $\lambda$, then the eigenvalue of its corresponding Laplace matrix is $1-\lambda$, i.e., the filter coefficient is $1-\lambda$. (According to GSP, the larger the filter coefficients) At this point, the filter coefficients obtained by utilizing the Tree2Token method of aggregation are $1+(1-\lambda)+[(1-\lambda)^{2}-(1-\lambda)-1]+\cdots$, note that $1+(1-\lambda)+(1-\lambda)^{2}$ behaves as a low-pass filter property and $-(1-\lambda)-1$ behaves as a high-pass filter. In contrast, the method of aggregating node information using $\mathbf{A}^{k}$ in traditional GNN, the filter coefficients are $1+(1-\lambda)+(1-\lambda)^{2}+\cdots$, which exhibits a low-pass filter. From the above analysis, it can be seen that the aggregation method of Tree2Token is able to retain more high-frequency signals than the traditional information transfer method, and therefore helps to improve the performance of the model on the heterogeneous graph.

This also illustrates the point in the paper that the stacking of node information brought by the traditional information transfer method is unfavorable to the performance of the GNN on the heterogeneous graph. This is because with the previous information transfer methods used by GNNs, after the node information passes through multiple low-pass filters, the node features become too smooth, making it difficult for the model to distinguish the differences between neighboring nodes; whereas, the aggregation method of Tree2Token retains some of the high-frequency signals by avoiding the stacking of the information between different hopping nodes, helping to improve the model's performance on heterogeneity.

1. Beyond Homophily and Homogeneity Assumption: Relation-Based Frequency Adaptive Graph Neural Networks

2. Beyond Low-frequency Information in Graph Convolutional Networks

We thank you for your reviews and address your concerns as follows.

General Response: Thank you very much for patiently reading our paper and providing valuable feedback. Based on your suggestions and inquiries, we have made the following modifications and discussions:

1. We compared the aggregation methods used in ESAN[1] with Tree2Token and analyzed the advantages of Tree2Token.

2. We designed an experiment to verify the effectiveness of HeterPos in addressing heterogeneity issues. Specifically, HeterPos can help the model handle heterophily by capturing heterophily relationships between nodes.

3. We discussed the training cost of the method and demonstrated through experiments that such cost is worthwhile.

4. We supplemented the effectiveness experiments for Tree2Token

5. We will modify our paper based on your suggestions to ensure its clarity and readability.

Q1: How information aggregation differs from [1] and why Tree2Token is useful on heterogeneous graphs.

A1: Thank you for taking the time to provide feedback on our paper. We appreciate your insightful comments and the opportunity to address the concerns raised regarding the innovation of our research. We understand the importance of demonstrating the novelty of our work within the context of existing literature.

Our research aims to improve the model's ability to model on heterogeneous graphs by proposing a new information aggregation method as well as location coding, which aggregates k-hops information by serializing the information of neighboring nodes and captures the differences between a node and its neighboring nodes using location coding. We believe our work is sufficiently innovative:

1. On the differences between Tree2Token and the aggregation methods in [1].

   a. The way ESAN obtains k-hops neighbors is different from Tree2Token. while ESAN obtains k-hops neighbors using the features of the original node and its corresponding index information, our method obtains k-hops neighbor node information by $\mathbf{A}^{k}\mathbf{X}-\mathbf{A} ^{k-1}\mathbf{X}-\mathbf{I}$, and for a node, its $k$-hop node information is represented by a weighted summation of the features of all nodes that are $k$-distance from its shortest path. A structure like Tree2Token can retain high-frequency signals within node information, thereby enhancing the model's capacity for modeling heterogeneous graphs. We will provide detailed elaboration on this point later.

   b. The subgraph representation is different. the subgraph representation of ESAN is represented by the information of k-hop neighbor node indexes and the node features in the original data(The node features that have not undergone information aggregation), and the relative positional relationship between the neighbor nodes and the center node is obtained by indexing; whereas Tree2Token puts the information of nodes that are at the same distance from the center node in the same dimension, and information of nodes with different distances in different dimensions, i.e., it uses the dimension to represent the neighbor nodes and the center to characterize this positional relationship, and further determines this relative positional relationship by means of positional encoding at the end.

Q5: Why HeterPos can help models deal with heterophily

A5: HeterPos is able to model the positional relationships between nodes as a way to help the model improve its performance on heterogeneous graphs. According to the analysis in Section 2.2.1, the position-encoded node features, when entering the self-attention part of the Transformer module for computation, will get $\cos(c\mathbf{x}_ {v}^{k}e^{-{k}{\rm{ln}}(10000/d)}-c\mathbf{x}_ {u}^{j}e^{-{j}{\rm{ln}}(10000/d)})$, where $\mathbf{x}_{v}^{k}$ denotes the information of the $k$-th-hop neighboring node with node $v, c$ is a constant, and $d$ is the dimension of the hidden layer. From this equation we can see that HeterPos can represent the difference between different nodes, the difference between a node and its neighboring nodes, and the difference between different hop neighbors of different nodes. Such differences can enable the model to better capture the heterogeneous relationship between neighboring nodes, thus helping the model to distinguish between homogeneous nodes as well as heterogeneous nodes among the neighboring nodes, thus helping the model to classify them.

To prove our point, we designed a simple experiment to validate our approach. By comparing the attention matrix weights of the graph Transformer before and after employing HeterPos, we observed a significant mutual influence among nodes belonging to different categories in the graph Transformer model without HeterPos. However, upon integrating HeterPos into the model, the interaction between nodes of the same category strengthened. This confirms that HeterPos helps the model recognize the relationships between nodes, thereby enhancing its performance. For details on the experiment, please refer to our supplement. For details on the experiment, please refer to our supplement.

Q6: position encoding is similar to [1, 2]

A6: Thank you very much for your question. Indeed, HeterPos is quite different from the positional encoding in [1,2]. We will illustrate the difference between HeterPos and [1,2] separately

1. Since the positional coding proposed by vanilla Transformer makes it difficult to model relative positional coding, for this reason Shaw et al. [1] proposed a truncated relative positional coding for encoding relative positional information in sequences. They achieve this by adding a bias to the self-attention mechanism that represents the relative position relationship.The differences between heterpos and their approach are centered on two main aspects: a. The methods for introducing relative positional information are different. Shaw et al. obtained the embedding of the relative position of the sequence by numbering the sequence. However, HeterPos uses the node features and the numbering of their neighboring nodes as inputs for position encoding, and jointly encodes the relative positional information by the similarity between nodes and their neighboring nodes and the difference in their numbering (shortest path distance). b. Although Shaw et al.'s method performs well in the NLP domain, it is not suitable for the graph domain. In the graph domain, due to the scattered arrangement of nodes in space, it is difficult to directly utilize position encoding methods from the NLP field. Although it is possible to encode nodes in the order of k-hops using relative positional encoding, such an approach does not effectively enhance the model's ability on graphs [3]. Furthermore, we have also verified this through experiments, as shown in Figure 4. in paper.

2. The author proposed the "reusing the hidden states obtained in previous segments" method to address the limitation of fixed-length context in language modeling with the Transformer architecture. To ensure consistency in position information when reusing hidden states, the author introduced a method of relative positional encoding. This method replaces the bias term representing positional encoding in the vanilla Transformer with a bias term representing relative position when calculating the attention matrix. Additionally, two learnable parameter matrices are added to ensure that the query vector and key vector are the same for all query positions and key positions. The differences between heterpos and their approach are centered on two main aspects: a. The motivations are different. The design of relative positional encoding in Transformer-XL is to ensure consistency in position information when reusing hidden states, and the design of learnable parameter matrices is to ensure that the query vector and key vector are the same for all query positions and key positions. On the other hand, HeterPos is designed to encode the relative positional relationships between nodes, and the meaning of the weight matrix is to find a trade-off between model generalization and performance. b. There are also formal differences. Transformer-XL indexes the tokens in the sequence and uses the relative size of the indexes to represent the relative positions between tokens, while HeterPos takes node features as input and characterizes the relative positional relationships between nodes by calculating the similarity and shortest path distance between nodes.

Q7: only provide the serializing case without the corresponding heterogenous graph in Figure 1.

A7: Thank you for your insightful comments and for raising concerns about the clarity of the drawing in our paper.

In this figure, we wish to illustrate that for heterogeneous graphs, MPformer serializes the graph via Tree2Token and captures the heterogeneous relationships between nodes using HeterPos to correctly classify the nodes. In Figure 1, we use the colors of the nodes to represent belonging to different categories and indicate the serialization process by arranging the nodes. The red box in the figure indicates that MPformer can capture the heterogeneous structure that has the same in both graphs and help the model to categorize the nodes correctly by capturing this heterogeneous structure.

Thank you for your time and consideration. We are committed to improving the clarity of the drawing to enhance the overall quality of our paper.

1. Self-attention with relative position representations

2. Transformer-xl: Attentive language models beyond a fxed-length context
3. Rethinking Graph Transformers with Spectral Attention

We thank you for your reviews and address your concerns as follows.

General Response: Thank you very much for patiently reading our paper and providing valuable feedback. Based on your suggestions and inquiries, we have made the following modifications and discussions:

1. We explain the reasons for using the Chameleon and Squirrel datasets.

2. We analyzed the Tree2Token method design idea and provided a theoretical explanation for our design idea as such.

3. We compare HeterPos with other positional encodings in detail

4. We experimentally demonstrate why heterogeneous relationships between nodes cannot be captured by the Attention mechanism alone, but can be captured by adding HeterPos.

Q1: train-test data leakage in the squirrel and chameleon datasets.

A1: Thank you for your thorough review of our paper and for raising the concern about potential train-test data leakage in the squirrel and chameleon datasets. We chose the Chameleon and Squirrel datasets for our experiments to ensure fairness. Most of the previous methods [1,2,3,4,5]and models on graph neural network heterogeneity have used these two datasets, and in order to have an adequate comparison with these methods, we still compare MPformer with other methods under these two datasets.

Q2: the standard deviation of our method accuracy is larger than 1% accuracy

A2: Thank you for your insightful comments on our paper and for noting the larger standard deviation in method accuracy. Your observation has prompted us to delve deeper into this aspect of our research.

In comparing MPformer to other methods, we did ignore possible causes for the model to be accurate with a standard deviation greater than 1%. However, our method performs well under all eight datasets, especially on the heterogeneous graph dataset. The consistency of the MPformer method is demonstrated under different datasets, and thus we believe that MPformer's improvement in model performance is significant.

Moving forward, we plan to explore strategies aimed at reducing this variability. This includes additional experiments, parameter fine-tuning et.al.

We are committed to addressing this concern and further enhancing the reliability and stability of our method. Your feedback is invaluable, and we appreciate the opportunity to strengthen our research.

Q3: Many notations are confusing.

A3: Thank you for your detailed review of our manuscript and for highlighting the issue concerning confusing notations. We genuinely appreciate your efforts in providing constructive feedback.

We acknowledge the importance of clear and consistent notations in facilitating reader comprehension and regret any confusion caused. To address this concern, we plan to:

Identify and Revise: We will carefully identify the specific notations highlighted as confusing. Our intention is to revise these notations to make them more intuitive and comprehensible for readers. We will also consider providing additional explanations or a notation key where necessary.

notion explanations：

1. ${\mathbf{X}^{(i)}}^{\top}$ in Theorem 1 means ${\mathbf{x}^{(i)}}^{\top}$. It has the same meaning as in lemma1 and represents the row vector of the input matrix
2. $\mathbf{x}^{(i)}$ in Theorem 1 represents the row vector of the input matrix, and $\mathbf{x}_ {v}^{k}$ means representation of neighboring nodes at the $k$-th hop of node $v$. Since $\mathbf{x}_ {v}^{k}$ is the vector of actual inputs to the model, what is actually required in Theorem 1 is a bounded number of norms for information about all nodes as well as their neighboring nodes.
3. $\mathcal{F}$ in Theorem I is an aggregation of vector-valued functions, for which we give the definition of $\mathcal{F}$ in Definition I as well. It represents the set of functions that the model may learn. The activation function, on the other hand, is a nonlinear function in the model and can be simply understood as a part of any function in the set $\mathcal{F}$.

We are committed to addressing this issue promptly to improve the overall quality of our paper.

1. Hierarchical Graph Transformer with Adaptive Node Sampling.

2. Specformer: spectral graph neural networks meet transformers
3. Ordered gnn: Ordering message passing to deal with heterophily and over-smoothing.
4. Beyond homophily in graph neural networks: Current limitations and effective designs.
5. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks.

Q3: Missing relevant baselines.

A3: Thank you very much for your advice. We have added a comparison of several methods based on your suggestion.

Q4: Paper writing can be further improved.

A4: We sincerely appreciate your thoughtful review and the valuable feedback regarding the improvement of our paper's writing quality. Your attention to this aspect is crucial, and we are dedicated to addressing this concern.

We fully recognize the importance of clear and effective communication in scientific writing. To enhance the manuscript's readability and coherence, we plan to:

Clarify and Refine: We will carefully revisit the manuscript to identify and revise sections that may lack clarity or coherence. Our aim is to improve sentence structure, rephrase ambiguous passages, and ensure a more logical flow of ideas throughout the manuscript.

Enhance Organization: We will focus on improving the overall organization of the paper to better present our research findings and arguments. This includes ensuring that each section contributes cohesively to the overall narrative of the paper.

We are committed to implementing these improvements diligently to enhance the manuscript's quality. Your feedback is immensely valuable to us, and we aim to address all concerns raised.

Thank you for your time and consideration.

Q5: the hyper-parameter sensitivity analysis is confusing.

A5: Thank you very much for pointing out our problems related to hyperparametric experiments. We will respond to each of the questions you have pointed out.

1. In the experiments with hyperparameter $c$, we set the number of hop to 1 and the hidden layer dimension to 128, where k is the $k$ in Eq. 5 for the number of hop.
2. In the experiments on the hyperparameter h, what we wish to verify is the effect of d in Eq. 5, i.e., the hidden layer dimension, on the performance of the model.
3. Figure 6 illustrates the change in the output result of Equation 8 as the hyperparameters are varied (since Equation 8 is the result obtained by HeterPos after the self-attention operation). The vertical axis of the graph represents the values of the hyperparameters and the horizontal coordinates represent the magnitude of each component of the input vector. The color bar in the figure represents a visualization of the size of the output results, with brighter colors representing larger values.
4. First, let me restate the experimental setup with respect to the hyperparameter $k$. The hyperparameter $k$ represents the number of hops. In order to evaluate the effect of a change in the value of $k$ on the HeterPos output, we are illustrated by the change in the output of Equation 8. In this experiment we randomly generate two vectors $x, y$. Vector $x$ represents the features of node $v$ and node $y$ represents the information about the neighboring nodes of node $v$ at the $k$-th hop. During the experiment, we set the hyperparameter $c$ to 100, the hyperparameter $d$ to 128, and the values of the hyperparameters are set to [1,2,3,... ,20]. Fig. 6, shows the experimental results about hyperparameter k. According to Equation 8, we can see that the value of $e^{-k\ln(10000/d)}$ will become smaller and smaller as the value of $k$ keeps increasing, and when $k\rightarrow \infty$, the value of $e^{-k\ln(10000/d)}$ will tend to 0, i.e., Equation 8 will tend to $\cos(c\mathbf{x}^{k}e^{-k\ln(10000/d)})$. i.e., the influence of the neighboring nodes of the node $v$'s $k$-th hop on $v$ will also decrease as $k$ increases, which also matches with our intuition that nodes that are farther away from each other influence each other less.

We thank you for your reviews and address your concerns as follows.

General Response: Thank you very much for patiently reading our paper and providing valuable feedback. Based on your suggestions and inquiries, we have made the following modifications and discussions:

1. We illustrate the advantages of our approach by comparing Tree2Token in detail with the aggregation method proposed by NAGphormer.

2. We design an experiment to show that HeterPos is able to capture the relative positional relationships between nodes as a way to improve the model's ability to model on heterogeneous graphs.

3. We also added the relevant baseline based on the reviewers' suggestions

4. We have further explained and illustrated the relevant parameters and experimental results in the experimental part of our paper.

5. Finally, we will revise our paper according to the reviewers' suggestions to make the paper clearer and more fluent.

Q1: The notion of TREE2TOKEN has been introduced in NAGphormer[1].

A1: Thank you for taking the time to provide feedback on our paper. We appreciate your insightful comments and the opportunity to address the concerns raised regarding the innovation of our research. We understand the importance of demonstrating the novelty of our work within the context of existing literature.

Our research aims to improve the model's ability to model on heterogeneous graphs by proposing a new information aggregation method as well as location coding, which aggregates k-hops information by serializing the information of neighboring nodes and captures the differences between a node and its neighboring nodes using location coding. We believe our work is sufficiently innovative:On the differences between Tree2Token and the aggregation methods in NAGphormer.

1. The motivation of Hops2Token is different from the motivation of our proposed Tree2Token. hops2Token is designed to address the difficulty of graph Transformer models in expanding to large scale graphs, whereas our approach is proposed to address the difficulty of graph Transformer models in handling and characterizing heterogeneous relationships.

2. Hops2Token computes the nodes at the $k$-thop based on $\mathbf{X}_{k}=\hat{\mathbf{A}}^{k}\mathbf{X}$, where $\hat{\mathbf{A}}$ is the normalized adjacency matrix with self-loops, i.e., the main diagonal elements of this adjacency matrix are not 0. Our proposed Tree2Token approach, on the other hand, adopts the without selfloops approach in order to make the model can have a better performance on the heterogeneous graphs. loops, i.e., the main diagonal elements of this adjacency matrix are not 0. Our proposed Tree2Token method, on the other hand, uses an adjacency matrix without self-loops in order to allow the model to have a better performance on heterogeneous graphs (according to the conclusions of the existing literatures ([1], [2]) it is shown that the use of adjacency matrices WITHOUT self-loops doesn't not handle the Heterophily problem well.

3. The aggregation algorithm process and the way of calculating the adjacency matrix are also different. hops2Token simply calculates the kth power of the adjacency matrix when calculating the characteristics of the neighboring nodes at the kth hop, i.e., $\mathbf{A}^{k}\mathbf{X}$; whereas our method is $\mathbf{A}^{k}\mathbf{X}-\mathbf{ A}^{k-1}\mathbf{X}-\mathbf{X}$, which we do to prevent overlap between the $k$-hop node information and the information from the previous hop.

Q2: train-test data leakage in the squirrel and chameleon datasets.

A2: Thank you for your thorough review of our paper and for raising the concern about potential train-test data leakage in the squirrel and chameleon datasets. We chose the Chameleon and Squirrel datasets for our experiments to ensure fairness. Most of the previous methods [3,4,5,6,7]and models on graph neural network heterogeneity have used these two datasets, and in order to have an adequate comparison with these methods, we still compare MPformer with other methods under these two datasets.

1. Beyond Homophily in Graph Neural Networks: Current Limitations and Effective Designs

2. Is Homophily a Necessity for Graph Neural Networks
3. Hierarchical Graph Transformer with Adaptive Node Sampling.
4. Specformer: spectral graph neural networks meet transformers
5. Ordered gnn: Ordering message passing to deal with heterophily and over-smoothing.
6. Beyond homophily in graph neural networks: Current limitations and effective designs.
7. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks.

We thank you for your reviews and address your concerns as follows.

General Response: Thank you for pointing out many important references, and some writing issues. We have made revisions to address your concerns in the revised version. The revision can be summarized as follows:

1. We add the comparison with more positional coding methods, such as shortest-path distances (Ying et al., 2021) and tree-based encodings (Shiv and Quirk, 2019).

2. We changed the nomenclature about our methods and provided a full name of acronyms

Q1: Insufficient Benchmarking Against Alternative Methods.

A1: Thank you for pointing out this, we add experiments comparing with more positional encoding methods in the revised version. The results of the experiment are shown below：

I couldn't find the positional encoding method mentioned in your suggestion, "Novel positional encodings to enable tree-based transformers," in the GitHub repository provided by the authors. Therefore, I was unable to compare it. I would greatly appreciate it if you could provide the relevant code.

1. Rethinking Graph Transformers with Spectral Attention

2. Do Transformers Really Perform Badly for Graph Representation?
3. Gophormer: Ego-Graph Transformer for Node Classification

Q2: Unclear methodology acronyms.

A2: Thank you very much for your suggestion about naming the methods. MPformer is short for "Message Position graph Transformer". We chose the keywords "message" and "Position" to highlight the distinctive features of our method. In the subsequent version of the paper, we will change the short names of the methods as you suggested.

Once again, thank you for all the insights and suggestions.