Normalized Space Alignment: A Versatile Metric for Representation Analysis

We thank the reviewer for their feedback and are glad that they found the applicability of the method and background on LID satisfactory. We address their concerns below:

1. Lack of comparison to relevant existing methods and relevant baselines

Thank you for your feedback regarding comparisons to existing methods in link prediction and adversarial attack experiments. We appreciate the opportunity to clarify the intent and scope of these experiments.

General Scope of NSA: NSA is primarily a metric for comparing representations across two spaces. It can explain the performance of machine learning tasks and serve as a differentiable loss function that can be efficiently estimated in minibatches. However, NSA itself does not prescribe how it should be applied to specific machine learning problems. Its usage, like that of other similarity metrics such as RTD and CKA, depends on the method or task in question.

Link Prediction Experiments: The link prediction experiment was not designed to showcase NSA as a supplementary loss function for improving link prediction but rather to demonstrate NSA’s ability to retain the structural integrity of representation spaces after dimensionality reduction. For smaller models, retraining a GCN in the reduced dimension is feasible. However, for larger models where pretraining is computationally expensive, NSA-AE provides a more practical alternative by ensuring that the reduced-dimensional space preserves relative distances and contextual information. As shown in Table 2, the performance of NSA-AE on reduced-dimensional data is comparable to that of a newly trained model.

In response to your suggestion, we have added results in Table 2 showing how a base GCN performs on link prediction as a reference. However, it is important to emphasize that this reference is not a baseline NSA is competing against but rather a contextual benchmark to understand NSA’s behavior. For more details, please refer to Global Response 2, where we discuss this experiment in depth.

Adversarial Analysis Experiments: In the adversarial analysis experiments, NSA is not used as a defense mechanism but as a post-hoc analysis tool to evaluate the structural impact of attack and defense methods. While NSA could potentially serve as a structure-preserving loss function to facilitate robust neural network training, it is outside the scope of this manuscript which presents a breadth focused introduction to NSA. We leave the idea of using NSA to facilitate robust training of neural networks to future work. Instead, the goal is to introduce NSA’s ability to correlate with disagreements in predictions and to perform fine-grained, nodewise analysis.

It is worth noting that NSA has a unique capability in this regard. Unlike the next best similarity metric, RTD, which analyzes topological features of spaces, NSA can examine individual data points and quantify their local discrepancies. This makes NSA particularly suited for tasks requiring high-resolution analysis of adversarial perturbations.

2. Datasets used in the experiments are small and basic, and the generalization of the method is questionable.

Thank you for your concern regarding the datasets used in our experiments. We address the generalizability and scalability of NSA by presenting results on datasets of varying scale and domain:

1. Dataset Variety: Our experiments span visualization-focused datasets (e.g., COIL-20, Spheres, Swiss Roll, Mammoth), benchmark datasets (e.g., MNIST, F-MNIST, CIFAR-10, Cora, Citeseer, Pubmed), and large-scale datasets (e.g., ImageNet, WordNet, Amazon Computers with 13K nodes and 500K edges, Flickr with 90K nodes and 900K edges). These datasets cover diverse domains, including images, natural language processing (NLP), and graphs. We provide detailed statistics for each dataset in Appendix P.
2. Scalability and Generalizability: The variety in our dataset choices demonstrates NSA’s scalability and generalizability across domains and tasks. Generalizability is evidenced by NSA’s effectiveness across different dataset types and scales, which is a key focus of this paper.
3. Theoretical Guarantees: In Section 4.2.1, we provide theoretical guarantees showing that GNSA converges to the global dataset value irrespective of dataset size when using mini-batching. This ensures NSA remains effective even for large-scale datasets. Additional Experiments:
4. Empirical results: In Appendix I, we present layerwise experiments on ResNet models evaluated on 100K ImageNet images. In Appendix Q, we provide ablation studies for GNSA and LNSA on ImageNet, showing that with mini-batching (using multiple trials on subsets), NSA can accurately approximate global values even for large datasets.

We believe these results comprehensively demonstrate NSA’s scalability, generalizability, and robustness across a wide range of datasets and tasks.

We sincerely thank the reviewer for taking the time to review our rebuttal in detail. We present responses to their queries below:

1. I do not understand what Figure 11 is showing, could you please provide context?

Thank you for your feedback. Figure 11 illustrates the effect of varying the parameters $l$ and $g$ in the NSA formulation. The experiment involves reducing the Spheres dataset (comprising 10 spheres within an 11th sphere) from its original 100 dimensions to 2 dimensions. Figure 15 (c) plots the first 3 dimensions of the original dataset for reference.

Using an NSA-AE trained solely with the NSALoss, we systematically vary $l$ and $g$ to observe how these parameters influence the dimensionality reduction visually. The figure demonstrates how different combinations of $l$ and $g$ affect the structural preservation of the dataset during this reduction.

We have added additional explanatory text to the relevant subsection and caption to clarify this further. We hope this resolves any confusion regarding the purpose and interpretation of Figure 11.

2. The influence of the choice of k and clarifications on selecting k.

Figure 13 demonstrates how LocalNSA performs when varying both the $k$ value and the width of the Swiss Roll. As the width increases, we observe that a slightly larger $k$ value is required to preserve the manifold effectively. This occurs because increasing the width spreads the data points further apart, reducing the density of local neighborhoods. When $k$ is too small, the immediate neighbors of a point may fail to fully capture the local structure, leading to distortions in the reconstructed manifold. By slightly increasing $k$, we include more neighbors, compensating for this sparsity and ensuring that the manifold remains well-preserved.

LocalNSA’s primary objective is to preserve local neighborhoods, and despite the changing width in Figure 13, it successfully achieves this for a wide range of $k$ values, failing slightly only when $k$ is extremely small and width is very high. The minimum requirement for preserving the manifold of the Swiss Roll is maintaining its 2D plane structure and local point neighborhood consistency. The variations in global structure observed in Figure 13 arise because LocalNSA operates without access to global information. Consequently, it converges as soon as it reconstructs the manifold perfectly. This reconstruction does not necessarily have to resemble the Swiss Roll’s original 3D geometry, as long as the manifold’s local properties are preserved. We demonstrate this using a gradient-based coloring scheme: Each point on the Swiss Roll is labeled with a color gradient that changes minimally along the manifold. In the reconstructed figure, this gradient remains consistent, confirming that while the global geometry may be distorted, the local manifold structure is effectively preserved.

Almost all the variability across the $k$ values in Figure 13 can be explained by the fact that LNSA does not preserve global structure and that the swiss roll does not align the same way in different reconstructions (since LNSA is invariant to scaling and transformations it does not preserve these when reconstructing the structure) with some minor variance explained due to the increasing width of the swiss roll.

On selecting k

The guidance provided in line 1695 is not for determining an optimal absolute $k$, but rather for maintaining an optimal relative $k$ when working with mini-batches. Specifically:

• If the dataset size is $N$ and you wish to preserve local neighborhoods up to $k_{global}$ nearest neighbors in the full dataset, the value of $k$ in mini batches ($k_{mini}$) should be chosen to maintain the following proportion:

This ensures that the mini-batch LNSA value approximates the global LNSA value

• There is no specific value of $k$ that works "best" for the swiss roll experiment in Figure 13 as we do not place any constraints on approximating the mini batch LocalNSA
• For example, if we were working with the Swiss Roll dataset from Figure 13 and had specific local neighborhood consistency constraints, for a full dataset size of 20480 and a mini batch size of 128, we would set the values using the following proportionalities:
  • Set $k_{mini} = 2$ if we want $k_{global} = 320$ (dataset size: $k_{global}$ = 64)
  • Set $k_{mini} = 5$ if we want $k_{global} = 800$ (dataset size: $k_{global}$ = 25.6)
  • Set $k_{mini} = 10$ if we want $k_{global} = 1600$ (dataset size: $k_{global}$ = 12.8)

Thus if you had a specific degree of local neighborhood preservation you wanted to guarantee on your dataset, you could do so by appropriately adjusting the $k$ value to the mini-batch size. But for most datasets we observed that a wide range of $k$ values work very similarly in practice.

While experimenting with DCA to evaluate specificity and sensitivity tests, we encountered several limitations in its current definition.

• DCA and similar generative model evaluation metrics are unable to compute $\text{metric}(A, B)$ when $\text{dim}(A) \neq \text{dim}(B)$. This restriction prevents their use in generating specificity heatmaps, as they can only compute values across corresponding layers (i.e., along the diagonal). The inability to evaluate intermediate embeddings for mismatched dimensionalities severely limits our ability to evaluate their performance against structural similarity metrics.
• Furthermore, during our experiments, we observed that the precision and recall values from DCA dropped drastically as the dimensionality of the embeddings increased. For example, while the final layer embeddings (10 dimensions) yielded precision and values around 0.98, the first layer embeddings (4096 dimensions) return a recall and precision of 0. This drastic disparity makes it nearly impossible to generate meaningful sensitivity plots across layers.
• Additionally, given the same experimental setup, we found DCA to be significantly inefficient even when using GPUs. Computing the metrics once on a subset of 3000 points in 4096 dimensions takes ~7 minutes. This is magnitudes higher than the compute cost of NSA, which takes only a few seconds for larger subsets and much larger dimensionalities (we present empirical runtimes on 200,000 dimensional data in Appendix H).
• The reliance on multiple separate metrics (precision, recall, authenticity etc) rather than a single global similarity measure further complicates the process of comparison and their utility in structural similarity tasks.

These limitations make us believe DCA and other generative model evaluation metrics are best suited to compare output embeddings only and are ill-suited for comparison with structural similarity metrics like NSA, CKA and RTD, which are designed to evaluate and quantify discrepancies across spaces irrespective of dimensionality differences or the specific layer being analyzed.

We hope this satisfies the reviewer's concerns regarding comparison of NSA to these metrics. Please let us know if you have any additional questions.

We thank the reviewer for their feedback. We address their concerns below:

1 Dependence on Euclidean distance in high dimensional spaces might not be effective in high dimensional spaces

We appreciate the reviewer’s observation regarding the potential limitations of Euclidean distance in high-dimensional spaces due to the curse of dimensionality. While it’s true that Euclidean distance can face challenges in high-dimensional settings, we chose it deliberately for a few key reasons:

1. Prevalent Use Across Neural Network Tasks: Euclidean distance remains a widely adopted metric in representation learning, particularly in tasks involving contrastive and triplet losses, where it has shown strong empirical performance. Many state-of-the-art neural network architectures still rely on Euclidean distance for computing similarity between embeddings, even in high-dimensional spaces.
2. Empirical Robustness: In our experiments, NSA demonstrated robust performance across multiple datasets and tasks, even when applied to high-dimensional embeddings (we test up to 200,000 dimensions). This suggests that, while Euclidean distance may not be theoretically optimal under the curse of dimensionality, it continues to yield practical benefits in diverse applications, including representation analysis and alignment tasks.
3. Potential for Adaptability: The NSA framework is flexible enough to accommodate alternative distance metrics if future work finds this necessary. In this study, we prioritized Euclidean distance for its simplicity, efficiency, and empirical success in maintaining local and global structural alignment across different model architectures.

Additionally we include a local component for NSA for this specific purpose. LocalNSA only looks at the k-nearest neighborhood of a point and can alleviate some of the discrepancies that can arise from GlobalNSA running at high dimensionalities.

2. No Access to source code

Thank you for your feedback. We do provide access to our source code in Appendix T (Appendix R in the un-revised version), where detailed instructions for reproducing our results are included. We will update the codebase with the experiments performed for the revision and update the anonymous repository soon.

3. Lack of universality without parameter tuning. NSA's performance across different tasks relies heavily on parameter tuning and specific integration with other loss functions. The choice of k in the construction of k-nn graph is essential in the definition of LNSA. The weights in front of the local and global parts of NSA clearly lead to drastically different results depending on their values.

Thank you for your feedback. We kindly direct the reviewer to Global Response 1 where we discuss the parameter tuning requirements for NSA. We present extensive ablations for $l$,$g$ and $k$ in Appendix Q where we show that NSA is mostly effective on a large range of values. We also added visual ablations in Appendix Q on the swiss roll dataset to demonstrate how the value of $k$ could potentially affect convergence.

4. No thorough guidance is provided for the choices and tuning of these hyperparameters. For example how to do 'appropriately adjusting the number of nearest neighbors considered in each mini-batch' on line 366 remains unspecified.

Thank you for your feedback. We have included an experiment in Appendix Q demonstrating empirical convergence of LNSA with the right value of $k$ and best practices for selecting $k$. As stated in Section 4.2.1 LNSA does not formally converge with mini-batching but has several favorable properties that help it perform well. For example, consider the scenario without mini-batching, where we examine a k-sized neighborhood of each point. When using mini-batches of size N/10, in expectation, we have k/10 points from the original neighborhood present in each mini-batch. Therefore, using LNSA with the number of nearest neighbors set to k/10 results in comparable outcomes to the non-mini batched case. By extension this applies to any fraction. We demonstrate this in Figure 12.b in Appendix Q.2

We thank the reviewer for their feedback and for appreciating our work. We address the reviewer's concerns below:

1. The reliance on Euclidean distance as a primary metric may limit performance in high dimensional spaces due to curse of dimensionality

We appreciate the reviewer’s observation regarding the potential limitations of Euclidean distance in high-dimensional spaces due to the curse of dimensionality. While it’s true that Euclidean distance can face challenges in high-dimensional settings, we chose it deliberately for a few key reasons:

1. Prevalent Use Across Neural Network Tasks: Euclidean distance remains a widely adopted metric in representation learning, particularly in tasks involving contrastive and triplet losses, where it has shown strong empirical performance. Many state-of-the-art neural network architectures still rely on Euclidean distance for computing similarity between embeddings, even in high-dimensional spaces.
2. Empirical Robustness: In our experiments, NSA demonstrated robust performance across multiple datasets and tasks, even when applied to high-dimensional embeddings (we test up to 200,000 dimensions). This suggests that, while Euclidean distance may not be theoretically optimal under the curse of dimensionality, it continues to yield practical benefits in diverse applications, including representation analysis and alignment tasks.
3. Potential for Adaptability: The NSA framework is flexible enough to accommodate alternative distance metrics if future work finds this necessary. In this study, we prioritized Euclidean distance for its simplicity, efficiency, and empirical success in maintaining local and global structural alignment across different model architectures.

Additionally we include a local component for NSA for this specific purpose. LocalNSA only looks at the k-nearest neighborhood of a point and can alleviate some of the discrepancies that can arise from GlobalNSA running at high dimensionalities.

2. NSA is versatile but may require careful tuning and modifications to work effectively in specific scenarios

Certain applications may benefit from tuning NSA parameters to optimize performance. However, our findings suggest that NSA requires minimal parameter adjustments compared to traditional manifold learning techniques such as ISOMAP, where parameter selection can heavily impact results. NSA’s tuning process is straightforward and produces only incremental gains, indicating that the method performs robustly without extensive parameter optimization.

To provide further transparency, we included ablation studies in Appendix Q to identify the optimal parameter configurations for NSA and present a more detailed response to the need for parameter tuning in Global Response 1.

3. The limitations of NSA are not explored beyond high-dimensionality issue

Thank you for raising this point. We acknowledge that while NSA has proven effective across a variety of tasks, it has a few limitations worth noting:

1. Minor Parameter Tuning: NSA requires minimal parameter tuning, though, as shown in our ablation studies (Appendix Q), these adjustments result in only small gains compared to manifold learning methods like ISOMAP, which are more sensitive to parameter selection.

2. High-Dimensional Euclidean Distance and Geometric vs. Topological Similarity: As noted, NSA leverages Euclidean distance, which can be affected by high-dimensionality. However, the versatility and widespread success of Euclidean distance in neural network applications make it a pragmatic choice for NSA. To address this, we present studies in Appendix R where we evaluate the reconstruction of a Swiss Roll using NSA-AE and demonstrate how geodesic distance can be used as replacement for euclidean distance in high dimensional spaces or in spaces where the manifold lies in a lower dimension than the original data. NSA-AE with geodesic distance as the distance measure can unravel complex manifolds.

3. Inability to measure functional similarity: NSA is primarily a structural similarity index and is therefore incapable of measuring functional similarity accurately. We illustrate this with cross architecture experiments in Appendix V. While NSA shows promise, it is incapable of perfectly capturing layerwise similarity even though both GNN architectures are trained on the same task and on the same dataset making them functionally similar.

These considerations are relatively minor and do not affect NSA's definition and superiority in its domain, and our experimental results demonstrate NSA’s robustness across datasets and tasks.

We thank the reviewer for taking the time to evaluate our work in detail and for their comprehensive review. We have added several experiments to the revised manuscript as responses to the raised concerns and present our responses to the queries below:

1. Figures on the left in Figure 1 are wrong.

We apologize for the error. We have fixed this in the revised manuscript

2. Specificity assumptions, especially in ResNets

Thank you for the constructive feedback. Numerous studies have shown that similar layers across neural networks trained on the same data, differing only in initial weights, often exhibit high structural similarity upon convergence. This has been observed across various architectures, including ConvNeXt, VGG, ResNets, and Transformers, as demonstrated by prior structural similarity metrics. We present a more detailed response to this in Global Response 3. We also provide additional results in Appendix V showing consistent layerwise similarity across multiple GNN architectures and datasets, further supporting this phenomenon across diverse network types. For additional evidence across architectures and datasets, we refer to [1,2,3], which provide a broader set of results on various architectures. Since NSA performed on par or better than these metrics in our tests, we expect it to extend effectively to other architectures not explicitly covered in this paper.

Explanation of ResNet Analysis and Layer Selection: Our ResNet similarity heatmaps take outputs from the end of each residual block rather than from intermediate layers within blocks. This choice aligns with findings from Kornblith et al.[1] with CKA, where ResNet layerwise comparisons show distinct patterns only when examining post-residual outputs. When every layer (including intermediate layers) is compared, as in Figure 4 of [1], the similarity matrix forms a grid-like pattern due to mismatches within block layers. By focusing on outputs at the end of each block (8 blocks in ResNet-18, plus the final fully connected layer), we observe a clearer similarity pattern across instances of the same architecture. This approach also aligns with findings by Veit et al. (2016)[4], which demonstrated that in ResNets, most significant gradient flow occurs along the short paths (skip connections and post-residual outputs), as longer paths contribute minimal gradients.

Experimental Setup: For thoroughness, we compute average similarity over 10 trials for each metric when using it in subsets. For Figure 1, we average the performance across 3 separate runs (6 models trained with different seeds forming 3 pairs of comparisons). The standard deviation of most metrics on subset computation is a few orders of magnitude below the mean value and does not affect the results, so we do not include it in Figure 1. For reference we provide the standard deviation on both RTD and NSA in Table 1 and provide standard deviation of NSA's individual components in Appendix Q. Additionally we also present layerwise specificity tests in Appendix U where we show the standard deviation of the metrics across runs for Figure 1's specificity tests. We also report layerwise similarity results of ResNet-18 and ResNet-34 trained on ImageNet in Appendix I for further validation on image datasets. We hope these additional experiments will alleviate the reviewer's concerns.

3. Query on Equation 6

Thank you for your question. While LocalNSA is inspired by the Local Intrinsic Dimensionality (LID) measure proposed by MacKay and Ghahramani, we intentionally deviate from their formulation for the following reasons:

• LocalNSA's Objective: While LID computes the dimensionality of the entire space, LocalNSA does not aim to measure the LID itself. Instead, it is designed to quantify pointwise discrepancies between two spaces, ensuring that its theoretical properties remain valid.
• Pointwise Discrepancy Requirement: LocalNSA needs to operate at the pointwise level to preserve its role as a structural similarity metric. This requirement distinguishes it from global estimators like the one in MacKay and Ghahramani's formulation.
• Normalization and Range Stability: The use of the inverse in LocalNSA is a deliberate design choice. This ensures that: (a) LocalNSA values remain consistent across different mini-batch sizes, maintaining stability during training. (b) The values stay within a reasonable numerical range, avoiding issues such as skewed loss functions. As demonstrated in Figure 3b of Mackay and Ghahramani’s note, directly averaging the inverses (as in their estimator) results in a much wider range of values, which could lead to instability when used as a loss function.
• Theoretical Properties: The choice to use the inverse does not affect LocalNSA's theoretical properties. For instance, the convergence of x−y is equivalent to the convergence of 1/x - 1/y. Thus, the design choice to not inverse again is primarily for practical and numerical convenience.

4. GNSA can have problems with Swiss Roll and similar datasets

Thank you for your valuable feedback. We agree that GNSA, as currently formulated with Euclidean distances, preserves the exact geometric structure rather than the topology of the manifold. This design choice means that NSA is sensitive to changes in global distances, as observed in cases like the Swiss Roll dataset when flattened. Appendix R specifically addresses this situation, highlighting that NSA’s use of Euclidean distance is intended to preserve geometric fidelity rather than topological structure.

To extend NSA’s applicability to manifold topology, one can substitute Euclidean distance with geodesic distance. This modification enables NSA to capture the underlying manifold by approximating geodesic distances between points, preserving topological similarity while maintaining all NSA properties. Appendix R compares NSA-AE’s performance on the swiss roll dataset when it aims to minimize euclidean distances vs geodesic distances.

5. Addition of toy and visualization focused datasets

Thank you for your feedback. In our study, we incorporate several well-established visualization focused datasets to validate NSA’s efficacy and to demonstrate its flexibility across different use cases. We also added results from the Mammoth dataset taken from [5] in the revision. Please find below a list of visualization focused datasets in the manuscript:

• In Section 4.2, we utilize the COIL-20 dataset to evaluate NSA’s ability to preserve structural similarity during dimensionality reduction.
• The Swiss Roll dataset is used to illustrate the difference between geometric and topological preservation. By comparing NSA with Euclidean distances to NSA with geodesic distances, we show how the choice of distance metric influences NSA’s ability to capture manifold structure. The Swiss Roll dataset is also used to perform ablation studies on the parameter $k$ of LNSA.
• In Appendix R, we analyze NSA’s performance on toy datasets like Spheres and Mammoth to evaluate its capacity to maintain both global and local structure. Additionally, we also use the Spheres dataset to perform visual ablation studies in Appendix Q that demonstrate how different parameter choices affect NSA’s performance.
• In Section 4.4 we demonstrate how NSA can pinpoint sources of discrepancy by switching to a node wise variation of NSA (refer to the formula here). This showcases NSA’s ability to not only identify structural discrepancies but also pinpoint the source of these deviations.

6. Mean and Standard Deviation values for Figure 1 and Figure 3

Thank you for your feedback. As previously stated, we compute average similarity over 10 trials for each metric when using it in subsets. For Figure 1, we average the performance across 3 separate runs (6 models trained with different seeds forming 3 pairs of comparisons). The results on mean and standard deviation values across runs and the experimental setup is presented in Appendix U. In Figure 3 we run our experiments on the Cora dataset which has 2708 nodes and hence we do not need to use subsets to compute NSA. We have clarified in the main text

7. Clarifications on the sensitivity test

Thank you for the question. In principle, the idea proposed by Ding et al.[6] states that as principal components are sequentially removed, starting with the lowest-variance components, the dissimilarity score should increase. A good metric should be sensitive to the removal of these high-variance components. Figure 1 demonstrates that NSA’s dissimilarity score rises significantly when high-variance PCs are removed, indicating greater sensitivity to impactful structural changes. In contrast, RTD and CKA show a flatter response, reflecting reduced sensitivity until the most significant components are removed. In line with Ding et al.'s[6] setup, we perform this analysis on the representations extracted from the first and last layers of a neural network.

Since different metrics operate within different ranges, the absolute values of the curves may not be directly comparable. To address this, [6] proposed a detection threshold, which defines the dissimilarity score above which changes become “detectable.” This baseline is the dissimilarity score between representations from differently initialized networks, serving as a reference for determining when structural changes due to PC removal become significant.

The main text refers to the average percentage of principal components that can be removed before crossing the detection threshold. The confusion arose because we referred to this average percentage as the detection threshold itself. We have updated the text to clarify this distinction.

We sincerely thank the reviewer for engaging in a discussion and their feedback.

1. Clarification on Geodesic Distance computation

We use the Floyd-Warshall algorithm to approximate geodesic distances, ensuring that the pairwise shortest path distances on the manifold are accurately represented. We first create a k-nearest neighbors graph from the dataset then compute the all pairs shortest path using the FW algorithm. Given that the reference space remains fixed, this is a one time preprocessing cost. And since this matrix inherently reflects Euclidean distances when the representation space is reduced to its manifold dimension, we can use NSA with no modifications to the metric to operate as a loss function. We have added a line in Appendix R mentioning the algorithm used. We hope this clarifies the process used to perform the swiss roll manifold approximation experiment. Please let us know if you require additional details.

2. Addition of toy dataset experiments

We did perform experiments with a few toy datasets to visually observe the ability of NSA to preserve the structure of the representation space.

• Figure 14,15 and 16 showcase NSA’s ability to preserve local and global structure with the 3D Swiss Roll, Spheres and Mammoth Dataset
• In the latest revision we also added visualization experiments with a 1D spiral in 2D space in Appendix Z, where we introduce targeted transformations to the original space (we perform global structure preserving transforms, manifold preserving transforms, structure altering transforms and manifold altering transforms) and present the GNSA and LNSA values to demonstrate their invariances and properties. Both GNSA and LNSA reflect the changes made to the ground truth based on their definitions i.e GNSA changes when the global structure is modified, LNSA changes when local structure is changed.

We hope this satisfies the reviewer’s concerns on using Toy Datasets to visualize the effects of NSA.

While I appreciate the authors' efforts, in my view the changes to the main text of the manuscript were minimal, despite important concerns raised by me and other reviewers. The material added to the appendices does not convincingly demonstrate how NSA represents a significant advance as a structure representation metric other than in the particular examples chosen. Thus I will maintain my original score.

3. Different initializations produce different models and there is no reason to assume these should have the same structures

The idea that corresponding layers of architecturally similar models show high relative structural similarity is not novel to NSA but is a well-established benchmark for evaluating similarity metrics. This approach was first introduced by Kornblith et al.[1] in their seminal work on CKA, and it has since been validated and adopted by several subsequent studies [2,3,4,5,6,7]. The idea of similar generalization and performance between networks trained with different seeds has been extensively examined by Thomas et al.[8]. Their experiments with 100 BERT models confirm that, while functional differences may lead to significant variability in out-of-distribution (OOD) performance, networks trained to convergence on the same dataset exhibit highly similar structural representations, particularly for in-distribution data. This idea is key to the validity of not only the layerwise analysis presented in similarity metric papers but also to the grounding tests proposed by Ding et al [7]. This structural alignment is a key factor contributing to the consistent performance of such networks on in-distribution tasks.

All our specificity tests are conducted exclusively on in-distribution data, ensuring that our analysis aligns with the well-validated findings from prior literature. We have fixed the error with the figures in Section 4.1 and also improved the writing on Specificity tests. We also include an in-depth layerwise breakdown of the specificity test in Appendix U.

In our initial presentation, we claimed that a good structural similarity index should show high similarity between architecturally identical networks trained with different weight initializations. We have revised this claim to a more nuanced one: a good similarity index should exhibit the highest relative similarity for corresponding layers in architecturally identical networks with different initializations, compared to non-corresponding layers.

We present a detailed changelog of all the improvements made to the manuscript in this revision:

• Fixed the error in Figure 1 (left) which had the same figure for all 3 metrics.
• Improved the readability of some figures by replacing them with higher quality versions
• Added reference to Figure 3 that was missing
• Rewrote Section 4.1.1 and improved the clarity of Section 4.1.2
• Added additional Specificity Tests in Appendix U
• Added ablation studies visualizing the effect of k on the performance of LocalNSA in Appendix Q.
• Added explanations for the choice of k and figures demonstrating empirical convergence of LocalNSA in mini batching scenarios to -Appendix Q.
• Added reconstruction results of NSA-AE on the Mammoth Dataset to Appendix R -Added GCN performance on Link Prediction at different dimensionalities in Section 4.3
• Added heatmaps showing cross layer similarity performance on a subset of the ImageNet dataset (100K images) on ResNet-18 and ResNet-34 to show NSA’s performance on large datasets in Appendix I
• Added a plot to show the variation of mean NSA and standard deviation of Global NSA and Local NSA to show the standard deviation of both metrics is significantly lower when working with subsets in Appendix Q
• Added results on NSA’s correlation to misclassification rate in CNNs in Appendix I
• Added visualizations on the 1D spiral in 2D Space dataset to provide simple visual explanations on how LNSA and GNSA reflect the ground truth when different types of transforms are applied to the data in Appendix Z.

[1] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neural network representations revisited. IN ICML 2019

[2] Serguei Barannikov, Ilya Trofimov, Nikita Balabin, and Evgeny Burnaev. Representation topology divergence: A method for comparing neural network representations. In ICML 2022

[3] Enric Boix-Adsera, Hannah Lawrence, George Stepaniants, and Philippe Rigollet. 2022. GULP: a prediction based metric between representations. In NeurIPS.

[4] Zuohui Chen, Yao Lu, Wen Yang, Qi Xuan, and Xiaoniu Yang. 2021. Graph-Based Similarity of Neural Network Representations. ArXiv preprint (2021)

[5] Trofimov, I., Cherniavskii, D., Tulchinskii, E., Balabin, N.,Burnaev, E., and Barannikov, S. Learning topology preserving data representations. In The Eleventh International Conference on Learning Representations, 2023.

[6] Klabunde, M., Schumacher, T., Strohmaier, M., and Lemmerich, F. Similarity of neural network models: A survey of functional and representational measures.

[7] Ding.,et al. Grounding representation similarity through statistical testing. Advances in Neural Information Processing Systems, 2021

[8] R. Thomas McCoy, Junghyun Min, Tal Linzen. BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance. 2020 BlackboxNLP workshop