Estimating Shape Distances on Neural Representations with Limited Samples

We thank the reviewer for their positive and thoughtful comments on our work.

When referencing the shape distances such as (2) and (3) the authors cite Williams 2021 but these are simply common shape distances which have been around for much longer. Unless this specific formulation is novel to Williams 2021 I don't see how it's different than just measuring the distance on a circle.

We now clarify that the shape metric estimator is a common formulation and include older citations in the introduction. Specifically, we emphasize in a new paragraph 3 of the introduction that although shape distances are drawn from an older, established literature, they were, to the best of our knowledge, proposed as a method to measure similarity in neural network representations by (Williams et. al. 2021). We then cite this paper frequently due to the fairly self-contained appendix that explains these concepts while using notation that is similar to ours.

When considering the "failure modes of plug-in..." there is the missing component of mentioning scale. If $h_i$ differ in scale their distance can be large but their distance can be very small as it rescales in the denominator.

The reviewer’s understanding is correct. We have now added a sentence to convey this explicitly to the reader under the definitions in equations (2) and (3). .

Answers to questions:

When setting up the background, the authors define as a function representing each neuron. I have no problem with this definition. My question is, why do the authors limit their writing to that of NNs while this is true for a much wider class of functions. I may have missed something in an assumption but it seems these functions can be quite generally considered.

We agree that the estimator is broadly applicable, we only need IID responses from a function where the responses have finite covariance. We have crafted our exposition to make the paper more accessible to practitioners studying neural networks. However, we have edited the text to clarify that the theory we develop can be considerably generalized to measure shape distances between any two functional mappings (not only neural nets).

For (5) isn't the last equation enough?

We have notated the covariance matrices (for systems i and j) and cross-covariance matrices (between systems i and j) separately to make their distinction and definitions in equation (6) clear. We have also reformulated the text above equation (5) to make the definitions easier to parse for the reader.

We thank the reviewer for their positive and thoughtful comments on our work. We agree that the mathematical details of this paper can be difficult to digest. To address the reviewers point that our paper should contain more intuition-building sentences to help the reader digest the mathematical formulation, we have added the following:

• Edited wording in front of eq. 1 to make clear that these are assumptions.
• Beginning of appendix A: “We can intuitively think of the Procrustes distance as the Euclidean distance between two vectors remaining when the rotations and reflections have been “removed”. Similarly, the Riemannian shape distance can be thought of as the angle between two vectors after these rotations and reflections are removed. These definitions in eq. (2) and eq. (3) also make clear that Procrustes distance, like Euclidean distance, is sensitive to the overall scaling of $h_i$ or $h_j$, while the Riemannian shape distance, like the angle between vectors, is scale-invariant.”
• Added wording above eq. (5) to improve clarity.
• We have revamped the introduction and edited the text (in response to this and other reviewer’s comments) to improve clarity as much as possible while remaining within the page limit.

We have now included an experiment where we compare the representation of two neural networks trained on the same task but from different initialization points and with different training procedures (Appendix E: Applications to deep learning). Interestingly we find that the plug-in estimator, similarly to simulations, shows a high bias that slowly converges as we increase the number of samples, but the moment estimator maintains little bias regardless of the number of samples (Fig 5). We discuss how the ways in which the plug-in estimator’s bias can depend on irrelevant nuisance variables including number of samples and effective dimensionality may lead to erroneous scientific conclusions about the similarity between networks.

We thank the reviewer for their feedback and for their positive comments regarding clarity ("the paper is well-written" and "proof techniques are well-explained") and quality ("experiments are convincing"). Our reading of their review shows two major concerns which we address below.

First, It seems like the major concern is that the work is "out of scope" because there "is no link with neural networks." While we think this is overstated, this was useful feedback that helped us revise and re-frame the paper to appeal to a broader audience.

• In our revision, we emphasize that Procrustes/shape distances are actively being applied to artificial neural networks in the literature (see: here, here, here, here, here, and here). More broadly, there is vast literature on comparing hidden layer representations in artificial networks (see 24 page review by Klabunde et al.). Notably, this interest is highly interdisciplinary between machine learning, neuroscience, and psychology as reviewed by Sucholutsky et al. This interdisciplinary interest is why we chose to submit this paper to the Neuroscience & Cognitive Science track of ICLR, and why our original submission contained an application to biological networks. Given all of this, plus the positive feedback from the other three reviewers, we think it is clear that our work is a good fit for ICLR.

• To make our envisioned applications more clear and concrete for the ML community, we now include a new direct application of our method to artificial networks (Appendix E: Applications to deep learning ). We show that plug-in estimators of shape distance converge slowly, and our new method provides a viable alternative (Fig 5). The details of this new experiment are described in our global response to reviewers.

• The reviewer says our paper has "nothing to do with artificial neural networks" because our theorems are applicable to a more general setting (i.e. they could apply to "any r.v. X"). In one sense this is true—our theorems apply to a fairly broad variety of problems. We argue that this generality is a strength not a weakness! We agree with the comments from reviewer emQE who suggested that we edit the text to explain that our analysis (and the shape distance) can be used to compare any two functions, not just neural networks. Such comparisons between functions may be useful in other contexts within machine learning (e.g. quantifying differences between functions sampled from a Gaussian Process).

Second, the reviewer brings up a concern about "novelty." However, the concern is less about novelty and more about the technical sophistication ("all proofs are classical bias-variance decomposition and concentration inequalities"). We show below this concern is based on a fundamental misunderstanding of what we proved; the techniques involved are considerably more complex than the reviewer suggests. The reviewer also requests that we add two citations, but does not explain why these prior references reduce novelty of our results. Overall, we think these concerns are not generous to our efforts.

• First, simple but novel mathematical proofs should be viewed as a strength! Contrary to the reviewer, we believe that results are particularly valuable and accessible when the mathematical techniques are established, but the conclusion is novel. This is especially true for an applications-focused venue like ICLR. It would be fair to count novelty as a weakness if our theorems / practical guidance afforded practitioners were already widely known. But the reviewer does not make this case against the paper.

• We put considerable effort into refining the proof arguments and explaining their intuition to a broad audience. But this clarity should not be confused with the proofs being obvious. Furthermore, we use a broad range of mathematical techniques rooted in several fields. We use random matrix theory (not mentioned by the reviewer) to prove Theorem 2. We use principles from convex optimization (epigraph reformulation) to arrive at a tunable bias-variance tradeoff in our method-of-moments estimator. We use the Matrix Bernstein inequality to prove theorem 1; this inequality is not a "classical concentration inequality" in the usual sense of the term---it was derived in contemporary literature by Tropp (2010), and the proof is quite involved (not just a simple application of Markov's inequality). Altogether, we believe the combination of approaches is non-trivial and merits the community's interest.

• We thank the reviewer for the references, and we have added them to our literature review in section 2, ‘Background and Problem Setting’, as well as in paragraph 3 of our introduction. These classic papers both motivate the importance and heritage of the problem we consider here, but are largely concerned with the problem of estimating form and shape from noisy and nuisance-transformed data, as opposed to estimating shape distances, and in particular, the convergence properties and uncertainty of estimating shape distances. Specifically, Goodall 1991 covers various methods for mean shape estimation in special cases, but contains only a brief discussion of shape differences, and no discussion of the statistical estimation of shape distances in the same form as we do here. Kent 1997 studies the asymptotic consistency of Procrustes estimators, again of shape, not shape distances, and in the special, restricted case of 2-dimensional shapes (configurations of points in a plane). Our results are non-asymptotic and provide explicit rates of convergence. This is a significant advance over proving asymptotic consistency.

Responses to questions:

Which results would your method yield in the latent space of an artificial neural network?

In our original draft we did not apply the estimator to artificial networks. We have now run the requested experiments and shown in this setting the plug-in estimator shows a significantly larger bias than the moment estimator (see Appendix E and Fig 5).

We thank the reviewer for their thoughtful comments about our work, in particular for highlighting our novel theoretical analysis of this problem.

We disagree with the reviewer’s comment that “experiments showing tangible gains on at least one real world application would greatly strengthen the paper” since our original submission did include an application to experimental neuroscience where data are noisy and sample-limited. Nonetheless we acknowledge that the importance of studying the uncertainty in estimating shape distances may not have come across clearly in our introduction. We have added paragraph 3 in the introduction to make clear that, although the Procrustes and shape distances brought to light by Williams et. al. (2021) are currently being applied in this field to measure representational similarity between both artificial and biological neural networks (see: Schuessler et. al. 2023 (https://arxiv.org/abs/2307.07654), Lange et. al. 2022 (https://arxiv.org/abs/2206.10999), Boix-Adsera et. al. 2022 (https://arxiv.org/abs/2210.06545), Alvarez-Melis et. al. 2018 (https://arxiv.org/abs/1806.09277), and Ding et. al. 2021 (https://arxiv.org/abs/2108.01661)), there has been little progress in understanding the theoretical and empirical behavior of the estimators for these distances. This is a major gap in understanding, because—as we show—the naive plug-in estimators are biased and can converge slowly, and there are circumstances when less naive estimators such as the estimator we propose can be advantageous. For instance, Figure 2B shows us that in the fixed dimensionality case, one may reduce the variance of an essentially unbiased estimate by increasing the number of stimuli sampled, faster than the bias of the plug-in estimator can be reduced in the same way.

In our additional experiments on an artificial neural network (introduced in the rebuttal period), we can see that the bias of the naive plug-in converges slowly with respect to increasing samples whereas the moment estimator maintains small bias (Fig 5, Appendix E). These findings are relevant to the active research being performed to compare representations in neural networks using Procrustes/shape distances (see citations in previous paragraph). Furthermore, the bias of the plug-in estimator may depend on nuisance properties of the populations being compared leading to confounds. We concretely demonstrate this by showing that the bias of the plug-in estimator depends on the effective dimensionality of the neural populations (Fig 6, Appendix E). Thus plug-in estimated differences across experimental conditions could simply result from differences in effective dimensionality and not the actual similarity. The quantification of bias in the moment estimator allows the practitioner to precisely determine if differences cannot be explained by estimator bias – a highly practical scientific benefit to the novel estimator.

Lastly, we would like to emphasize that the objective of this paper is not to provide a universally ‘correct’ choice of estimator for shape distances, and we have added wording to make this fact more clear (introduction, final paragraph). As the reviewer points out, there are cases when the plug-in estimator performs comparably or better than the method-of-moments estimator. The method-of-moments estimator then provides the practitioner a way to study the uncertainty of shape estimates and explicitly trade off bias and variance by imposing a bias constraint. The reviewer is correct that this can mean for some choices of bias constraint that the variance becomes so large that the estimator is not useful. However, this ability to trade off bias and variance with a tunable parameter is, we believe, a significant contribution. We also emphasize that this estimator is just one of the results of our paper. Theorems 1 and 2 theoretically characterize the plug-in estimator and provide worst-case performance guarantees, potentially guiding the design of biological experiments which use shape distances. We also provide intuition regarding the bias of the plug-in estimator and how the error decreases as a function of neurons and input samples, and how this can converge slowly. We believe that the culmination of these results and novel estimator methodology is an important step toward a thorough understanding of the statistical challenges of estimating shape distances.

Answers to questions:

In the last equation on page 5, why is $x \in [0,1]$?

$x$ is in $[0, 1]$ for mathematical convenience. To optimize the polynomial approximation we must choose a domain – ideally this domain would be between the min and the max singular value of the cross-covariance matrix. Because both are unknown we set the lower end of the domain to be 0 (singular values are positive) and we estimate the maximum singular value then scale the data so that it is 1. This is the same approach taken in Adams et al. (2018).

In Section 4.2 it is acknowledged that the proposed estimator is highly variable in low SNR regimes and so neurons with the highest SNR are selected. Can you provide the relative proportion of such neurons in the presented scenario and also (if possible) comment on their prevalence in general? This is important because if the relative proportion is small then it means that the proposed estimator can only be applied in very few cases.

These 40 neurons are the maximal SNR units so they are exceedingly rare, 40 out of 10,000 neurons recorded. This is a wide-field calcium imaging experiment— known to be noisy. These results suggest that accurate shape metric estimation in large neural populations will require higher SNR, potentially with electrophysiological recordings. The confidence intervals of our moment estimator quantify this whereas the plug-in estimator naively interpreted would have led to biased results.

Thank you for the detailed response. I appreciate the honest acknowledgement of the limitations of the proposed estimator and the modification of the wording in the introduction to reflect this and I do not think that this takes anything away from the merits of the work. I had already recommended accepting the paper and so I will keep my score unchanged.