Quantifying Task-relevant Similarities in Representations Using Decision Variable Correlations

Thank you for your very detailed feedback and questions on our work. For several of these points, we have now performed additional analysis to address them. We feel that by addressing these critiques, the paper has been greatly improved.

(1) Explain the decision variable: Your intuition is not far off! The decision variable is a term derived from signal detection theory that is frequently adopted by behavioral psychologists as a model for decision making, which is why it is the perfect base ingredient for this method. It is a continuous variable that represents the sensory evidence of the observer. A threshold (the separatrix) is further used to make the decision. So here it is the projection onto the axis orthogonal to the separatrix.

(2)Justify split normalization: Your question about if the split halves are truly two copies of the same information (decision variable) contaminated by independent noise is an interesting one. Following your suggestion, we have performed additional analysis by decoding the individual splits. Although there is no ground truth available for the decision variable, we find the decoding performance to be high for the individual splits. In addition the DVC estimates with random splits are very consistent across repeats (normalized DVC for monkey vs. monkey is 0.568 ± 0.014), showing that it is likely that splits contain the same copy of task-relevant information. This result is consistent with the general idea that information about the task is distributed in the network. The split normalization, as you would expect, rescues DVC relating to noisier observers, making the monkey-monkey, monkey-model DVC higher than would otherwise be. As the math (described in the Supplementary Information section A1) shows, under the correct (if somewhat restrictive) assumptions, the procedure recovers the ground truth DV.

The splits are on the neurons instead of the images because we are trying to account for the overall noise in the representation instead of the image sampling noise. The sample size is large enough in this case (400 images in each category) that the latter is miniscule. However this could be useful in cases where the sample size is smaller.

(3)The relationship between Kappa and bias: This is a good question. The details on the simulation that addresses the relation between Cohen’s Kappa and decision biases are provided in the Supplementary Materials Section A2. We apologize if the description has been unclear. The general intuition is that DVC is agnostic to the behavioral biases of the observers while Kappa is inflated when there is a shared bias between observers. Low bias does not enforce a low Kappa.

(4)Accuracy of linear decoder on monkey recordings: Thank you for this suggestion. An optimal linear decoder on monkey V4/IT recordings actually performs very well (average pairwise LDA score 0.92 & 0.94 for the two monkeys respectively, which is at a similar level as the networks ~0.96) so it is very unlikely to be the cause of differences in DVC. The accuracy for the logistic regression classifier for all the classes (instead of binary classification) is 0.80 & 0.81, respectively. With cross validation it drops to 0.74 & 0.71. In addition, the decision variable is conceptually decoupled from accuracy, since it cares about the information used to make the decision, not the decision itself. Which makes it great for addressing behavioral differences by different observers. We will include this result into the revised version of the paper.

(5)Pairwise or one vs. the others: Our choice of focusing on pairwise classification instead of one vs. the others classification is in part due to the fact that we use a smaller dataset. However, also consider that between two (e.g. gaussian) distributions with equal covariances, the best classifier is linear. But in the case of one vs. many gaussian distributions, it can be very hard, or even impossible, to find a good linear classifier depending on the arrangement of the classes, especially in settings where the effective signal dimension is low.

(6)The uncertainty of DVC: DVC estimate with split is very robust with the amount of samples in this dataset. As previously mentioned, normalized DVC for monkey vs. monkey is 0.568 ± 0.014, where the uncertainty comes from different splits. The uncertainty of DVC with respect to sampling can be estimated using bootstrap procedure. Thank you for this suggestion.

(7)DVC between networks initiated with different seeds: As established in studies such as Mehrer et al. 2020 (Nat Comm), networks with varying initiation can learn very different intermediate representations, therefore should be viewed as different models. To address your question, we tested DVC on Resnet50 variants trained using different initialization, augmentations, training recipes etc. Notably, DVC between models with different initialization but otherwise underwent the same training procedure is 0.80.

(8)DVC results with randomly initialized networks: We have since examined DVC on randomly initialized networks. Under special circumstances where networks of the same family are initialized using the same seed, DVC reports could be quite high. However in general DVC between randomly initialized networks fall in the 0.10 - 0.20 range.

Thank you again for taking time to carefully examine our manuscript. We hope our new analyses and the clarification above address your concerns. Please feel free to follow up with further inquiries.

Thank you for your constructive feedback. The concerns you have are very representative and we have worked on addressing them.

(1)Comparisons to other methods: Thank you for raising this great point. RSA, CKA and CCA are all useful correlation-based metrics and equivalent under certain conditions. We have compared these metrics in this dataset and find that DVC is indeed positively correlated with all the aforementioned metrics, albeit to different degrees. Of course, DVC is a supervised method that utilizes the labels to discern task-relevant dimensions from task-irrelevant ones, which distinguishes it from other correlation-based methods. The RSA between two simulated representations may be arbitrarily inflated while DVC remains the same. The fact that they are positively correlated, and the degree to which they are correlated, could help us pick apart how much of the variance in the representation is useful for task-solving. We will incorporate the new results on the comparison to relevant other methods in the final version of the paper. Thank you for your valuable suggestion which helped us improve the paper.

(2)Clarifying novelty: While the main contribution of the paper is the DVC method, we hope that we could use a principled, task-driven approach to examine the topics where other methods have been giving contradictory accounts. Take for example, the relationship between brain alignment and Imagenet accuracy, papers have reported increasing, saturating, peaking then dropping etc. While DVC reports show that the alignment has been dropping consistently since early days, with the exception of some noteworthy outliers. The finding that alignment between brains are on par with the alignment between models also contradicts previous results. We think additional novelty of these findings lies in the method used to derive them and the principles behind the method, making the findings more grounded than previous reports. We agree with the reviewer that these points regarding the novelty should be made more clear in the paper. We will revise the paper to make these points more salient.

Thank you for your positive evaluation of our paper and for providing great ideas to help further strengthen it. We address your suggestions below.

(1)Simulation to show difference between methods This is a great suggestion. The difference between unsupervised shape metrics such as RSA and DVC is that DVC focuses on the task dimension and ignores similarities that are irrelevant to the task. To demonstrate this, we simulated two observers with representations that consisted of 1D decision variable, individual noise and shared noise. As shared noise increases, RSA quickly increases and saturates towards 1, while DVC remains stable. We feel this is a great point to add to the paper because it makes the motivation of our method more clear. We will incorporate this point to the revised version of the paper.

(2)DVC to neuroimaging datasets: The existing neuroimaging datasets (e.g. Horikawa & Kamitani, 2017) typically contain hundreds or thousands of image categories, but only a handful of exemplar images in each category, which is the opposite of what we are looking for in this case (more images per category). We were looking at the popular Natural Scenes Dataset (NSD Dataset) which uses COCO images, which has no fixed number of images per category, and each image has multiple labels. This makes the NSD dataset not ideal for our purpose. If the reviewer can think of any publicly available datasets that would be appropriate for the application of our approach, please share them with us. It would be greatly appreciated and would be eager to test these out.

Thank you for your encouraging comments and constructive feedback. We find the criticisms to be very on point and have worked on addressing them with additional data.

(1)Missing citations: Thank you for giving context to some of the findings. Subramanian et al. discovered that artificial neural networks use wider channels than human observers and robust networks use even wider channels. Linsley et al. show that harmonized networks are attacked on features that are more detectable and human-aligned. In future versions of the paper, we will provide the context in the vein of “There has been an emerging consensus that even though adversarial training increased model robustness, these robust networks do not use human-like features unless explicitly aligned”.

(2)Generalizability/limited neural dataset: In the current version, we only reported data from Majaj et al., 2015, which is one of the best datasets that are available to us when considering the neurons recorded, trials repeated and data format.

To address the reviewer’s concern, we have performed additional analysis and examined more datasets. In particular, we have analyzed the dataset collected in Bashivan et al., 2019. This dataset contains only V4 neurons and a smaller number of recorded neurons – only one of the monkeys (monkey m) had sufficient neurons. As a result, the monkey-monkey DVC and monkey-model DVC concerning the monkey subjects with only ~20 neurons are lower due to insufficient sample size, but the relative scale of the DVC remains consistent. In addition, the observation that DVC drops with increasing ImageNet accuracy holds for monkey m (rho=-0.76, p=9.89e-04). We will edit the manuscripts so the words reflect the scope of implications accurately. Specifically, instead of “fundamental divergence”, we will use “substantial divergence”

(3)Robustness to decoder choice: This is a good question. To address the reviewer’s concern, we have now additionally tested the DVC with other linear decoders such as logistic regression and linear SVM and found that the estimate remained stable (for DVC between monkeys: LDA 0.567 ± 0.014, logistic regression 0.562 ± 0.017, linear SVM 0.554 ± 0.017).

Regarding nonlinear decoder, in the current work, we deliberately chose to focus on linear decoding because we are interested in understanding the trial-by-trial correlation of the linearly decodable information for classification tasks. The approach can be incorporated to deal with nonlinear decoder, however, the results obtained would be more difficult to interpret. For reference, using SVM with RBF kernel or MLP with ReLU activation seems to result in lower DVC across the board.

We will incorporate these new results in the final version of the paper. Thank you again for your thoughtful and constructive advice which further strengthens the paper. We hope the additional results we provided above help mitigate your concerns.

Thank you for your thoughtful review on our work. The main criticism is why our method is needed given there are already several prior methods to modeling the alignment of neural/behavioral responses across different observers/models. This is an important point that requires further clarification.

(1)Reason to adopt DVC over other methods: As a representational similarity measure, DVC is a supervised method that utilizes labels for task-specific comparisons, which distinguishes it from the methods that focus on general geometry. In addition, it is less biased and more principled. We think it should be able supersede previous methods where the conditions for its application are met.

DVC can be also applied strictly to behavior such as comparing models to behavior, comparing two behaving animals, or the same behaving animal on multiple occasions. In all cases, it provided an unbiased principled measure of decision correlations. For more on how DVC may be inferred from behavior, see Sebastian & Geisler, 2018.

(2)Scope of application: Biological vision systems evolve to perform two kinds of specific tasks that are relevant for survival: categorization tasks, and estimation tasks. DVC can be measured for both kinds of tasks. We are interested in understanding biological vision. Most of the rigorous studies of biological vision have been some form of categorization or estimation task. A rigorous unbiased measurement technique should find plenty of applications.

It is true that there are tasks that the DVC methods would not apply, but we have deliberatively developed our method to study a specific kind of question, in particular, how well the decision strategies of two observers are correlated on a trial-by-trial basis in classification tasks. This is a unique feature of our method that distinguishes our work from other methods, such as RSA which quantifies the similarity of the geometry of two representations.

We appreciate your thoughtful comments and will revise the paper to incorporate the key points above.