Vision CNNs trained to estimate spatial latents learned similar ventral-stream-aligned representations

Hi reviewer wqcV, a huge thank you for giving such a thoughtful and detailed review and helping us improve this paper. We will address your first question in this response and work on addressing your other questions in a later response.

Response to your questions: The ground-truth rotation of objects in our dataset used for training is defined relative to a canonical view of the object. The canonical view is tied to the object identity. Although there might be some simple heuristics to determine the canonical view of an unknown object, we believe that determining the canonical view mainly relies on correctly identifying the object first. So, as you pointed out, we also think that object rotation is entangled with object category, which may explain the spike in category decoding in rotation-trained models. The experiment you mentioned is a very interesting idea for testing this empirically; thanks for letting us know.

We are working on addressing your other comments, so stay tuned. Thank you!

• Dimensionality of the supervisory signal vs entropy. We appreciate the reviewer raising this insightful point. Ideally, we would like to measure how “strong” the supervisory signals provided to the models during training are. However, at the time of writing this paper, we were not aware of a robust measure for this, and we believe our core claims remain unaffected by the specific measure chosen. We acknowledge that using dimensionality alone has its limitations. However, using entropy as a measure has challenges as well. For instance, entropy may be meaningful when comparing distributions with the same support, but comparing continuous entropy with discrete entropy poses difficulties. Discrete entropy is always non-negative, whereas continuous entropy can take negative values. Additionally, continuous entropy is sensitive to the scale of the supervisory signal. For example, the continuous entropy of a Gaussian distribution is given by: $\frac{1}{2} \log{2\pi \sigma^2} + \frac{1}{2}$ This value depends solely on the spread of the target distribution. In practice, however, when training machine learning models, targets are often scaled to a reasonable range, and our intuition suggests that this scaling should not significantly affect the “strength” of the supervisory signal. Yet, such scaling can dramatically alter the computed entropy of the target distribution. We therefore chose to use the dimensionality of the supervisory signal as an intuitive way to visualize and present our results, rather than as a rigorous metric for comparing the efficiency of these models in utilizing supervisory signals. We did not claim that latent-trained models are superior to category-trained models in this regard. Our intention was to highlight that it is possible to achieve comparable models using supervision based on a few latent variables. If there are better ways to measure the desired properties, we are open to suggestions and welcome further input. Thank you again for your thoughtful feedback.

We hope our response answers your questions and addresses your concerns. If you have any other questions, please let us know!

We want to thank you for your effort in reviewing and helping us improve this paper.

• Layer mapping details. Thanks for pointing that out. In the new pdf, we added a supplementary figure C.4 to show the layer mapping details of our models. We also showed the layer mapping along the models’ CKA similarity to category-trained models. We found that many of the V1, V2, V4, and IT layers are mapped to layers before model representations start to diverge significantly from the category-trained models. V1 mapped layers are mostly consistent among models trained on different tasks. Compared to category-trained models, the V2, V4, and IT layers are usually shifted to earlier layers in spatial latent-trained models.

• Fine-grained predictivity-to-performance correlation analyses. In our new pdf, we added a supplementary figure C.6 and supplementary table C.3, which shows a more fine-grained analysis of how model performance in the trained latent task correlates with individual V1, V2, V4, IT, and behavior alignment scores separately. We found that all spatial latent estimation performance correlates strongly with all individual region neural scores. Distance and translation performance do not have a strong positive correlation with the behavior score, while rotation and category-trained models do. Our goal for the predictivity-to-performance analysis was to identify causally what training objectives (functional roles) could drive neural alignments if someone were to train a model, so we only analyzed the overall model performance on target latent estimation against the alignment scores and did not compare target/non-target latent decoding performance with neural alignment scores. We realized that our writing might not have been very clear in the previous version, which could cause confusion, so we revised some wording in the figure 3 captions and the main text. We think the idea of correlating the decoding performance of target/non-target spatial latents with neural alignment in the mapped layer is an interesting idea and could provide additional insights. However, we believe that conducting the proposed experiment falls outside the scope of the core claims of our current work. We leave this question to future investigations and hope our findings presented here will serve as a starting point for exploring such ideas.

• Results in section 3.3: Statistical significance of our results. We remade the figures in section 3.3 (figure 5 and others). The new version showed results from models trained with the same conditions as models investigated in section 3.2 (results from the previous version were from models trained with fewer batches). We found qualitatively similar results to those in the previous version, and the results in the new version are more significant due to longer training. We added a description in the captions of figure 5 about the error bars and indicated the statistical significance of the differences between the two groups in the figures using “*” marks. From these analyses, we are confident that the differences we described in the main text were significant.

• Results in section 3.3: Decoding performance. To help assess the scale of the improvement in decoding categories, we added a supplementary figure E.1, in which we showed the category decoding performance from spatial latent-trained models compared to category-trained models, as well as pixels and untrained models. We found that the category decoding performance of spatial latent-trained models is better than that of untrained models. So, spatial latent training increases category decoding performance in intermediate layers instead of decreasing it. Meanwhile, category decoding performance from these spatial latent-trained models is similar to category-trained models in the early to middle layers, but they start to diverge in late layers. Spatial latent-trained models become much worse at representing categories than category-trained models in late layers. This is consistent with our findings in Figure 4, which shows that the representations of spatial-latent trained models are very similar to category-trained models in early and middle layers and diverge in late layers.

• Results in section 3.3: Interpolating non-target latent variability. Thank you for proposing the intriguing idea of measuring the impact of latent variability by interpolating between full and very low variability cases. While this more detailed analysis could indeed provide additional insights, we believe our core claims still hold even without these experiments. Furthermore, the analyses in our revised version showed the statistical significance of our results, which we hope addresses some of your previous concerns. Conducting such an experiment would require generating new datasets and retraining models, which, unfortunately, was not feasible given the computational and time constraints of the revision period. We acknowledge the value of this idea and will leave it as a direction for future work.

• Results in section 3.3: Revise the language in line 430. Thanks for making the suggestion about revising the language describing our results in section 3.3. We revised the conclusion sentence based on your suggestion (now in line 432).

• Model-brain comparisons on more datasets. Our neural alignment results in the main text are primarily based on the public benchmarks on Brain-Score. We acknowledge that the Majaj-Hong benchmarks are based on images with a lot of similarity to TDW images we used to train the models since they all involve synthetic objects superimposed on random backgrounds. In the revision, we further evaluated our models on other non-public benchmarks in Brain-Score, many of which contain images that are out of the training distributions of our models, such as more complex natural images. These results were only available to us recently due to previous technical issues in obtaining model alignment scores on these non-public benchmarks in Brain-Score. We evaluated our ResNet-18 models and showed the results from 12 out of the 16 available high-level neural benchmarks in supplementary figure C.5. We excluded the results from the four tong.Coggan2024 benchmarks because they give much lower scores for ImageNet-trained models than untrained models, which is in contrast with most of the other benchmarks in Brain-Score – we are unsure whether they are implemented correctly. In the rest of the 12 benchmarks, we found that models trained on spatial latents performed comparably to category-trained models in most V1, V2, and V4 benchmarks. However, among the five IT benchmarks, spatial latent-trained models significantly underperformed category-trained models in two benchmarks (”SanghaviMurty2020.IT-pls”, ”SanghaviJozwik2020.IT-pls”) while outperforming them in one (”Bracci2019.anteriorVTC-rdm”). This results in a lower averaged IT alignment score for most of the latent-trained ResNet-18 models than category-trained ResNet-18 models if we simply average across the full set of non-public benchmarks. These three benchmarks (”SanghaviMurty2020.IT-pls”, ”SanghaviJozwik2020.IT-pls”, and ”Bracci2019.anteriorVTC-rdm”) are based on natural images that are out of the training distribution of our models. We think these results invite future research to determine the extent to which our results can be generalized to many other out-of-distribution images. We added this to the discussion of the limitations and future directions at the end of the main text in line 528.

• Why use regression-based metrics for brain-model comparison and CKA for model-model comparison? We used different metrics because we wanted to answer two different questions, which are traditionally studied in two different but related fields with their own methodological traditions. To assess how well spatial latent-trained models align with neural responses, we used Brain-Score, which incorporates many benchmarks based on regression-based metrics. These metrics are widely used in neuroscience not only for evaluating brain-model alignment but also for providing insights into how well neural activity can be predicted using model features. This is directly relevant to applications such as population neural control (Bashivan, Kar, & DiCarlo, 2019). We used CKA to compare model-model similarities because we want to answer the question: How different, if at all, are the representations learned by models trained on different spatial latents? CKA is one of the most popular approaches to compare different neural network model representations, mostly due to its thoughtful consideration of the invariant properties of similarity metric and its ability to reveal similarity between models trained with different random initialization. When designing CKA, the authors considered properties such as the sensitivity of neural network training to certain representational transformations (Kornblith, et al., 2019), which may not be relevant to real neural data. The decision to use regression-based metrics for brain-model comparisons and CKA for model-model comparisons reflects the origins of these methods in different fields that try to answer slightly different questions. Since our paper addresses both questions, we want to prioritize using the widely adopted approach so that our results can connect and remain comparable to previous studies. While it is possible to use CKA for brain-model comparisons or regression-based metrics for model-model comparisons, interpreting such results in the context of existing literature would be challenging. Moreover, brain data often differs from model data. Unlike artificial units in models, each unit response in the brain is typically an average of multiple repetitions, and the variability across these repetitions must be accounted for. That said, we appreciate the reviewer’s comment that it would be nice to have “all comparisons on the same ‘scale,’ subject to the same method-induced biases.” We acknowledge that alternative methods, such as the RSA framework, could potentially enable a more unified comparison between model-model and model-brain analyses (Schütt et al., 2023; Diedrichsen et al., 2020). While incorporating such an approach would undoubtedly add value, we believe it falls outside the scope of the current work and would not impact our core claims. We have included this point in the discussion of limitations and future directions at the end of the main text (line 535). Due to the limited time available during the revision period, we had to leave this as an avenue for future exploration.

• Determine whether different models explain the same or different portions of variance in neural data. We thank the reviewer for raising this very interesting question. We did not address this question directly because, at the time of writing the paper, we were not aware of a thorough and robust method to conduct the analysis required to answer it. Also, we think that our core claims in the paper are supported by other experiments, and this question, although interesting, seems out of scope. We are grateful for the suggestion to explore methods such as predicting regression residuals or using ensemble models. However, given the limited time available, we must leave this as an avenue for future research. Thank you for highlighting this valuable direction.

References:

1. Bashivan, P., Kar, K. and DiCarlo, J.J., 2019. Neural population control via deep image synthesis. Science, 364(6439), p.eaav9436.
2. Kornblith, S., Norouzi, M., Lee, H. and Hinton, G., 2019, May. Similarity of neural network representations revisited. In International conference on machine learning (pp. 3519-3529). PMLR.
3. Diedrichsen, J., Berlot, E., Mur, M., Schütt, H.H., Shahbazi, M. and Kriegeskorte, N., 2020. Comparing representational geometries using whitened unbiased-distance-matrix similarity. arXiv preprint arXiv:2007.02789.
4. Schütt, H.H., Kipnis, A.D., Diedrichsen, J. and Kriegeskorte, N., 2023. Statistical inference on representational geometries. Elife, 12, p.e82566.

Hi, reviewer acUt, thank you for your reviews and efforts in helping us improve the paper!

Response to your questions: We know some work, such as Conwell et al. 2022, that includes semantic segmentation models. We would appreciate it if you could point us to some other studies that you think are relevant to the neural alignment of semantic segmentation models.

The models we studied in this paper primarily focus on estimating object-centric spatial latent variables, including distance, translation, and rotation. We chose to examine models estimating these latents for three main reasons: (1) these variables constitute the core elements of the spatial information about objects, which people can intuitively estimate; (2) previous electrophysiological studies suggest that the ventral stream encodes these spatial attributes (Hong et al., 2016); and (3) models that predict these latents can be trained on the same dataset with the same architecture, allowing us to isolate the effect of the training objective while controlling for other factors.

Most segmentation models we know require an additional decoder module to output a segmentation map, making their architecture different from the latent estimation models studied here. Our study aimed to analyze the influence of training objectives on learned representations while keeping the dataset and architecture constant. We didn't study the segmentation models here because it is hard to control their architecture to be the same. We also don't know whether segmentation models have higher or lower neural alignment. The TDW dataset we proposed can also contain ground-truth labels for segmentation maps, which could be a resource for people interested in studying segmentation models in the future.

We are working on addressing your other comments, so stay tuned. Thank you!

Reference:

1. Conwell, C., Prince, J.S., Kay, K.N., Alvarez, G.A. and Konkle, T., 2022. What can 1.8 billion regressions tell us about the pressures shaping high-level visual representation in brains and machines?. BioRxiv, pp.2022-03.
2. Hong, H., Yamins, D.L., Majaj, N.J. and DiCarlo, J.J., 2016. Explicit information for category-orthogonal object properties increases along the ventral stream. Nature neuroscience, 19(4), pp.613-622.

We want to thank you for your effort in reviewing and helping us improve this paper.

• Thank you for your suggestions about the paper title. We realized that using “vision models” in our title might be confusing since it makes people think about vision transformers as well. Our current study focuses on CNNs. So, in the new pdf, we changed the title from “vision models” to “vision CNNs.”

• Biological vision and estimating 3D spatial latents from 2D input. In biological vision, humans and animals can extract significant 3D information from 2D visual cues alone. For example, humans rely on various cues in depth estimation, including binocular disparity, motion parallax, relative object size, shading, texture gradients, and occlusion. Many of these cues, such as relative size, shading, texture gradients, and occlusion, are readily available in 2D images. The biological vision systems are likely wired to exploit these cues for 3D spatial estimations from 2D images. Our results showed that it is possible to train models to estimate 3D spatial latents from 2D images and that these models achieve neural alignment with the ventral stream comparable to category-trained models. We think the implications of these results are that we should not assume the ventral stream is optimized for object recognition only, as optimizing for spatial tasks could also improve neural alignment with the ventral stream.

• The reviewer has raised a very intriguing question: is it specifically the 3D spatial properties that drive this neural alignment, or could it also be explained by models trained solely on 2D property estimation? This remains an open question, and it is unclear whether models trained to estimate purely 2D spatial properties would outperform or underperform compared to our current 3D spatial latent-trained models. Answering this question would require controlled experiments to compare models trained on 3D spatial tasks with models trained on 2D spatial tasks, such as semantic segmentation tasks you mentioned. While this is a fascinating avenue for further exploration, it falls outside the scope of our core claims in the present study. We leave this question to future research. We also hope that our datasets and current results provide a valuable resource for future investigations on this question.

We hope our response answers your questions and addresses your concerns. If you have any other questions, please let us know!

Thank you very much for your thoughtful reviews and help in improving the paper!

Response to your questions: Thanks for pointing that out. We believe a couple of reasons could underlie the phenomenon you mentioned.

1. Figure 4b shows the CKA results until layer 4.0, an intermediate layer in the CNN (the 15th layer of the 18-layer ResNet). To decode categories from intermediate layers linearly could be challenging even for category-trained models. So, it is possible that even for category-trained models, the decoding performance will not be much better than the decoding performance from spatial latent trained models shown in Figure 5b at layer 4.0. However, this remains unknown; we can do an experiment to address that.
2. It is still possible that representations can have different category decoding performances while having similar CKA since CKA does not capture every aspect of neural representations.

We are working on addressing your other comments, so stay tuned. Thank you!

We want to thank you for your effort in reviewing and helping us improve this paper.

• Category decoding. First, we want to expand on our previous response to your question about category decoding results. In the revised pdf, we included a new supplementary figure E.1, showing how well we can decode categories from spatial latent-trained models compared to category-trained models. We found that the category decoding performance of spatial latent-trained models is similar to category-trained models in early to middle layers (until layer3.0.relu); even the category-trained models have very limited ability to decode categories in these layers. The decoding performance starts to diverge in late layers (layer4.0.relu). This is also the point where we see a significant divergence in the representation similarity, accessed by CKA, in Figure 4. In the last few layers, spatial latent-trained models become much worse at representing categories than category-trained models. We think the finding in figure E.1 is consistent with our findings in figure 4.

• Thank you for pointing us to the very interesting review paper by Connor and Knierim. In our new pdf, we cited this paper in the introduction section (line 48). We also revised our introduction section, which includes discussions and references to previous works emphasizing that the ventral stream is not responsible for classification alone.

• Natural image training and additional evaluation metrics. Our current study focuses on a detailed examination of many object-centric spatial latents in 3D space and their impact on the learned representations. We wanted to isolate the effects of different training objectives, so we deliberately kept the dataset and model architecture constant. Currently, we are not aware of large-scale natural datasets that provide ground-truth labels for all the desired spatial latents in a 3D coordinate frame. Therefore, we believe our approach using a synthetic dataset remains the most practical and effective way forward for this kind of study. We agree that comparing models trained on natural image datasets, such as the SA-1B segmentation dataset, is a compelling direction for future research. However, this falls outside the scope of our current work. Due to the limited time available for revisions, we were unable to include these results but have incorporated this suggestion into the discussion section as a potential avenue for future work. Additionally, our neural alignment analyses rely primarily on regression-based metrics, which are widely used within the community. While incorporating alternative metrics such as Representational Similarity Analysis (RSA) could provide additional insights, we believe it will unlikely alter our main findings. Given the time constraints of the revision process, we have deferred this to future research and added a discussion of this limitation and potential future directions in line 523 of the new pdf.

We hope our response answers your questions and addresses your concerns. If you have any other questions, please let us know!

Thank you, reviewer v5sv, for your thoughtful reviews and for pointing us to the many interesting related works.

Response to your questions: We were able to decode categories to some extent from the latent trained models. For example, in Figure 5b, we can decode the categories well above chance in a 117-way classification task. In Figure C.1, C.2, and Table C.1, C.2, we analyzed the behavior alignment scores of these latent trained models. This score measures how well the penultimate layer of the model supports categorization in a human-like way (Rajalingham et al., 2018). In summary, we found that spatial latent trained models are not as good as category-trained models in supporting categorization behavior if we decode from the penultimate layer. This may or may not hold in earlier layers; further analysis is needed.

We are working on addressing your other comments, so stay tuned. Thank you!

References: Rajalingham, Rishi, et al. "Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks." Journal of Neuroscience 38.33 (2018): 7255-7269.

We want to thank you for your effort in reviewing and helping us improve this paper.

• Category decoding. First, we want to expand on our previous response to your question about decoding categories from spatial latent-trained models. In the revised pdf, we included a new supplementary figure E.1, in which we compared the category decoding performance from spatial latent-trained and category-trained models. We found that the category decoding performance of spatial latent-trained models is similar to category-trained models in early to middle layers, but they start to diverge in late layers. Spatial latent-trained models become much worse at representing categories than category-trained models in late layers. This is consistent with our findings in Figure 4, which shows that the representations of spatial latent-trained models are very similar to category-trained models in early and middle layers and diverge in late layers.
• References about data diversity. We also want to thank you for pointing us to these related references. We discussed how our studies could connect to data diversity and domain generalization studies in line 498 of the new pdf.
• How do our models perform out of distribution? In the main text, we mainly focused on results from public benchmarks on Brain-Score. The V1 and V2 benchmarks already use images that are out of the distribution of our model training. The V4 and IT benchmarks are based on grey-scale images and have different objects and backgrounds than our training images in TDW, which is also out of distribution to some extent. However, V4 and IT benchmarks still have similarities to our training data in TDW since they all involve synthetic objects superimposed on random backgrounds. In the revision, we further evaluated our ResNet-18 models on other non-public benchmarks in Brain-Score, many of which include natural images that are substantially out of our models’ training distribution. We showed the results in supplementary figure C.5. We found that ResNet-18 models trained on spatial latents performed comparably to category-trained models in most V1, V2, and V4 benchmarks. However, among the five IT benchmarks, spatial latent-trained models significantly underperformed category-trained models in two benchmarks (”SanghaviMurty2020.IT-pls”, ”SanghaviJozwik2020.IT-pls”) while outperforming them in one (”Bracci2019.anteriorVTC-rdm”). These three IT benchmarks are based on natural images, which differ considerably from our synthetic training data. We believe our results based on public benchmarks provide a valuable foundation and invite future investigation into how models trained on different targets generalize to out-of-distribution images when evaluating neural alignment. We also discussed the limitations of our current results, including references to out-of-distribution generalization, in line 535 of the new pdf.
• Comparison to self-supervised models and vision language models. Thank you for your suggestion. We agree that comparing our models to self-supervised and vision-language models is an intriguing and valuable direction for further exploration. However, we believe that conducting such experiments, while insightful, lies outside the scope of the core claims and focus of our current work. Our primary focus has been investigating category-trained and spatial latent-trained models, drawing from the “what” and “where” frameworks discussed in cognitive science and neuroscience. Due to time constraints during the revision process, we were unable to perform a thorough and meaningful comparison with these models. We view this as an excellent avenue for future research and hope that our findings will inspire further investigations in this direction.

We hope our response answers your questions and addresses your concerns. If you have any other questions, please let us know!