When does perceptual alignment benefit vision representations?

We thank the reviewer for their helpful comments. We are glad that the reviewer found our evaluation to be comprehensive, the paper idea innovative, and the analysis insightful. We address questions and concerns below.

Writing comments

We thank the reviewer for their helpful notes and will fix the citation typos in revision. We intend for the “Discussion” section at the end of the paper to serve as the conclusion, and can revise it to clarify this – and expand upon our conclusions – in revision.

Further experiments on VLMs

We thank the reviewer for the suggestion. In preliminary experiments, we found that LLaVA, MiniGPT-4, and instructBLIP were not well suited to in-context prompting (potentially due to their instruction tuning and optimization for other tasks). However, we replicated our RAG experiment on IDEFICS2, a recently released 8B multimodal model achieving state-of-the-art results across several multimodal benchmarks [2]. Our results are shown in Fig. 1 of the PDF attached to our global response. We find that across the same four datasets evaluated in Section 4.2 of the paper, performance consistently improves when using NIGHTS-tuned models in the RAG pipeline. The exceptions are DINOv2 with the Diabetic Retinopathy dataset and DINO/DINOv2 for the SVHN dataset. Overall, these results support and validate our results on OpenFlamingo; we will add them to our revision.

CLIP-Blind image pairs

We thank the reviewer for suggesting the MMVP-VLM benchmark [2]. Unfortunately, evaluating on this benchmark requires computing the similarity of text-image pairs. We only fine-tuned the CLIP vision encoder; thus it no longer shares the same feature space as the text encoder, making it difficult to fairly run a multimodal evaluation and interpret any positive or negative results.

Clarification on patch-level objective

We provide further details and intuition regarding patch-level training in section 2 of the global response. Below we address specific questions:

How does the patch-level objective differ from the image-level objective? Does it show different performance on various downstream tasks?

Finetuned patch tokens were necessary to evaluate NIGHTS fine-tuning on dense tasks (depth estimation, segmentation). In preliminary experiments, we found that the image-level objective did not induce any significant changes in the patch tokens. The patch-level objective is functionally the same as the image-level objective, the only difference being that it propagates the similarity annotations more directly to the patch tokens.

Note that only the fine-tuned patch tokens are needed for segmentation/depth-estimation, and only the CLS token for image-level tasks (all others).

[1] Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? arXiv preprint arXiv:2405.02246, 2024b.

[2] S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie, “Eyes wide shut? exploring the visual shortcomings of multimodal llms,” CoRR, vol. abs/2401.06209, 2024.

We thank the reviewer for their helpful comments. We address questions and concerns below.

How does perceptual alignment affect model features?

We emphasize that this paper aims to answer this question in terms of the competency of representations. There is a rich precedence of understanding and comparing representations via their competencies at downstream tasks [1-2]. Evaluating general-purpose features on transfer tasks – particularly with simple methods such as KNNs and linear probes – allows us to quantify what the features capture and make decodable.

We further flesh out our empirical evaluation in the global response, section 1.1, where we additionally compare different levels and types of perceptual alignment across segmentation, depth estimation, and classification. We hope that our additional evaluations provide insight into which datasets and tasks are represented in a useful way by finetuned models. We also provide further discussion of the learned feature space in section 3.

We finally note that while we probe representations in terms of competency, understanding them in terms of their mechanism is an important direction for future work.

Why is mid-level alignment better than other levels?

We address this question directly in our global response, section 3. Please let us know if further detail is needed, or there are follow-ups.

Perceptual alignment for CNNs

We thank the reviewer for their suggestion. We assess the effects of perceptual alignment with NIGHTS on two popular CNNs: ResNet50 and ConvNeXt-B, using the same loss described in the paper*. We evaluate on counting and instance retrieval tasks; our results are shown below:

Counting (Clevr-Count):

Instance Retrieval (DF2) **:

*Due to time constraints we were unable to implement LoRA tuning for the CNNs, and instead trained MLPs on top of the final-layer improvements. Previous work [3] indicated training on NIGHTS using MLPs can steer alignment in the right direction but may be less effective; nonetheless we still see downstream improvements using this method.

**We report results for both ConvNeXt and ResNet, however acknowledge that the accuracy numbers for ResNet are likely too small to draw conclusions from.

Comparisons against other perceptual datasets.

We thank the reviewer for their suggestion. In addition to our comparison on counting and instance retrieval, we further evaluate models trained on BAPPS, THINGS, and ImageNet triplets on a wider range of downstream tasks: segmentation, depth estimation, and a and a selection of natural/specialized/structured datasets from VTAB. Our full results can be found in the global response, section 1.1, and we will include these in our revision.

[1] Yonglong Tian, Lijie Fan, Kaifeng Chen, Dina Katabi, Dilip Krishnan, and Phillip Isola. Learning vision from models rivals learning vision from data. ArXiv, abs/2312.17742, 2023.

[2] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICLR, 2021

[3] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. In NeurIPS, 2023

Thanks for your feedback. The hypotheses outlined in Section 3 are interesting and have partly addressed my concerns regarding "why mid-level alignment can lead to better general-purpose representations". I will raise my score accordingly.

We thank the reviewer for their helpful comments. We are glad that the reviewer finds the method convincing, the paper well-written, and the experiments comprehensive. We address specific concerns and questions below.

Global v. local representations

We provide details regarding training patch tokens in the global response, Section 4 of the paper, and further clarifications below.

Global representations refer to the output embedding for CLIP/OpenCLIP, and the CLS token for all other backbones. These alone are used for counting, instance retrieval, RAG, and classification, as they contain information across the image. In these global tasks, we either compute k-Nearest Neighbors over these global representations, or train linear probes. Local features are used for dense tasks (segmentation, depth estimation); good spatial features are necessary, as the output depends on per-pixel labels rather than image-level labels. Following the procedures from [1], we train segmentation and depth estimation heads on top of only the local features.

We thank the reviewer for raising these questions, and will clarify our methodology in our revision.

VTAB results

We appreciate that the reviewer brought up this concern yet recognized the significant coverage of experiments in the main paper. Our main contribution is to empirically identify how tuning on perceptual datasets impacts transfer performance; for a thorough investigation, this includes negative impact. We acknowledge, however, that readers may be most directly interested in where this alignment succeeds, and structure the paper to highlight these applications while retaining all our findings.

Comparison to state-of-the-art task-specific models

As the reviewer notes, we use the same backbones across all experiments and tasks. This was done to evaluate general-purpose representations, and we acknowledge that these may be outperformed in cases such as counting by task-specific models. We do not aim to achieve state-of-the-art performance on any single task; rather, we demonstrate how a representation becomes comparatively better (or worse) over an array of multiple tasks. By evaluating the competency of a single representation over multiple tasks – which would not be possible with task-specific models – we gain insight into how general-purpose feature spaces are affected by alignment.

Counting experiment parameters

Below we report the training performance for each value of $k$ in the counting task.

DINO:

DINO-HA:

Code release

Our code will be fully open-sourced along with the release of our paper.

We also appreciate the reviewer’s note of a missing reference and will address this in revision.

[1] Oquab, Maxime, et al. "Dinov2: Learning robust visual features without supervision." arXiv preprint arXiv:2304.07193 (2023).

We thank the reviewer for their helpful comments. We address questions and concerns below.

Which levels of alignment benefit or impair different tasks?

To strengthen our empirical results, we extend our comparison of different “levels” of alignment (i.e. fine-tuning on THINGS, BAPPS, ImageNet) to the majority of tasks evaluated in the paper: counting, retrieval, segmentation, depth estimation, and a selection of natural/specialized/structured datasets from VTAB. We refer the reviewer to section 1.1 of the global response for our new results and observations. We also examine how the strength of alignment (i.e. number of training steps) impacts performance, in section 1.2 of the global response.

Taken together with our paper, we glean the following insights:

• Finetuning on NIGHTS generally benefits representations over base models on object counting, segmentation, depth estimation, instance retrieval, RAG, and some structured classification datasets (such as smallnORB).
• Fine-tuning on perceptual datasets that are solely low-level (BAPPS) or high-level (THINGS) typically performs worse than fine-tuning on NIGHTS, and in many cases worse than the base models themselves. This demonstrates that representation quality depends on the type of perceptual data, and in fact some forms of perceptual alignment can harm representations.
• Perceptual alignment impairs performance on most natural classification datasets, particularly fine-grained. This phenomenon is consistent across perceptual datasets; we discuss potential reasons in section 6 of the paper. Low-level alignment (BAPPS) seems to best preserve base model performance in this case.
• A small amount of alignment may yield the most benefits; training for >1000 timesteps appears to cause a decline in retrieval performance.

Why is mid-level alignment better than other levels?

We address this question directly in our global response, section 3. Please let us know if further detail or follow-up is needed.

Performance drop on dense tasks.

The reviewer is correct that NIGHTS is the only difference for models with/without HA. We flag downstream datasets that the pretrained model has seen because if a dataset is already in-distribution for a backbone, fine-tuning on different data may be more likely to change the feature space such that that dataset is more out-of-distribution. We do not see this as the sole explanation for the results, simply a possible factor. We will clarify this in our revision.

Why does THINGS hurt retrieval?

The triplet choices in THINGS reflect coarse-grained/high-level semantic similarity, i.e., the concepts that humans use for judging object similarity [1, 2]. Thus, THINGS is ill-suited to retrieval tasks (of which counting is one) because they rely on the local/fine-grained similarity structure of representations, rather than the global/coarse-grained similarity structure. Muttenthaler et al. [3] found that learning a linear transform to match the THINGS triplet odd-one-out choices on top of ImageNet-trained models deteriorates downstream performance for tasks where local similarity structure is important, such as few-shot learning on single domain datasets. The authors observed that changing a model’s representation space without preserving the local similarity structure of the pretrained representations decreased few-shot learning performance, and harmed their nearest neighbor structures.

Fu et al. [4] further found that fine-tuning vision models on NIGHTS, which better reflects human local similarity structure, leads to decreased performance on THINGS. This indicates that the concepts captured in NIGHTS – which are helpful for retrieval – are different from those useful for THINGS.

Questions on patch-level training

We provide further details and intuition regarding patch-level training in section 2 of the global response. Below we address specific questions:

Isn’t there redundancy between the average pooling of patches and the CLS token?

While there is some redundancy among patch and CLS features, they do not encode identical information as the former is a uniform pooling over the image space, and the latter does not have such a constraint (previously, Zhai et al. [5] train the SigLIP model and use attention-pooled patch tokens as their global embedding instead of using a separate CLS token). In our work, we take the simplest approach to include all feature information available to tune on, and we find it effective for dense prediction tasks.

Since some patches from negative samples and reference images are similar, it seems counterintuitive to push them away in the embedding space.

We agree with the reviewer; this was our intuition for supervising on the average-pooled patches, rather than using a dense contrastive loss. Average pooling allows us to supervise on a global representation, but also propagates that supervision to the patch tokens, enabling evaluation on dense tasks. In preliminary experiments, we found that patch-level alignment without pooling led to poor results, likely for this reason.

[1] Hebart, M.N., Zheng, C.Y., Pereira, F. et al. Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nat Hum Behav 4, 1173–1185 (2020).

[2] Muttenthaler, L., Dippel, J., Linhardt, L., Vandermeulen, R. A., & Kornblith, S. (2022). Human alignment of neural network representations. In ICLR 2023.

[3] Muttenthaler, L., Linhardt, L., Dippel, J., Vandermeulen, R. A., Hermann, K., Lampinen, A., & Kornblith, S. Improving neural network representations using human similarity judgments. In NeurIPS, 2023.

[4] Fu, S., Tamir, N., Sundaram, S., Chai, L., Zhang, R., Dekel, Tali., and Isola, P. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. In NeurIPS, 2023.

[5] Zhai, Xiaohua, et al. "Sigmoid loss for language image pre-training." In ICCV, 2023.

We thank the reviewer for their helpful comments and appreciate that they found the evaluation comprehensive, the paper clear, and the topic interesting. We address questions/concerns below.

“It is expected that human preference make generally better representations.”

We broadly agree with the reviewer that human preference is a form of fine-grained “classes”, with flexible class definitions, depending on the level/type of similarity judgments. Our key contribution is to elucidate how these classes should be defined to benefit representations: we find that the particular preferences in NIGHTS improve performance across many tasks.

A key finding, in fact, is that not all human preferences are always better. In the global response section 1.1, and section 4.5 of the paper, we compare finetuning on multiple types of human preferences (both high- and low-level). Finetuning on NIGHTS outperforms other perceptual datasets (BAPPS, THINGS) across segmentation, depth estimation, retrieval, and counting. Finetuning on triplets formulated without any human preferences (from ImageNet) often outperforms models fine-tuned on BAPPS/THINGS, showing that some human preferences harm representations.

Synthetic-Real domain gap

We briefly discuss the synthetic-real domain gap in section 4.5. We ablate the use of synthetic images in NIGHTS by including SynCLR in our experiments, which was pre-trained solely on generated images. SynCLR exhibits the same performance drops across fine-grained/natural datasets as real-trained backbones, indicating that they must result from the perceptual alignment. We will clarify this in our revision.

When can human preference be harmful?

As detailed further in the next section, “human preference” lacks a rigorous definition in vision and language literatures; it can include aesthetic preferences and other harmful types of annotations. The scope of our work is to evaluate human perceptual alignment, which extracts human preference at a psychophysical level rather than a more cognitively-penetrable level. Nevertheless, we empirically observe several cases in which human preferences may harm representations:

• In the global response, section 1.2, we show that overfitting to perceptual datasets harms performance.
• In our dataset ablations (global response section 1.1, paper section 4.5) we find that finetuning on perceptual datasets with solely high-level (THINGS) or low-level (BAPPS) variations hurts performance for many tasks.
• Finetuning on NIGHTS seems to degrade how well representations to distinguish between distinct fine-grained categories that are very visually similar (paper section 6.1). As mentioned above, due our evaluation of the synthetically-pretrained SynCLR, we attribute the performance drop to perceptual alignment.

There are other possibilities for harm outside the scope of the type of perceptual annotations we study:

• Human preferences may reflect unwanted biases. A long-standing problem in both language and vision is that biases (e.g. gender, racial) reflected in Internet language/images are inherited by large models, and reflected in their embeddings. One can imagine similar phenomena in visual preferences [3].
• Humans may disagree on a preference label, or even disagree with themselves if asked at different times. This may lead to noisy data if not filtered carefully, thus harming representations [1].
• Without sufficient demographic diversity in the annotator group, emerging biases may be reflected in the model. For example, some RLHF-trained language models have been shown to develop a bias towards the opinions of high-income, liberal individuals over their non-RLHF counterparts. [2]

How should human preference be better defined and categorized?

We thank the reviewer for pointing this out; it is important to define human preferences in vision. In the language space, human preferences largely refers to ethical judgments, or aspects of the user interaction experience. In vision, we define preferences as concepts (or categories) that humans use for making image similarity judgments [1]. By training on these preferences, we make models learn a concept space that is aligned with the concepts that humans use to navigate the visual world. We will clarify this in our paper. In addition, we will refer the reader to a recent review paper, “Getting aligned on representational alignment” [1] that clarifies definitions related to aligning vision representations with human judgments.

We also note that aesthetic judgments (i.e. concepts that humans use to determine visual appeal) have been used to improve diffusion models [3]. We consider this a separate category of annotation, however can refer the reader to relevant works.

Distracting factors in human preference datasets

The signal-to-noise ratio of human cognitive data (may it be behavioral or neural) is often low. Thus, it is important to perform a denoising step before using the data downstream. For behavioral data this can be achieved by collecting a large number of judgments and filtering out the low quality judgments or a strong regularization technique. Distracting factors may relate to long response times or other confounding factors. Isolating such effects could be achieved by showing human participants triplets of different kinds, with varying background or object complexity where complexity can be the visual richness of the scene(s) or the number of objects.

[1] I. Sucholutsky, L. Muttenthaler, A. Weller, A. Peng, A. Bobu, B. Kim, B. C. Love, E. Grant, J. Achterberg, J. B. Tenenbaum, et al. Getting aligned on representational alignment. arXiv, 2023.

[2] Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P. and Hashimoto, T.. Whose opinions do language models reflect?. In ICML, 2023.

[3] Y. Kirstain, A. Polyak, U. Singer, S. Matiana, J. Penna, and O. Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. 2023.