Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms

W1: Cumbersome description of the method

R: Thanks for your suggestion. We will find a way to further simplify the description of the method. The purpose of Fig. 2 is to illustrate the consistency of our approach and aesthetic concepts, and we will add more specific details to Fig. 3 that illustrates the specific steps. While at the same time, we will write a clearer index in the labels of Fig. 2 and Fig. 3 to avoid confusion for the reader.

W2: Conflate "no-reference image quality assessment" with "image aesthetic quality assessment"

R: Thank you! We will modify the description to distinguish between MANIQA and the aesthetic quality assessment model. We use this model because we have experimented with more image quality-related models that include these three in the paper and found MANIQA performs well.

W3: Number of samples in Table 7

R: Thanks! There may be some misunderstanding on the concept of stride, which refers to the interval between samples in our re-ordering sequence. The number of samples (as described in line 158-169, Sec. 2.3) is determined in terms of $u$ and $v$. The number of sampled images is $uv$ and the equation $|\mathcal{D}_{po}| = uC_v^2+vC_u^2$ shows the number of comparison pairs.

For example, given a sequence of 100 images with $u$=$v$=5 and stride=2, we will sample $uv=25$ images with 2 as interval: [1,3,5,...,49]. If stride=3, it will be [1,4,7...,73]. The number of selected images is irrelevant to the stride.

$|D_{po}|$ is the number of valid comparison pairs. When $u$=$v$=5 and stride=2 and we sampled [1,3,5...,49], we first split it with semantic dimension to [[1,3,5,7,9], [11,13,...],...,[...,47,49]]. There are 5 sequences of 5 images. For each sequence, such as [1,3,5,7,9], it contains $C_5^2$ valid comparison pairs: [(1,3), (1,5), ..., (7,9)]. Thus it will contain $5C_5^2$ pairs for all 5 sequences. Similarly, when we split with aesthetic dimension and result in [[1,11,21,31,41], [3,13,…]…, […,39,49]], it will also contain $5C_5^2$ pairs. Thus in this case $|\mathcal{D}_{po}|=5C_5^2+5C_5^2=100$.

W4: Typos

R: Thanks! We will fix the typos in the final version.

Q1: Explanation on why IAA dataset cannot be used for aesthetic retrieval evaluation

R: A data item in IAA can be formulated as (Img, Text, Score), for example, (<cat img>, cat, 0.6); and (<dog img>, dog, 0.73). To be fair, retrieval must be compared with the same query, therefore our required data should be in the following format: (Text, [{img_1, score_1}, {img_2, score_2}...]); for example, (dog,[{<dog img1>, 0.76}, {<dog img2>, 0.54}…]). An alternative way is to retrieve images based on the IAA's queries, but we still need human resources to label the comparison. In addition, the queries' distribution does not match to the distribution of retrieval system's users. Thus, we decided to directly label the testing set.

Q2: Explanation on "topic" in data preprocessing

R: We expect that the distribution of the test set resembles the distribution of queries that are commonly searched by target users. User queries often involve user privacy that most companies will protect and cannot be directly obtained. A common alternative is to count the topic distribution, which are tags like "financial scene" or "indoor decoration" for queries. We generate queries based on the distribution of these topics to ensure that our query distribution is consistent with that of target users.

W: The design of the proposed metric

R: Thanks. We proposed two metrics in the paper: HPIR weighted accuracy and win rate.

1. The construction of HPIR requires retrieving and filtering images according to the query, and then manually picking representative images to label. In order to exclude the influence of models during retrieving and filtering process, as depicted in Sec. 3.1, we leverage multiple retrieval systems (including Google, Getty, Bing etc.) and retrievers built upon various base models (CLIP, OpenCLIP, DataComp etc.) with different sizes and training data, to make it as comprehensive as possible. In addition, we merged results from these sources together and manually picked representative pairs. The performer does not know which model the data is coming from, and this process decoupled the data from the model's influence.
2. We use win rate to quantify the comparison between different systems, which is a fair metric for different models. Additionally, the retrieval database we use is confidential, and we use human labeler to ensure that the results of GPT-4V are not seriously biased, and we use such indicators in the end-to-end evaluation at the system level, making it robust and reliable.

Q1: The correlation to the sophistication of language

R: Yes, higher levels of language sophistication is also correlated with higher visual aesthetics. But, "higher level of language sophistication" is also coupled or correlated with "aesthetic expectations and details" in our paper. Like we found in Sec. 4.2, even using longer query brings increase to the results. Higher level of language sophistication is also correlated with higher visual aesthetics due to the inductive bias within pre-training data.

However, a nice system should be user-friendly, without the need to write complex languages, and reduce the budget that comes with calling GPT. With our RL-finetuned model, users can achieve similar results with simple language as with complex language.

Q2: The scaling ability of RL-based approach

R: Similar to RLHF in LLM, the more essential factor for whether an RL algorithm can scale mainly comes from the feedback. As an entry point to research, our feedback comes from aesthetic models, and it has a limited scaling upper-bound. In real application, we can obtain human feedback by, for example, user click-through rate. Feedback like this will exhibit excellent scaling properties and contribute to the continuous improvement of model capabilities, but due to privacy and policy reasons, it is out of the scope of our work.

W1 & Q1: Human variance

R: Thank you for the suggestion. We provide a metric 'confidence' for representing the robustness of the label. The confidence score means the degree of agreement among all annotators, rather than a value provided by labeler. It is calculated through Equation 10. This confidence score and variance have similar indicative meanings. For example, if a piece of data has the confidence score of 0.6, it means that 24 labelers hold the same choice and the other 6 labelers choose the opposite. Thus, $Confidence=\frac{2\times24}{24+6}-1=0.6$ We used this indicator and named it as confidence score because when both choices are supported by half of the labelers, $Confidence=0$, and when all labelers agree with a choice, $Confidence=1$. The distributions of all confidence scores are shown in Fig. 15 of Appendix E.

As you mentioned, "aesthetics is a subjective concept," we used the confidence as a weight (see Equation 11) to ensure that our accuracy was only calculated on less controversial comparisons. This ensures the robustness of the experiment and excludes the influence of subjectivity.

Per your advice, we further calculate the variance for these labels. Let the positive choice be 1 and the negative choice be 0, and then the labeling results can be formulated as a sequence like [1,0,0,1,...]. It is easy to derive the following formula: $\text{variance} = \frac{ 2N_{pos} N_{neg}}{( N_{pos}+ N_{neg})^2}.$ Here are the details of human variance of our labeled data, accompanied with confidence score:

Last but not least, perhaps our statement of confidence is bothering you because we didn't explain its similarity to variance. We'll modify the description to make it more explicit.

W2 & Q2: Lack of human evaluation

R: Thank you for the suggestion. In Table 2 on page 8 of the main paper, we have presented a user study (last two rows), where we let multiple human labelers judge the images retrieved from models w. and w/o. alignment (using the same queries). These labelers are expert search engine users. Users judged which retrieved results from system A and B were better. The first six rows of Table 2 are judged by GPT-4V.

Our description of this part does not link it to the phrase "user study", causing confusion to the readers, and we will revise the description of this section.

We supplement more user studies over the experiments in Table 2 here:

W1 & Q: Lack of user study

R: In Table 2 on page 8 of the main paper, we have presented a user study (last two rows), where we let multiple human labelers judge the images retrieved from models w. and w/o. alignment (using the same queries). These labelers are expert search engine users. Users judged which retrieved results from system A and B were better. The first six rows of Table 2 are judged by GPT-4V.

Our description of this part does not link it to the phrase "user study", causing confusion to the readers, and we will revise the description of this section.

We supplement more user studies over the experiments in Table 2 here:

Q: Computational Cost

R: The RL-finetuning process only used 4 NVIDIA A100 GPUs for fewer than 5 hours. If using smaller batch size or techniques like gradient accumulation, it can be trained within an acceptable time even using other consumer-grade GPUs with more than 4GB of memory. This is a small burden compared to most of today's work, including our pre-training phase.

W2: Related work

R: Thank you for your comments, and we will make adjustments to the related work so that readers can better understand the article.