CUPID: Evaluating Personalized and Contextualized Alignment of LLMs from Interactions

Thank you very much for the thorough comments and feedback!

evaluation of the inferring and generation capabilities of LLMs is not a new research issue in itself

We agree that evaluating LLMs’ general inference and generation capabilities is not novel by itself. Instead, as also pointed out by this review, our work’s novelty lies in specifically evaluating LLMs’ capabilities at (1) inferring user contextual preferences from prior interactions, and (2) generating responses that can satisfy these contextual preferences.

purpose of the evaluation is to provide some suggestions from the perspective of LLM applications

If the paper gets accepted, with the extra page for the camera-ready, we will add a discussion subsection with suggestions for LLM applications based on our findings: (1) retrieve contextually relevant prior interaction sessions when responding to new user requests; (2) storing and retrieving summaries of prior sessions when using smaller LLMs; and (3) further prompt or model developments to better infer user preferences from multi-turn interactions.

it is not enough if only used to evaluate LLMs… not enough only using one dataset to evaluate LLMs

This is a great point. We agree that datasets are of greater utility when they can be used for both training and testing. While LILAC is intended solely for evaluation, we will release the data generation pipeline, which can be used to easily create separate training sets to fine-tune LLMs.

We agree that multiple datasets should be used to evaluate LLMs. Our dataset, which focuses on assessing a specific capability (i.e., inferring and applying users’ contextual preferences), should be used alongside other benchmarks to more comprehensively assess models’ performance.

more related work is the evaluation for LLMs

Based on this suggestion, we plan to add a Related Work subsection on LLM evaluation methods and benchmarks. While the current subsections cover evaluation related to Personalized LLMs and Interaction with LLMs, a separate section on general LLM evaluation methods can help position our work. For example, evaluation on user preferences [1], LLM-as-a-Judge [2], evaluation through user simulations [3], evaluation on synthetic datasets [4], etc.

The Desiderata should be given definition or statement clearly

Section 3 begins by defining three Desiderata (i.e., requirements) for our dataset and then refers to them through the section to note how each design choice satisfies these requirements (e.g., "Desiderata 1"). We agree that this presentation can be unclear and will explicitly explain this when first referring to a Desiderata in the section.

A period “.” is missing … / Figure 2 is not mentioned in the paper

Thank you for pointing out these issues. We will ensure to revise and double-check the camera-ready version.

does it mean that ci and pi are one-to-one corresponding in each Si

Yes. There is a one-to-one correspondence between the context factor $c_i$ and preference $p_i$ in each interaction session $S_i$. Multiple sessions can share the same context factor and relevant user preference.

effect if you fine-tuned the LLMS using your dataset

Similar to prior work [5, 6], we focus on evaluating the “out-of-the-box” capabilities of LLMs at the proposed tasks. As a result, we did not experiment with fine-tuning or prompt tuning on the dataset. While we expect that tuning on the dataset (or additional samples generated from our pipeline) would improve performance, we leave this for future work.

[1] https://arxiv.org/abs/2403.04132

[2] https://arxiv.org/abs/2306.05685

[3] https://arxiv.org/abs/2310.11667

[4] https://aclanthology.org/2023.emnlp-main.890/

[5] https://arxiv.org/abs/2410.10813

[6] https://openreview.net/pdf?id=YfHxQSoaWU

Thank you for the positive review and the constructive feedback!

further discuss the practical relevance of the benchmark

Our benchmark reflects long-term and frequent use of LLM-based chat assistants, like ChatGPT. Through frequent interactions, the user may provide the assistant with diverse requests that may share contexts. For example, a user can ask the following requests to an assistant: write an email to PhD advisor requesting time off, write another one asking for paper feedback, revise an email responding to the feedback, etc. If the assistant recognizes that these requests share the same context (PhD advisor), it can infer the user’s preference for this context from prior interactions and personalize future responses.

available to the community?

Yes. We are excited to publicly release our dataset. Additionally, we will release the code for the data generation pipeline and the evaluation suite (including the fine-tuned metric).

Thank you for the positive and insightful comments!

bias towards Claude models

This is an important point. To assess the presence and significance of potential self-enhancement bias, we conducted an analysis (Appendix F.5) where we used our pipeline to synthesize small datasets with three other models: Claude 3.7 Sonnet, GPT-4o, and DeepSeek-R1. Each of these models was evaluated on each other’s datasets to test whether a model performed better on data that it generated. Results showed no clear evidence of self-enhancement: Claude 3.7 Sonnet outperformed on the Inference task in all of the other models’ datasets, and DeepSeek-R1 outperformed in the Generation task in all datasets. This analysis of self-enhancement bias was based on similar analyses from prior work [1, 2].

overlook that real users often express preferences more implicitly and subtly

We agree with this point. Our benchmark was designed with more explicit signals from simulated users to assess whether LLMs struggle even in these “easier” cases. Our results show that they do struggle and reinforce the need for further model developments to adequately provide personalized and contextual support in real-world applications.

the dataset … size is very small … / unclear if … can be used directly for model alignment / possible to automate the verification process

Dataset size was limited by the cost of human verification. We opted for a relatively smaller but more rigorously and carefully validated dataset. This thorough verification revealed that ~9% of generated samples required manual correction. Given this relatively low error rate, we plan to open-source the data generation pipeline to support large-scale data generation for model training and alignment. Also, as most errors were related to the current request revealing the user’s preference, the pipeline can be extended with a simple LLM-based validation module that can identify and revise such cases—as suggested by Reviewer puYD. We believe that this can help further reduce errors and increase the overall data quality for alignment.

possible pathways and potential challenges in building .. from resources that have real user interactions

This is a very interesting point. If the paper gets accepted, we will use the extra page for the camera-ready to add a short discussion on how to extend the data generation pipeline to be grounded on real user data (e.g., WildChat/WildFeedback). Specifically, user requests in these datasets can be used to seed the generation of other requests with the same or different contextual factors—creating interaction session histories based on realistic requests but controlling for the instance type (e.g., Consistent, Contrastive). Similarly, the simulated user in our pipeline can be guided with real user messages to produce more realistically styled feedback.

Figure 2 may need to be recomplied

Thank you for pointing this out! We will make sure to revise this.

[1] https://openreview.net/forum?id=3GTtZFiajM

[2] https://arxiv.org/abs/2412.10424