Preference Discerning in Generative Sequential Recommendation

Practical value of our benchmark:

The practical value of our work is that (i), sequential recommendation systems can be conditioned on user preferences in-context, i.e. a user can provide their preferences in textual form which leads to more informed recommendations, (ii) our benchmark allows evaluation of different aspects, like measuring whether a recommendation system is capable of comprehending sentiment, which is lacking in the literature. Further we now give concrete real world examples that relate to our different evaluation axes:

• Sentiment Following: This axis is crucial for leveraging organic data. For instance, on social media, we not only have access to users' interactions with ads but also their organic data, such as posts, comments, and likes. A user may express dislike for a specific brand of phone in their posts or comments, but then accidentally click on an ad for the same brand. Sentiment following enables the system to address these cases and transfer preferences from social interactions to ad recommendations.
• Fine-Grained & Coarse-Grained Steering: This feature can also be useful in utilizing organic data for ad targeting. As an example, if a user advocates for exercise and fitness and engages in discussions expressing this sentiment, the model can steer recommendations to avoid weight-loss medications, even if the user has purchased them in the past.
• History Consolidation: User preferences often change over time, and users typically have different preferences for different items. For example, a user may prefer running shoes built on a certain type of foam but also values lightness. However, after some time, the foam may become less important to the user. In this case, the recommendation system should adapt its recommendations based on recent purchases and discard outdated user preferences. This is precisely what we aim to evaluate with history consolidation.

Overall, we are convinced that in-context conditioning on user preferences drastically enhances the flexibility of the recommendation system and enables us to evaluate for these different scenarios.

Confusion around preference steering:

The rationale behind constructing this evaluation is that the originally assigned preference to the ground-truth item may not accurately reflect it semantically, since the preference generation is dependent on the timestep. This means that for each ground truth item only the preceding items have been used for preference generation to prevent information leakage. Therefore there is a chance that the preference does not accurately reflect the semantics of the ground truth item, which preserves the underlying aleatoric uncertainty of the recommendation task, as sometimes the interaction history is not informative for predicting the next item. In fact, our conducted user study (Appendix F) confirms this, as there is approximately a 30-50% chance that a generated preference that correctly approximates the user’s preferences does not relate to the target item.

To ensure that preferences and items are semantically related, we conduct what we call the “association” of preferences and items. To this end we collect all generated preferences across all users and match them to items via cosine similarity in SentenceT5 space, i.e. $p_1$ is the preference that yields the highest cosine similarity to $\tilde{i}_t$ and $p2$ has the highest cosine similarity to $\hat{i}_t$. Therefore we ensure that $p1$ is semantically related to $\tilde{i}_t$ and $p1$ is semantically related to $\hat{i}t$. The final sequences are then constructed out of the original sequence, where we replace the original ($p$, $i_t$) pair with either ($p_1$, $\tilde{i}t$) or ($p_2$, $\hat{i}t$).

Finally, the rationale for the fine/coarse-grained steering is as follows: We ask whether the model is capable of predicting a very similar item to the ground truth item or a very distinct one, only by altering the preference. Intuitively, both scenarios evaluate whether the model can accurately follow a user preference, as it semantically reflects the item. However, there is one important difference, namely that in fine-grained steering the interaction history may provide useful information that helps predicting the next item, as the item used to replace the ground truth item is very similar to it. However, in coarse-grained steering this is not the case. In coarse-grained steering the interaction history is not helpful and the model must rely solely on the user preference. Therefore the two evaluation scenarios are complementary.

Selection of baselines:

Our VocabExt_LM baseline IS actually the LC-Rec method [3] without the auxiliary tasks, however we accidentally mixed up the references. The reason we named it differently was that we did not use the auxiliary tasks, however we corrected this in our revised version. Our results for LC-ReC indicate that it requires its auxiliary tasks to align the semantic-ID and language spaces effectively, while Mender does not require auxiliary tasks, as verified by training on the preference-based recommendation data.

Furthermore, we added an additional baseline that we now call $\text{VocabExt}_{\text{LM}}$, which takes past items in natural language along with user preferences and initializes both encoder and decoder with the pretrained LLM. For this method the modality gap between language and semantic ids is the same as for Mender. Our Mender variants significantly mostly outperforms this baseline as well (see Table 1 and updated Figure 3 and 4). For convenience we provide a reduced version of that table below.

Finally, we were not able to find the LETTER method, can the reviewer provide a reference to it?

Efficiency concerns:

Line 282 talks about “training efficacy”, i.e. training speed, not about performance. Since MenderEmb relies on pre-computed embeddings, it trains significantly faster than MenderTok and is also faster during inference. We added Table 7 in Appendix E, which compares the performance vs efficiency trade-off of MenderEmb to MenderTok. For convenience we also show a reduced version of this table below. MenderEmb trains an order of magnitude faster, while reaching lower performance.

Design of sentiment understanding:

We believe that there may be a misunderstanding. With “sentiment” we refer to the ability to distinguish between a positively formulated preference and a negatively formulated one. Due to our adapted evaluation for sentiment following (we added a paragraph in the methodology section), sentiment following accurately evaluates for accurately following sentiment, as the model is only rewarded if it does not predict the item for a negative preference, but does predict the item for a positive preference.

Value of history consolidation:

This is a misunderstanding, the performance on the history consolidation axis is expected to be lower than for the recommendation axis. We realize that this may be counterintuitive, but there is a simple explanation. For preference-based recommendation we matched each ground truth item to one of the five user preferences. This matched preference is contained in the set of the five available ones and it is the semantically most related one. The remaining preferences are usually orthogonal, i.e. they describe aspects of different items the user purchased in the past. Therefore they are not necessarily related to the ground truth item anymore and thus can be considered noise. This effect is reflected in the evaluation score.

Further, during the matching process, however, one of the five preferences is associated with the ground truth item. This preference is contained in the set of five preferences provided to the model. Therefore the history consolidation axis IS evidence that the model exhibits the capability of inferring the correct preference that was originally matched with the ground truth item.

We agree with the reviewer that it is interesting to investigate how results shift when training on all five user preferences for the Amazon datasets. We added these results in Table 8 in the appendix. The outcome verifies that training on all user preferences leads to detrimental performance across all axes, confirming our intuition. For convenience we also add a reduced table below that depicts Recall@10/NDCG@10 for all Amazon datasets.

Additional discussion:

We added more in-depth discussion on the observed results for history consolidation and sentiment following. The validity of the preference design is verified by the improved performance on preference-based recommendation. If the generated preferences were faulty, they would lead to detrimental results on the recommendation axis. To provide additional evidence that verifies this finding, we conducted a user study to assess the quality of generated preferences. The results for this user study (Appendix F) demonstrate that the preferences indeed reflect the user’s preferences.

Furthermore, we investigated the data distribution of the Amazon and Steam datasets and found that the item distribution differs substantially for Steam compared to Amazon. In the Steam dataset there are few items that are heavily overrepresented. This leads to the generally higher scores on Steam and we believe it is also the reason why there is no emerging steering when training on this dataset, as the model tends to overfit on overrepresented items. Prior work has also confirmed that data distribution is a driving factor to elicit emerging capabilities [1].

Typos and inconsistencies:

Thank you for pointing them out, we corrected them in the revised version.

References:

[1] Data Distributional Properties Drive Emergent In-Context Learning in Transformers, Chan et al., NeurIPS 2022

We would like to thank the reviewer for the constructive feedback and address the raised weaknesses as follows.

Time factor:

Thank you for making us aware of this clarification issue, we agree that some of our phrasing was misleading. We actually did consider the time factor for generating the user preferences. (answering Q2) In fact, we generated user preferences depending on the timestep, therefore they do change after a user purchased another item (answering Q3). We rephrased Section 4 and merged it with Section 3. To alleviate ambiguities, we also added pseudocode for the preference generation pipeline.

• Why 20 most recent items?

We agree with the reviewer that this setup is restrictive and that it is important to go beyond a certain interaction history length. The main focus of our work is on preference discerning though, therefore we followed the setup from [1] to obtain a fair comparison. In fact, most other works on recommendation systems use a similar setup, e.g. [4,5]. We now added this point in our limitations section. Future work should investigate the effect of going beyond this setup.

Selection of baselines:

Our VocabExt_LM baseline IS actually the LC-Rec method [3] without the auxiliary tasks, however we accidentally mixed up the references. The reason we named it differently was that we did not use the auxiliary tasks, however we corrected this in our revised version. Our results for LC-ReC indicate that it requires its auxiliary tasks to align the semantic-ID and language spaces effectively, while Mender does not require auxiliary tasks, as verified by training on the preference-based recommendation data.

Furthermore, we added an additional baseline that we now call $\text{VocabExt}_{\text{LM}}$, which takes past items in natural language along with user preferences and initializes both encoder and decoder with the pretrained LLM. For this method the modality gap between language and semantic ids is the same as for Mender. Our Mender variants significantly mostly outperforms this baseline as well (see Table 1 and updated Figure 3 and 4). For convenience we provide a reduced version of that table below.

Interpretation of MenderTok-Pos-Neg results (Q1):

For our experiments including negative data we include a weighting factor that downweighs the maximization objective for negative samples. This is necessary as otherwise it results in training instabilities. We believe that more sophisticated data mixing strategies (as explored in [2] for example) may enable training of a model that improves across all axes. Our results on MenderTok-All for the Beauty dataset hints in that direction, as it performs well on different axes simultaneously. We consider this a promising avenue for future work.

NDCG for sentiment following (Q7):

As mentioned in line 235 we use a combined recall measure to evaluate this scenario, since conventional Recall or NDCG do not capture whether the model correctly follows the sentiment of the preferences. We agree that this should be made more explicit, therefore we added a paragraph where we properly introduce this metric and also note in Table 2 that we report this metric instead of Recall.

References:

[1] Recommender Systems with Generative Retrieval, Rajput et al., NeurIPS 2023

[2] Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, Raffel et al., JMLR 2020

[3] Adapting large language models by integrating collaborative semantics for recommendation, Zheng et al., ICDE 2024

[4] Self-Attentive Sequential Recommendation, Kang et al., ICDM 2018

[5] Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects, Ni et al., ACL 2019

Confusion on fine/coarse-grained steering:

We answer all raised questions collectively in this response. The rationale behind constructing this evaluation is that the originally assigned preference to the ground-truth item may not accurately reflect it semantically, since the preference generation is dependent on the timestep. This means that for each ground truth item only the preceding items have been used for preference generation to prevent information leakage. Therefore there is a chance that the preference does not accurately reflect the semantics of the ground truth item, which preserves the underlying aleatoric uncertainty of the recommendation task, as sometimes the interaction history is not informative for predicting the next item. In fact, our conducted user study (Appendix F) confirms this, as there is approximately a 30-50% chance that a generated preference that correctly approximates the user’s preferences does not relate to the target item. Therefore we need to match them and construct new sequences (answering Q4,Q5).

To ensure that preferences and items are semantically related, we conduct what we call the “association” of preferences and items. To this end we collect all generated preferences across all users and match them to items via cosine similarity in SentenceT5 space, i.e. $p_1$ is the preference that yields the highest cosine similarity to $\tilde{i}t$ and $p2$ has the highest cosine similarity to $\hat{i}t$ (Eq. 3). $p_1$ and $p_2$ stem from the entire set of preferences (answering Q5). Therefore we ensure that $p1$ is semantically related to $\tilde{i}_t$ and $p1$ is semantically related to $\hat{i}t$. The final sequences are then constructed out of the original sequence, where we replace the original ($p$, $i_t$) pair with either ($p_1$, $\tilde{i}t$) or ($p_2$, $\hat{i}t$). The motivation for combining ($p_1$, $\tilde{i}t$) with an additional sequence of $\hat{u}$ is merely to add additional variability in the generated data (answering Q6).

Finally, the rationale for the fine/coarse-grained steering is as follows: We ask whether the model is capable of predicting a very similar item to the ground truth item or a very distinct one, only by altering the preference. Intuitively, both scenarios evaluate whether the model can accurately follow a user preference, as it semantically reflects the item. However, there is one important difference, namely that in fine-grained steering the interaction history may provide useful information that helps predicting the next item, as the item used to replace the ground truth item is very similar to it. However, in coarse-grained steering this is not the case. In coarse-grained steering the interaction history is not helpful and the model must rely solely on the user preference. Therefore the two evaluation scenarios are complementary.

Novelty:

We would like to thank the reviewer for making us aware of other existing approaches to preference approximation, which we included in the related work section. However, we did not claim that the preference extraction pipeline is our main contribution. In fact, it is not even mentioned in our explicit contribution list.

To clarify, our contributions are:

• Introducing a novel paradigm called preference discerning, where the generative sequential recommendation system is conditioned on user preferences within its context.
• We propose a comprehensive benchmark for evaluating preference discerning, comprising of five distinct evaluation scenarios that provide a holistic assessment of its capabilities
• We present Mender, a multimodal baseline that integrates collaborative semantics with language preferences, achieving state-of-the-art performance on our proposed benchmark.

To relate our preference generation pipeline to the mentioned works we extended the related work section. [1] relies on conextualization via user embeddings, [2] relies on a complex multi-stage pipeline using distillation from teacher LLMs, and [3] evaluates whether LLMs can implicitly model user preferences. Our preference approximation pipeline simply prompts an LLM given user and item-specific data, without the need for distillation. We also conducted a user study with 22 participants, during which we assessed 2,200 generated preferences to evaluate their accuracy (see Appendix F). The results indicate that, on average across datasets, approximately 74% of the preferences accurately reflect the users' true preferences.

Preference approximation is a necessary prerequisite to enable preference discerning, i.e. in-context conditioning on user preferences. We acknowledge, however, that some of our phrasing might have been misleading in terms that it is one of our contributions. Therefore we rephrased parts of Section 4 and merged Section 3 into the Methodology section.

Benchmark generation

We thank the reviewer for pointing out the simplicity of our benchmark design. In fact, our aim was to keep it simple and intuitive while effectively evaluating for different real-world scenarios. Furthermore, we conducted a user-study in which we gathered feedback on 2200 generated user preferences and asked participants whether those accurately approximate the user’s preferences (see Appendix F). The result of the study is that on average across datasets around 74% of the preferences accurately reflect the user’s preferences. Finally, we did not claim that the five different axes in our benchmark are equally important (answering Q1). They were designed in a manner to allow decision makers to prioritize them based on their downstream requirements.

References:

[1] User-LLM: Efficient LLM Contextualization with User Embeddings, Ning et al., arXiv:2402.13598

[2] Review-driven Personalized Preference Reasoning with Large Language Models for Recommendation, Kim et al., arXiv:2408.06276

[3] Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction, Kang et al., arXiv:2305.06474

​​We thank the reviewer for the positive response and the constructive feedback. We have addressed the identified weaknesses as follows.

Presentation:

For clarity, we combined Section 3 with Section 4 and included pseudocode along with a more concise mathematical formulation. This clarifies that the preference generation process is conducted for each timestep $t$ in the user sequence $s_u$ and also provides a more concise descripion of the benchmark generation process.

Additional discussions:

We define personalization as a recommendation system's ability to align with user preferences, which we assess through a proxy: performance on the recommendation task. Essentially, the more personalized a system is, the better its recommendation performance. Inspired by recent studies that utilize language to represent user preferences [1,2,3], we enhance recommendation systems by conditioning them on user preferences within their context. This approach reveals emergent capabilities; for instance, a system trained in this manner can be steered via user preferences, or can be trained to improve sentiment understanding. We evaluate these emerging capabilities using our benchmark. Our work has profound implications for the field of sequential recommendation, as it enables enhanced personalization, allowing us to leverage organic data such as posts, comments, and likes given by social media platforms. We added a more in-depth discussion on the different evaluation axes (Section 3) and linked them to practical use cases they mirror.

References:

[1] Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences, Sanner et al., RecSys 2023

[2] Review-driven Personalized Preference Reasoning with Large Language Models for Recommendation, Kim et al., arXiv:2408.06276

[3] Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction, Kang et al., arXiv:2305.06474

We would like to thank the reviewer for the feedback which helps us significantly improve our manuscript.

Contribution of Mender (Q1):

The key aspect that distinguishes Mender from existing methods is that it follows a modular architecture where the encoder processes high-level user preferences and item descriptions and primes the decoder which learns to predict the low-level fine-grained interactions. Therefore, it leverages two different representations for items that differ in either components. To the best of our knowledge, no other recommendation system is based on this concept. We believe Mender is a novel architecture that unlocks new capabilities for recommender systems, and sets a new SoTA, without unnecessary overcomplications. We see this as an advantage, proven by the provided empirical results on multiple datasets and compared to several baselines.

The key aspect that distinguishes Mender from existing methods is its modular architecture. In this setup, the encoder processes high-level user preferences and item descriptions, priming the decoder to learn and predict low-level, fine-grained interactions. This approach utilizes two distinct representations for items, differing in either component: 1) natural language as input representation to encoder, and 2) semantic IDs as output representation from decoder. To the best of our knowledge, no other recommendation system is based on this concept. We believe Mender represents a novel architecture that unlocks new capabilities for recommender systems and establishes a new state-of-the-art without unnecessary overcomplications. We see this as an advantage, proven by the empirical results provided across multiple datasets and in comparison to several baselines.

Cross-validation of benchmark (Q2):

As correctly pointed out by the reviewer, the benchmark validates the effectiveness of incorporating generated preferences. In fact, it is the preference-based recommendation scenarios that truly validate the effectiveness of incorporating generated user preferences. The remaining evaluation axes are carefully designed to assess alternative real-world use cases. Below, we provide a few examples:

• Sentiment Following: This axis is essential for leveraging organic data. On social media, we have access not only to users' interactions with entities such as items or ads, but also to their organic data, such as posts, comments, and likes. For example, a user might express dislike for a specific phone brand in their posts or comments but may not have interacted with that item/ad. Sentiment following allows the system to handle these situations by transferring preferences from social interactions to item/ad recommendations.
• Fine-Grained & Coarse-Grained Steering: This feature is valuable for utilizing organic data in recommendation. For instance, if a user advocates for exercise and fitness, and against using weight-loss drugs, and participates in forum/comment discussions expressing this sentiment, the model can steer recommendations away from weight-loss medications, even if the user has purchased them in the past.
• History Consolidation: User preferences often evolve over time, and users typically have varying preferences for different items. For instance, a user might initially prefer dark roast coffee, but gradually develop a taste for lighter, more floral blends. In such cases, if the recent user preferences are absent, the recommendation system should adapt by updating its recommendations based on recent purchases and discarding outdated preferences. This is precisely what we aim to evaluate with history consolidation.

To incorporate objective feedback, we have conducted a user study in which 22 participants have evaluated the quality of the generated user preferences and their relevance to items (see Appendix F). Participants assessed 2,200 preferences across four different datasets and found that, on average, approximately 74% of the generated preferences accurately approximated the users' true preferences across all datasets.

Codebase:

We will release the codebase along with the camera-ready version.

Motivation:

We define personalization as the ability of a recommendation system to follow user preferences and examine personalization via a proxy, which is performance on the recommendation task, i.e. the more personalized a system, the better its recommendation performance. Motivated by recent works that leverage language for representing user preferences [1,2,3] we enhance recommendation systems by conditioning them on user preferences in their context. By doing so, we observe emergent abilities, for example a system trained in this manner can be steered by user preferences, or can be trained to improve sentiment understanding. We evaluate such emerging capabilities via our benchmark. Our work has profound implications on the field of sequential recommendation as it enables improved personalization and opens a more interactive view on sequential recommendation systems, as users may interact with the system by providing their preferences to them.

Efficiency (Q4):

We added Table 7 in the Appendix which compares both Mender variants compared to SASRec in terms of recommendation performance, training time, and inference time. For convenience we also show this table below. Mender-Emb trains an order of magnitude faster, while reaching lower performance than Mender-Emb, however it still significantly outperforms traditional methods such as SASRec [4] on most datasets, while approximately matching its training time. Finally, the improved performance of Mender comes with additional inference costs.

Why only generative models? (Q3)

We focused on generative retrieval since several recent works have shown that they usually perform on-par or better than dense retrieval methods, establishing themselves as state of the art approaches [5,6,7]. Additionally, generative retrieval methods offer a main advantage over dense retrieval methods in recommender systems: inference time efficiency at scale. A generative model with proper tokenizer can directly predict the next item whereas a dense retrieval model needs to perform pairwise comparisons between a user and all items to rank them. Theoretically, it is possible to obtain a new method that incorporates dense retrieval with preference conditioning. We believe this is an interesting question to explore in future work.

References:

[1] Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences, Sanner et al., RecSys 2023

[2] Review-driven Personalized Preference Reasoning with Large Language Models for Recommendation, Kim et al., arXiv:2408.06276

[3] Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction, Kang et al., arXiv:2305.06474

[4] Self-Attentive Sequential Recommendation, Kang et al., ICDM 2018

[5] Recommender Systems with Generative Retrieval, Rajput et al., NeurIPS 2023

[6] GenRec: Generative Sequential Recommendation with Large Language Models, Cao et al., ECIR 2024

[7] Generative Sequential Recommendation with GPTRec, Petrov et al., Gen-IR@SIGIR2023