Fine Tuning Out-of-Vocabulary Item Recommendation with User Sequence Imagination

Weakness：

W1. Figure 2 can be further improved. The presentation of Figure 1 is excellent, why not keep Figure 2 in the same style?

Thank you for your advice. We have revised Figure 2 in our updated PDF file. We hope this revised figure provides a clearer understanding of our work.

W2&W3. Typos and suggestions

Thank you for pointing these out, we will polish our paper in the revision.

Q1: The paper mentioned that USIM has been deployed on prominent e-commerce platforms. Are there any AB-Test results or some results from the industrial side? Thanks for your comments, we have conducted a two-week A/B test on billion-scale recommender systems and we'd like to provide the details.

Platform We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour.

Framework: Our online implementation consists of two core components: 1. Online Recommendation; and 2. USIM Imagination. We present our framework in Fig 1 in the uploaded PDF. When an OOV item is uploaded, we utilize the Large Language Model to embed the content features, including the product name and description. We then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced as the IV embeddings in the online recommendation model.

Baseline Comparisons: We conducted an online A/B test on 5% of users for each group over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI. The results of the online A/B tests on the industrial platform are summarized below.

The online recommendation results demonstrate that our USIM model significantly outperforms the baselines in all three metrics. Inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.

Q2.Could you discuss whether this model can be used for short video recommendation platforms?

Thank you for your interest in applying our model in short video recommendations. Our model can be applied to OOV recommendations in short videos. We obtain the item content information, such as tags, titles, and covers, and use pre-trained models to encode them and obtain the representations. These representations are then input into the USIM model, which generates the corresponding embeddings for each OOV item.

Thank you for your thoughtful review and feedback. We're glad our response addressed most of your concerns, and we appreciate your continued support for our paper.

Weakness and Questions：

W1: The RecPPO, which is a crucial component of the proposed framework, is not clearly defined in Section 3.3.2. Providing more detailed information about RecPPO would help readers better understand its role and implementation within the USIM framework.

Thank you for your comments. We incorporate recommendation-specific supervision signals into PPO, referring to this enhanced approach as Recommender-Oriented PPO (RecPPO), to train our USIM. When the state representation $h^i_t$ of specific item equal to its item embedding $e_i$, expected value should be 0. So we use these supervision signals to assist in training the value network $V_{\omega}$, the specific loss function of the value network is defined as follows:

$

\begin{aligned} \mathcal{L}(\omega)=\frac{1}{|B|} \sum_{(s_t, r_t, s_{t+1}) \in B} \left[ \left(r_t + \gamma V_\omega(s_{t+1}) - V_\omega(s_t)\right)^2 \right] + \frac{1}{|\mathcal{I}|} \sum_{i \in \mathcal{I}} V_\omega([{e}_i, \text{random}(0, N)])^2, \end{aligned}

$

where $B$ denotes tuples sampled from the buffer pool, and $\text{random}(0, N)$ is a random number between 0 and $N$. The first term is the Temporal Difference loss used in value network training, while the second term includes our recommendation-oriented supervision signals. Regardless of previous actions, when the state representation ${h}_t^i$ matches ${e}_i$, the agent should terminate. Therefore, $\text{random}(0, N)$ serves as the countdown for each termination state.

W2: Figure 2, which aims to illustrate the overall USIM process, could be further refined. Enhancing this figure to more clearly and comprehensively present the USIM process would improve the paper’s clarity and help readers grasp the proposed framework's intricacies.

Thank you for your insightful comments, we have revised and improved Fig 2, as illustrated in Fig 2 in the uploaded PDF. The updated figure provides a more detailed depiction of the USIM process, including the composition of the exploration set, the configuration of the reward function, and the state transition function. We hope this enhances your understanding of our work.

W3: Although the authors mention that USIM has been deployed on commercial platforms, there is a lack of detailed discussion on its industrial implementation and experiments. Providing insights into the deployment process, challenges faced, and specific industrial experiment results would add valuable context and strengthen the paper’s practical relevance.

Thanks for your comments, we have conducted a two-week A/B test on billion-scale recommender systems and we'd like to provide the details.

Platform We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour.

Framework: Our online implementation consists of two core components: 1. Online Recommendation; and 2. USIM Imagination. We present our framework in Fig 1 in the uploaded PDF. When an OOV item is uploaded, we utilize the Large Language Model to embed the content features, including the product name and description. We then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced as the IV embeddings in the online recommendation model.

Baseline Comparisons: We conducted an online A/B test on 5% of users for each group over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI. The results of the online A/B tests on the industrial platform are summarized below.

The online recommendation results, it presents that our USIM model significantly outperforms the baselines in all three metrics. Inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.

We greatly value your thorough review and kind feedback. If any additional concerns arise, please feel free to reach out to us. We’re ready to assist with more information.

W1 & W4 & Q3. Performance on large-scale datasets and online environments.

Thanks for your suggestion, we conduct further experiments on other OOV recommendation datasets and on real billion-scale online recommender systems.

Offline Experiments

We have conducted experiments on an additional larger-scale Book-Crossing dataset, containing 92,107 users, 270,170 items, and 1,031,175 interactions. Detailed results are presented as follows:

Experiments on the larger-scale Book-Crossing dataset also verify the strength of USIM.

Online Experiments

Platform We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour. The LLM service can parallelly process 20 requests of embedding, while the OOV service can parallelly process 512 items at the same time.

Framework (PDF Fig. 1): Our online implementation consists of two core components: 1. Online Recommendation; and 2. Offline USIM Imagination. When an OOV item is uploaded, we utilize LLM to embed the content features and then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced embeddings as the IV embeddings for online recommendation.

Baseline Comparisons: We conducted an online A/B test over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI. The results of the online A/B tests on the industrial platform are summarized below.

The online recommendation results, it presents that our USIM model significantly outperforms the baselines in all three metrics. Inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.

Efficiency

• USIM is Not the Bottleneck: The LLM is the major time consumer, while USIM is comparable to MetaEmb and ALDI.
• OOV Recommendation is an Offline, One-Time Process: As shown in PDF Fig. 1, the OOV recommendation speed does not impact online recommendations, and each item undergoes the OOV process only once.
• OOV Recommendation Can Be Parallelized: The online platform supports parallel processing, allowing USIM to be computed efficiently.

W2. Less significant improvement on MovieLens.

Thanks for your comments. USIM provides a new attempt at solving the OOV recommendation problem. We admit that previous models also can achieve good performance. However, USIM can still outperform the best-performing OOV recommendation baselines, especially on CiteULike and Book-Crossing.

W3 & Q2. Efficiency analysis.

Thank you for your insightful comments. We acknowledge the importance of evaluating the time efficiency of USIM, especially in comparison to SOTA baselines. Below, we present the detailed experimental time efficiency for USIM and SOTA methods.

• USIM is Faster than Heater and CLCRec: USIM computes embeddings only for OOV items, whereas Heater and CLCRec must compute embeddings for both OOV and IV items.
• USIM is Comparable with MetaEmb and ALDI: USIM imagines sequences only for OOV items, resulting in inference times comparable to MetaEmb and ALDI.
• USIM is Efficient in Training: By fundamentally addressing OOV recommendation, USIM converges in fewer epochs, making training more efficient.

Q1. Can the author provide the results of significance tests.

Thanks for your suggestion. In our main table, we have already run all experiments 10 times and reported the average performance. Following your advice, we have provided the significance test results in the PDF file.

W1. It is mentioned in the abstract that 'USIM has been deployed on a prominent e-commerce platform for months, offering recommendations for millions of OOV items and billions of users'. However, there seems no online comparison involved in the experiment. More details about online A/B tests should be discussed to support the claims in the abstract.

Thanks for your suggestion, we have conducted a two-week A/B test and we'd like to provide the details.

Platform We have implemented our USIM on the homepage of one of the largest e-commerce platforms, which boasts hundreds of millions of users and billions of items. The homepage features a feed recommendation system that recommends items to users. Thousands of new items are uploaded every hour.

Framework: Our online implementation consists of two core components: 1. Online Recommendation; and 2. USIM Imagination. We present our framework in Fig 1 in the uploaded PDF. When an OOV item is uploaded, we utilize the Large Language Model to embed the content features, including the product name and description. We then employ the USIM structure to predict the most suitable user sequence and optimize the embedding accordingly. Finally, we use the USIM-produced as the IV embeddings in the online recommendation model.

Baseline Comparisons: We conducted an online A/B test on 5% of users for each group over two consecutive weeks. We compared USIM against three different baselines: 1. Random, 2. MetaEmb, and 3. ALDI The results of the online A/B tests on the industrial platform are summarized below.

From the online recommendation results, it presents that our USIM model significantly outperforms the baselines in all three metrics. Further, inspired by these A/B test results, USIM is serving mainstream users and providing OOV recommendations for all newly uploaded items.

W2.There exist some statements lacking literature or empirical evidence, and some statements lacking clear description. For example:

• In Line 47 : 'The substantial distinction between the content features and the behavioral embeddings may result in substantial discrepancies between the IV and OOV items', which is not verified with evidence.
• Even though mentioned a lot, there is no clear definition about 'imagine user sequence', which seems not a commonly-used term.

Thank you for your comments. We will add literature and empirical support for the gap, and provide a clearer description of the user sequence imagination. Below are our responses to the specific points raised:

1. Evidence and Literature Support

   • Evidence: In recommendation systems, identical content items can elicit different user intentions. For example, two items with the same content may show vastly different engagement metrics—one may receive over a million clicks while the other gets none. This indicates that there are gaps between content features and behavioral embeddings.

   • Literature: Papers on out-of-vocabulary (OOV) recommendations, such as GAR [1], ALDI [2], and UCC [3], demonstrate that using content features to simulate behavioral embeddings can result in differences in embedding distributions and rating distributions. These studies support the existence of gaps between content features and behavioral embeddings.

2. Clear Definition of 'Imagine User Sequence'

   The term "imagine user sequence" refers to generating a sequence of hypothetical users using real user behavioral embeddings to refine the embeddings of an OOV item. We use "imagine" because these generated users are synthesized solely for embedding optimization. This technique allows for a more accurate representation of potential interactions, thereby improving recommendation performance for OOV items.

We hope these clarifications address your concerns. We will incorporate these explanations into our revised manuscript to ensure greater clarity.

[1] Chen H, Wang Z, et al. Generative adversarial framework for cold-start item recommendation. (SIGIR, 2022)

[2] Huang F, Wang Z, et al. Aligning distillation for cold-start item recommendation. (SIGIR, 2023)

[3] Liu T, Gao C, et al. Uncertainty-aware Consistency Learning for Cold-Start Item Recommendation. (SIGIR, 2023)

W3.The writing of the submission is too poor. The presentation of the submission is hard to follow. For example, in section 3.3.1, while the definition of MDP is described, how the MDP is related to the OOV issue is not discussed. The correspondence relation between (state, environment, action, reward) and OOV should be presented, otherwise it is quite challenging for readers to get through the proposed algorithm.

Thanks for your comments, we will revise our manuscript to improve the overall writing quality and ensure that the explanations are clear and comprehensive. Specifically, we will:

1. Provide a detailed explanation of how MDP is related to the OOV issue.
2. Clearly present the correspondence between (state, environment, action, reward) and the OOV problem.
3. Enhance the overall readability and flow of the paper.

We appreciate your constructive comments and will address these points in our revision to make the paper more accessible and easier to follow.