Thank you for recognizing the value of our work and for your constructive feedback. We appreciate your insightful comments and suggestions.

1. what is the exact role of LM in the designed method

Thanks for the interesting comment. Although we directly inject the embeddings into the LM, without using a pretrained LLM, we cannot guarantee the language quality of responses generated by the trained model. The generalization capability of pretrained LLMs helps improve response quality even for specific downstream domains, as observed in prior work.

In domains with large, diverse, high-quality domain-specific textual datasets, training an LLM from scratch followed by RLAIF training may be beneficial. However, using a pretrained (or instruction-tuned) LLM will likely significantly enhance language quality. The tradeoff may include potential hallucinations, as information from the pre-training dataset can be ingrained in the model, causing it to include information not supported in the given context in responses. In this case, P4LM's training framework, which ensures factually grounded responses through the Precision RM, can mitigate the impact of hallucinations.

2. the improvements of the endorsement quality will or will not eventually lead to the improvements of business metrics

This is indeed a very interesting question. We plan to explore this as future work, where P4LM can be integrated into a deployed ConvRec system to generate endorsement text for the items selected by the recommendation engine. For the scope of this work, we focus on the endorsement task itself, isolating this problem to better evaluate endorsement quality through extensive model-based and human evaluations.

Thank you for your thoughtful and constructive feedback. We appreciate your insights and have addressed each point as follows:

1. CF embeddings...

Our main contribution is identifying the four criteria for recommendation endorsements and using RLAIF to achieve them. Defining optimal textual user profiles is beyond the scope of this work. In this regard, mapping from a CF embedding space to a word-token embedding space allows us to use diverse sources of user data, reducing reliance on specific prompts. The output from adapter $W_{\mathcal{U}}$ compactly represents collaborative-filtering-based and other user details, minimizing dependence on specific profile formatting.

2. text-based RLAIF baseline

We agree that text-based RLAIF is an important baseline. However, we aimed to avoid relying on specific user profile formats. Preliminary trials showed this baseline struggled to maximize the rewards, possibly due to increased context lengths. We are re-evaluating this baseline and will include the results in the final version.

3. the word precision

We understand the concern, and we will adopt a different term in our revised paper.

4. conflict between criteria 3 and 4

Although we acknowledge a potential conflict between these criteria, these scores are positively correlated in practice. Tables 6 and 7 show that training with only the Ppr or Pcov RM resulted in high scores for both. This is because Pcov RM training does not exclude prioritization. We will expand on this in the final version.

5. human vs. model-based (MB) evaluation

As noted on page 8, MB and human evaluations align well on Amazon, but differ on MovieLens. Raters preferred P4LM much more w.r.t. Pcov, Ppr, and App scores than MB evaluations suggest. This likely arises because these RMs were trained with PaLM2-L data. Using a different LLM for data generation might help, but won't fully resolve the issue. We will evaluate RM alignment with raters' responses, comparing RM and rater preferences for each endorsement text.

6. overall improvement by P4LM

We do have overall quality comparison from the raters (ratios of preferred responses over SFT's):

These results show significant overall quality improvement with P4LM as evaluated by human raters. We will include these in the final version.

Thanks for your valuable feedback. Please find brief responses below.

1. many articles on combining large models with user information … should be compared with one or two traditional models.

Indeed, there is recent work that combines user information with large language models (e.g., [1, 2, 3], listed below). However, these papers focus on the recommendation task itself, whereas our work addresses the novel task of generating endorsement text for recommended items that satisfy the criteria of precision, appeal, preference coverage, and prioritization. As such, these methods do not admit a direct comparison with ours. Traditional recommenders are also unsuitable for our task as they do not generate natural language responses.

[1] CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation (Zhang et al., 2023)

[2] Representation Learning with Large Language Models for Recommendation (Ren et al., 2024)

[3] Collaborative Large Language Model for Recommender Systems (Zhu et al., 2024)

2. The abscissa in Figure 2 is not shown...

We appreciate the feedback on Figure 2 and will revise it to include a clear abscissa and ordinate, as well as combine the four indicators on a common coordinate axis to enhance clarity.

3. The main experiments of the article should be reflected in the main text, such as ablation experiments, etc., and should not be found in the appendix…

We acknowledge the importance of having main experiments in the primary text. Unfortunately, due to the strict 9-page limit excluding references, we were forced to place some details in the appendix. If accepted, in the camera-ready version, which allows an additional page, we will incorporate key results from the appendix into the main text.

We discuss ablation studies on page 8 in the last paragraph and refer readers to the corresponding figures and tables in the appendix. We would definitely welcome any suggestions you may have on specific results that should ideally be included in the main text.

4. ...not mentioned at all in the text, such as Figure 2

We reference Figure 2 on page 8, second line. We will ensure that all figures and tables are explicitly mentioned in the text to improve readability.

5. too many references in the appendix; no anonymous code link

We believe these references provide comprehensive context and support for our work. We plan to release the code once the blind review period is over.