Self-Monitoring Large Language Models for Click-Through Rate Prediction

As shown in the questions in weaknesses:

1. The authors could give some examples about prompts whose embedding has a higer or lower adaptive temperature T.
2. If T value is extremly large or small, do you need to calibrate it?
3. Fig. 5 is not very clear, α, β in fomular 4 and 5 jointly effects the AUC and logloss, please show the grid search results.

1. In Figure 1, the best traditional CTR baseline really outperforms fine-tuned LLM in both head and tail items. However, in Table 2 and Table 3, traditional CTR baselines almost cannot outperform LLM-based baselines. Why does this happen? In clickpromt, authors claimed that semantic information is helpful in tail item prediction compared with traditional CTR models.

2. The prompt strategy is really important to fine-tune LLM for CTR tasks and also influence Equation 2 $T$. Should this be considered when the author argues feature attribution is important?

3. The datasets in this paper are too small to verify the cost of SLLMCTR because the click embedding matrix $\mathbold{E}$ is heavily related to the batch size, which is set to a small number of 32.

4. Can authors give an in-depth analysis of why adaptive temperature and label matching loss works?

• [Q1] Could you please provide more explanation on the feature attribution score analysis? Why more positive feature interactions (as in traditional CTR models) are more feasible than fewer ones (as in LLM-based models)?
• [Q2] The paper asserts that SLLM4CTR improves feature utilization, particularly through adaptive temperature. However, it would be helpful if the authors could provide more clarification on how exactly the feature distribution changes after applying adaptive temperature. Could you provide visualizations of attention weights before and after applying adaptive temperature to highlight this effect?
• [Q3] The introduction of the label matching loss is promising, but it would be valuable to include a more detailed justification for the specific choice of this loss function. How does the label matching loss compare to other regularization techniques (e.g., contrastive loss)? Additionally, it would be insightful to see how the representation space evolves with and without the label matching loss via t-SNE or PCA visualizations.

1. What is the necessity of introducing LLM into CTR prediction models?
2. Why numbers in Table 4 are so strange?

1. What does x_q represent in Equation (1)? It would be better to provide a brief explanation of the principle behind integrated gradient attributions.
2. Could the author explain why the adaptive temperature can increase the click prediction loss for low-confidence predictions (line 262)? For a low-confidence prediction with a click label of 1, assume the initial prediction distribution is P_{\text{click}=1} = 0.6 and P_{\text{click}=0} = 0.4 . A lower T would make the prediction distribution sharper, for example, changing it to P_{\text{click}=1} = 0.9 and P_{\text{click}=0} = 0.1 , which would reduce the loss—contradicting the author’s explanation.
3. On line 269, e_c uses the embeddings of all tokens in the input rather than only the embeddings of the user-item feature tokens. This means that the cosine similarity measures attention to all input tokens, rather than specifically to the user-item features, which contradicts the author’s explanation.
4. In Section 4.2, the author states that the purpose of the label matching loss is to make click and non-click labels easier to distinguish in the representation space. Could the author explain the relationship among distinguishing click and non-click labels in the representation space, the effectiveness of the LLM in matching user-item features with click labels, and the performance on tail items?