SLMRec: Distilling Large Language Models into Small for Sequential Recommendation

We truly appreciate your constructive criticism and suggestions, which have helped us refine and strengthen the paper. We aim to clarify any misunderstandings about our technical contributions by addressing your questions point by point.

"Q1. Novelty of knowledge distillation for LLM."

Distinctions in knowledge distillation. Recent LLM knowledge distillation methods, as illustrated in Fig. 1 of [1], focus on transferring knowledge from closed-source teacher models (such as GPT-4 and Claude) to improve smaller open-source models (like LLaMA). Due to the closed-source characteristic of these teacher models, researchers can only access their generated results, not their intermediate representations.

In contrast, our SLMRec takes a different approach. We utilize the hidden representations from our open-source teacher model to guide the student model in developing similar feature extraction capabilities. Our teacher and student models are open-source and specifically trained on recommendation tasks, making them domain-specialized models. This technical distinction sets our proposed method apart from recent LLM knowledge distillation approaches [1].

Distinctions in motivation. The knowledge distillation approaches in [1] aim to leverage advanced capabilities from proprietary models like GPT-4 or Gemini to enhance open-source LLMs. They aim at distilling high-quality data from the closed-source LLMs

In contrast, our SLMRec's motivation stems from our motivational experiments detailed in Section 2. Based on these experimental insights, we implement feature distillation to guide the student model in learning to encode hidden representations similar to those of the teacher model.

Efficiency. Experimental results show that SLMRec achieves state-of-the-art performance using just 13% of the parameters of current LLM-based recommendation models while providing substantial computational benefits: 6.6x faster training and 8.0x faster inference times. Based on Google Colab's pricing for A100 GPU instances, our algorithm can reduce model serving costs by approximately $1,000 per month per A100 GPU. From a practical perspective, our SLMRec offers insights into efficiently deploying lightweight language models for recommendation systems.

"Q2. Explanation of the claim in the motivational experiment."

Thank you for your interesting question. We have discussed the phenomenon in the initially submitted PDF.

Explain about 32-layer model performance. When the number of layers increases, the performance of the model also improves. We specifically focus on the 8-24 layer range because the performance improvement in this range is slight. While we observe a more significant performance gain from 24 to 32 layers, we hypothesize this is due to the complete 32-layer architecture being utilized during pre-training despite not being pre-trained specifically for recommendation tasks.

Our findings. Our motivational experiments are based on empirical observations, with additional supporting evidence in the appendix showing consistent patterns. When varying the number of layers from 8 to 24, we observe only marginal performance improvements. Specifically, an 8-layer E4SRec can generate user representations nearly as informative as those from a 24-layer E4SRec. This observation suggests that intermediate layers in LLMRec models may be redundant, and smaller language models may potentially match the feature extraction capabilities of larger models through effective knowledge transfer.

"Q3. Explanation of the theoretical part."

Emergent abilities. Emergent abilities show how model capabilities suddenly appear at certain parameter scales [2]. All the experiments and insights are designed in the NLP fields. All LLMs employed in our study are trained and optimized for recommendation tasks, serving as domain-specific recommendation models. Meanwhile, the phenomenon of emergent abilities within the recommendation domain remains largely unexplored and presents an open research question. Therefore, there is no conflict between their conclusions and our research findings.

Recent works [3-6] have focused on aligning LLMs with human preferences and implementing data-cleaning mechanisms to improve small language model performance. Their findings show that well-curated data allows smaller models to achieve comparable results to larger ones, paralleling the core principle of our SLMRec approach.

Our theoretical justification. Our theoretical justification indicates that for any multi-layer stacked decoder, an equivalent single-layer decoder can be constructed to encode hidden representations in a similar way. Moreover, while the multi-layer model optimizes distinct objectives at each layer, this may introduce redundancy compared to a single-layer model that achieves its objective in a single step.

Reference:

[1] Xu, Xiaohan, et al. "A survey on knowledge distillation of large language models." arXiv preprint arXiv:2402.13116 (2024).

[2] Wei, Jason, et al. "Emergent abilities of large language models." arXiv preprint arXiv:2206.07682 (2022).

[3] Chen, Zixiang, et al. "Self-play fine-tuning converts weak language models to strong language models." ICML 2024.

[4] Bansal, Hritik, et al. "Smaller, weaker, yet better: Training llm reasoners via compute-optimal sampling." arXiv preprint arXiv:2408.16737 (2024).

[5] Rosset, Corby, et al. "Direct nash optimization: Teaching language models to self-improve with general preferences." arXiv preprint arXiv:2404.03715 (2024).

[6] Wu, Yue, et al. "Self-play preference optimization for language model alignment." arXiv preprint arXiv:2405.00675 (2024).

Dear Reviewer,

We have submitted our detailed rebuttal addressing all raised concerns. We would greatly appreciate it if you could review our response and let us know if any aspects require further clarification.

Best regards,

Authors

Thank you for your thorough and insightful review of our work. We appreciate your recognition of SLMRec's significant efficiency improvements while maintaining competitive performance, its practical applicability within real-world resource constraints, our novel approach to knowledge distillation for enhanced representation learning, and our comprehensive experimental validation across multiple domains. Your detailed feedback highlights the broad impact and technical contributions of our research.

"Q1. The implementation of knowledge distillation is too complex to apply."

Our methodology primarily employs offline knowledge distillation, wherein the teacher model undergoes pre-training before the distillation process. As illustrated in Figure 4 and extensively discussed in Section 5.3, we also investigate the efficacy of online knowledge distillation in SLMRec, which eliminates the need for teacher model pre-training.

For practitioners interested in implementing our framework with reduced computational complexity, we recommend exploring the online knowledge distillation variant. However, it is crucial to understand the inherent trade-offs: offline knowledge distillation typically ensures training stability and consistent performance, while online knowledge distillation offers implementation simplicity but may exhibit convergence variability across different scenarios [1]. Considering our objective of developing a robust and generalizable methodology across diverse datasets, we ultimately selected the offline knowledge distillation approach as our primary implementation strategy. Improving the stability of online knowledge distillation strategies represents a promising direction for future SLMRec development.

Reference:

[1] Gou, Jianping, et al. "Knowledge distillation: A survey." International Journal of Computer Vision 129.6 (2021): 1789-1819.

Dear Reviewer,

We have submitted our detailed rebuttal addressing all raised concerns. We would greatly appreciate it if you could review our response and let us know if any aspects require further clarification.

Best regards,

Authors

Thank you for your positive evaluation of our work. We appreciate your recognition of our key findings regarding model redundancy, the effectiveness of SLMRec in achieving competitive performance with significantly fewer parameters, and its compatibility with other efficiency-enhancing techniques. We address the two question points that can help to further improve the clarity.

"Q1. The details of the knowledge distillation process."

In this work, we adopt feature-based rather than logit-based knowledge distillation, thus eliminating the need for temperature hyperparameters. Our goal is to enable the student model to learn hidden representation encoding similar to the teacher model rather than simply mimicking predictions. To achieve this, we implement two straightforward feature distillation functions: feature similarity ($D_{cos}$) and feature norm regularization ($D_{norm}$). As shown in Equations 3 and 4, these distillation functions are applied after each block of multiple layers, enabling fewer layers to extract feature representations as effectively as many layers.

"Q2. What are multiple supervisory signals? How do we combine them in the training process?"

Multiple supervisory signals are designed to guide the student model in learning specific aspects of recommendation-related knowledge. For each block, we introduce prediction layers that compute user-item prediction scores from the feature representations, optimized using cross-entropy loss (Eq. 1). During training, these multiple supervisory signals work in conjunction with the two feature distillation loss functions to optimize the student model, as formulated in Eq. 6. For clarity, we have revised Figure 3 of the updated PDF to better illustrate the multiple supervisory signals.

Dear Reviewer,

We have submitted our detailed rebuttal addressing all raised concerns. We would greatly appreciate it if you could review our response and let us know if any aspects require further clarification.

Best regards,

Authors

We truly appreciate your constructive feedback and suggestions, which have helped us strengthen the paper. In the following, we clarify the misunderstanding and supplement more comparison with three suggested baselines.

"Q1. What is the motivation for distilling a small language model rather than finetuning this model? Moreover, what are the experiment results without distillation?"

Motivation. In our motivational experiments, we utilize E4SRec [1] to investigate potential parameter redundancy in LLMs. Notably, both E4SRec and SLMRec enhance prediction efficiency by adopting a BERT-style prediction head [2] for recommendation tasks, rather than generating next-item predictions autoregressively. This approach positions LLMs in embedding-based methods, including Lite-LLM4Rec [3], CLLM4Rec [4], E4SRec, and our SLMRec, primarily as sequential feature extractors.

Our experiments reveal that while model performance improves with increasing layer depth, the gains become marginal when scaling from 8 to 24 layers. This observed parameter redundancy motivates our adoption of knowledge distillation, enabling the student model to learn effective feature extraction strategies from the teacher model, as directly fine-tuning small language models may fail to achieve global optimization.

Experiment Results. In Tables 2 and 3 of the initially submitted PDF, we show the experiment results of directly finetuning a small language model, which is highlighted by the grey color. The E4SRec$_{4}$ model, that has the same decoder layers, signicantly lose the performance compared to SLMRec.

"Q2. How do we ensure that the teacher model can transfer the recommendation-related information?"

We aim to clarify the role of the teacher model in our approach.

LLM for Rec. Before knowledge distillation, we first train the LLMs (teacher model) on downstream recommendation datasets. This transforms the LLM into a recommendation-specific teacher model. The model and training details can refer to Figure 1 and Section 3 of our initially submitted PDF. During knowledge distillation, the student model learns to mimic the feature representations of the teacher model, which now contains domain-specific recommendation knowledge.

Given this background, it ensures the transfer of recommendation-specific expertise rather than general NLP knowledge.

"Q3. Authors should add self-supervised sequential recommendation methods."

Following the reviewer's feedback, we have added three self-supervised baselines [5-7] to our evaluation. Also, we report the average ranking of each method. Our comparative analysis encompasses 11 baseline methods, spanning traditional sequential approaches, self-supervised methods, and both generation-based (G-LLMRec) and embedding-based (E-LLMRec) recommendation architectures. Furthermore, a comprehensive introduction to these three self-supervised methods is provided in the Appendix of the updated PDF.

The self-supervised sequential methods demonstrate superior performance over traditional sequential methods, highlighting the effectiveness of incorporating self-supervised learning signals for enhanced sequential recommendation capabilities. Nevertheless, these approaches are inherently constrained by their limited model capacity and parameter space, resulting in performance gaps when compared to LLM-based recommendation systems.

Dear Reviewer,

We have submitted our detailed rebuttal addressing all raised concerns. We would greatly appreciate it if you could review our response and let us know if any aspects require further clarification.

Best regards,

Authors

Thank you for your reviewing. If our response indeed addresses your concerns, we'd appreciate it if you could adjust the rating accordingly.