P-Law: Predicting Quantitative Scaling Law with Entropy Guidance in Large Recommendation Models

We sincerely thank you for your valuable comments. We hope the following responses can address your concerns.

W1 Although an approximate LZ compression algorithm is proposed, the computational cost may still be high in large-scale online settings, which is unfavorable for industrial applications

We thank the reviewer for pointing out the effectiveness trade-off. We have extended the fragmented averaging strategy to accelerate computations on industrial datasets by segmenting the original dataset into fragments of 100,000 sequences each, and then calculating their averages. This slicing strategy allows for full parallelization.

For the industrial dataset we used, which contains 513,878,761 sequences, the computation time was 1 hour and 32 minutes. This is relatively insignificant compared to the training time of recommendation models of a similar scale. For the 7,076,238-sequence Amazon Books dataset, the computation time was 15 minutes. (Due to parallel computation, the time does not scale strictly proportionally with the size.)

Additionally, this method is advantageous for updates, as new sequences can be simply trimmed to the same fragment size. To demonstrate the stability of fragmented averaging, we present the relationship between the computed values and fragment sizes as follows:

It can be observed that when the slice count exceeds 10,000, the results are already quite close to the true value. This demonstrates the rationality of using fragmented approximation.

W2.1 The sensitivity of the method to these hyperparameters can pose challenges in practical deployment.

Thank you for your attention to parameter sensitivity. For each model parameter (such as the number of layers $N$ and data $D$), our approach introduces only one additional hyperparameter compared to the Scaling Law: $p1$ for the model and $p2$ for the data. We fixed these additional parameters at 1 to align with the sensitivity of the Scaling Law and conducted the following comparison: the fitting capability differs from the original Performance Law by at most 0.5%, still outperforming other fitting approaches.

W2.2 Moreover, the optimal configuration is likely to vary significantly across different domains

The Scaling Law also varies significantly across different domains; however, it has found widespread application. Our shared aim with the Scaling Law is to predict model performance in each domain using the Performance Law fitted from small-scale experiments. The key difference between the Performance Law and the Scaling Law is that, rather than relying on the loss, the Performance Law directly fits the model's effectiveness.

W3 The experimental evaluation in this study lacks testing in several important special scenarios, such as graph-based recommendation and long-tail user segments.

Thank you for your suggestion on validation towards important special scenarios. We have included validation under LightGCN, as well as verification using the Amazon Beauty data, which is widely utilized in enhancing long-tail recommendations $1$

$2$ . Performance Law still outperforms other fitting approaches in these scenarios.

$1$ Liu Q, Wu X, Wang Y, et al. Llm-esr: Large language models enhancement for long-tailed sequential recommendation[J]. Advances in Neural Information Processing Systems, 2024, 37: 26701-26727.

$2$ Yang, K., Zheng, S., Li, T., Li, X., & Li, H. (2025). GENPLUGIN: A Plug-and-Play Framework for Long-Tail Generative Recommendation with Exposure Bias Mitigation.

W4 The experiments only compare the proposed method with general scaling law approaches, without comparing it to recommendation algorithms based on scaling laws.

Answer: Thank you for your reminder! Please see Q1.

Q1 It remains unclear why recommendation algorithms based on SL were not included in the evaluation. Could the authors clarify the distinctions between the proposed PL approach and other SL-based recommendation algorithms, such as Wukong?

The primary focus of Performance Law and Scaling Law is to provide theoretical proofs for scalability, setting them apart from frameworks like Wukong and HSTU, which primarily aim to enhance framework effect and stability.

Our main goal is to improve the predictive capabilities of frameworks under varying parameters. Both Performance Law and Scaling Law can demonstrate their effectiveness on different frameworks. This is evidenced in the Wukong table below, which includes examples such as Wukong and LightGCN in W3, as well as the transformer from our original paper. Notably, on the Wukong framework, Performance Law still outperforms both variations of Scaling Laws.

Q2.1 Why does not compare it with other information-theoretic measures (e.g., Shannon entropy or Kolmogorov complexity)

Apologies for the misunderstanding. This is because Shannon entropy and Kolmogorov complexity are used to compute the entropy of a set, not a sequence. To demonstrate the robustness of our model, we conducted additional experiments replacing Real Entropy with Approximate Entropy [3], Shannon Entropy, and Kolmogorov complexity. Real Entropy maintained optimal fitting capability.

$3$ S.M. Pincus, Approximate entropy as a measure of system complexity., Proc. Natl. Acad. Sci. U.S.A. 88 (6) 2297-2301, PNAS 1991.

Q2.2 Additionally, how does the choice of LZ compression—instead of other complexity estimators—affect the robustness of Real Entropy in noisy or non-stationary user behavior scenarios?

Thank you for pointing out any confusion regarding the concept of approximation. LZ compression is not an approximation; it is equivalent to the original computation. Therefore, it has no impact on robustness, noise, or non-stationary user behavior scenarios. We discussed our extension on its approximation towards computation cost in W1.

Q3.1 LZ compression may introduce computational bottlenecks.

Our method's computational complexity is entirely adequate for adoption in industrial scenarios, as referenced in W1.

Q3.2 Furthermore, how would the method handle dynamically evolving user interactions where Real Entropy needs incremental updates?

As stated at the end of W1, this method is also quite advantageous for updates, as new sequences only need to be trimmed to the same fragment size, or the user's corresponding fragment can be replaced.

We are grateful for your kind remarks. We hope the following responses can address your remaining concerns.

Q1 Since the experiments mainly focus on transformer-based models, I am curious whether the proposed formulation would also apply to other types of recommendation models (e.g., RNN, GNN, Mamba)

Answer: We appreciate your feedback! We have added a comparison of Performance Law within the LightGCN and Mamba frameworks, where it continues to outperform other Scaling Laws.

Q2 Have you conducted any experiments on recommendation models based on large language models (LLMs)?

In recommendation systems, LLMs are often utilized through fine-tuning, which differs from the large recommendation models studied in this paper. However, some studies [1] have mentioned that the input sequence length can also affect scaling outcomes. Therefore, we conducted a scaling analysis with sequences of similar user input lengths, as shown in the table below. The performance law still exhibits superior predictive capability compared to the scaling law.

$1$ Yue, Z., Zhuang, H., Bai, A., Hui, K., Jagerman, R., Zeng, H., Qin, Z., Wang, D., Wang, X., & Bendersky, M. (2025). Inference Scaling for Long-Context Retrieval Augmented Generation. ICLR 2025

We sincerely thank you for reviewing our manuscript. We hope the following responses can address your concerns.

W1 The claim that Performance Law is generalizable beyond SR is not empirically supported; all experiments are limited to sequential recommendation tasks.

Thank you for your advice! We evaluated the predictive performance of the Performance Law in graph learning (LightGCN) and content-based recommendation (Wukong), where it continues to demonstrate leading capabilities.

W2 First-order stationary Markov assumption for user sequences may not hold in real-world applications.

The Markov assumption for user sequences is considered reasonable in the recommendation domain $1$

$2$ , making its application in real-world scenarios appropriate as well. We introduce this condition mainly to ensure that the probability of a user's next item interaction can be calculated for all users. Since the Real Entropy for all datasets can be computed, this condition can effectively be overlooked.

[1] Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing personalized Markov chains for next-basket recommendation[C]//Proceedings of the 19th international conference on World Wide Web. 2010: 811-820.
[2] Yang Y, Jang H J, Kim B. A hybrid recommender system for sequential recommendation: combining similarity models with Markov chains[J]. IEEE Access, 2020, 8: 190136-190146.

W3 The baseline comparison is narrow, mainly focusing on SL variants; recent performance estimation techniques are not included？

Apologize for any misunderstanding. Recent research on large recommendation models has focused on model-based rather than theoretical approaches, with all enhanced framework recommendation models being guided by the Scaling Law. We have already discussed the comparison with the latest Precision Scaling Law [3], as shown in W5.

[3]T. Kumar, Z. Ankner, B. F. Spector, B. Bordelon, N. Muennighoff, M. Paul, C. Pehlevan, C. Ré,494 and A. Raghunathan, “Scaling laws for precision,” arXiv preprint arXiv:2411.04330, 2024

W4 The efficiency (runtime, memory) of computing Real Entropy, especially on industrial datasets, is not discussed.

We thank the reviewer for pointing out the effectiveness trade-off. We extende our computation on averaging approach to expedite computations on industrial datasets by segmenting the original dataset into shards of 100,000 sequences each and calculating their average values.

This segmentation strategy allows for complete parallelization, resulting in a computation time of 1 hour and 32 minutes for the industrial dataset containing 513,878,761 sequences. This is relatively efficient compared to the training time of recommendation models of this scale. For the Amazon Books dataset, which contains 7,076,238 sequences, the computation takes 15 minutes. (Due to parallel computing, the time is not strictly proportional to the dataset size.)

This method also facilitates updates, as new sequences just need to be divided into segments of the same size. To demonstrate the stability of sharded averaging, we enumerate the relationship between the calculated values and shard size as follows:

It can be observed that when the slice count exceeds 10,000, the results are already quite close to the true value. This demonstrates the rationality of using fragmented approximation.

W5 The evaluation metric is overly dependent on R²; additional metrics such as MAE/RMSE or downstream utility of predicted parameters would strengthen the empirical support.

Thank you for your valuable suggestion. We have supplemented the paper with all the compared MAE/RMSE values, and we still outperform the remaining Scaling Law analyses.

We sincerely thank you for your valuable comments. We hope the following responses can address your concerns.

W1 It is unclear whether the law applies equally well to other SR architectures (e.g., RNN- or GNN-based SR in Sec. 2) / advanced generative techniques (e.g., DM-based SR).

Thanks for the suggestions on validation for different structures. We have provided a comparison of the predictive performances between the LightGCN[1] and DiffuRec[2]. Overall, our results still surpass those of the Scaling Law and Precision Scaling Law.

[1]He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020). LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR'20

[2]Li Z, Sun A, Li C. Diffurec: A diffusion model for sequential recommendation[J]. ACM Transactions on Information Systems, 2023, 42(3): 1-28.

W2 The definitions of S^{real} (Real Entropy) and S^{\prime real} (the final measure of Real Entropy) are overly similar

Thanks for pointing out the presentation issue of this paragraph. In subsequent revisions, we will rename $(S^{real}$ to "Original Real Entropy" and modify $S^{\prime real}$ to $\overline{S^{real}}$ to enhance their distinctiveness.

W3 There are only the comparison between #Tokens·S^{\prime real} and #Tokens in In Figure 3. Including additional entropy-based data quality metrics in the comparison could further support the validity of the proposed Performance Law.

We have added experiments comparing the replacement of Real Entropy with Approximate Entropy [3], Shannon Entropy, and Kolmogorov Complexity (Implemented by compression rate)[4]. The results indicate that Real Entropy maintains the best fitting ability.

$3$ S.M. Pincus, Approximate entropy as a measure of system complexity., Proc. Natl. Acad. Sci. U.S.A. 88 (6) 2297-2301

[4] Yin, M., Wu, C., Wang, Y., Wang, H., Guo, W., Wang, Y., Liu, Y., Tang, R., Lian, D., & Chen, E. (2024). Entropy Law: The Story Behind Data Compression and LLM

Q1 Can you discuss the feasibility of the proposed Performance Law in other SR architectures / advanced generative techniques?

We appreciate your feedback! We have compared the fitting and predictive capabilities of the Performance Law against other baselines within the advanced generative frameworks of Mamba and DiffuRec (For DiffuRec, see reference W1 above). The Performance Law continues to demonstrate superiority.

Q2 It would be beneficial to include additional entropy-based data quality metrics for comparison.

Thank you for your suggestions regarding entropy value comparisons. For detailed information, please refer to W3.