Calibrating Video Watch-time Predictions with Credible Prototype Alignment

Dear Reviewer KkGb,

We appreciate your valuable questions and suggestions. We summarize your concerns below and provide responses.

Q1: The motivation is clear and explicit.

We mention that existing WTP models struggle to achieve high accuracy. We believe the main reason lies in instance representation confusion, which we explain from two perspectives:

1. Mathematical Explanation

let the instance representation of a sample $(x,y)$ be $f(x)$, with its ideal center being $\mu_y = \mathbb{E}[f(x) \mid y]$, where $y$ is the ground-truth. The degree of instance representation confusion is defined as the distance between the instance representation and the ideal center $d(f(x), \mu_y) = \|f(x) - \mu_y\|.$ Then, the model's prediction error $\Delta_x = |y - \hat{y}|$ is predominantly correlated with the degree of instance representation confusion $d(f(x), \mu_y)$.

2. Model Performance

In existing WTP models, the instance representation space is often disorganized, with instance representations of different data types failing to form well-defined clusters, as shown in Figure4. Also, we provide additional results of different duration buckets in Reviewer WSao Q3. ProWTP's performance is even better in medium and long videos with large prediction errors, which further illustrates that ProWTP optimizes the distance between instance representations and prototypes to achieve more accurate predictions.

We believe that ProWTP improves prediction accuracy primarily by alleviating instance representation confusion and directly incorporating label distributions into the model, thereby providing additional information gain. So the motivation is clear and explicit.

Q2: Suitable for any deep recommendation model

This statement wants to show that ProWTP is a model-independent scheme, i.e., $f$ in $f(x)$ can be replaced by any depth model (i.e. DCN), and there is no overclaim.

Q3: Comparison with traditional clustering

In Table 3, we compare Kmeans' scheme for generating prototype. As for the clustering-based contrastive learning scheme, we have not searched relevant literature in WTP at this time.

Q4: Efficiency

A.4. has discussed this problem in details, OT is only present during training, OT is removed during testing, and online inference time complexity is linear O(CK).

Q5: lack of references

Its not TRUE. line 096 in Related work part, [1] proposes bimodal distribution, which we have already cited. In our work, we focus on the multi-modal distribution characteristics of videos. Previous studies have overlooked the distribution of watch-time beyond the video duration, and we are the first to propose focusing on the multimodal characteristics rather than just bimodal distribution. Importantly, we need to emphasize that the main contribution of our paper is the transformation of label distributions into prototypes that provide a credible reference for model calibrations.

Q6: Visualization

Thanks for your suggestion, we will add the use of kmeans-generated prototypes for ProWTP visualization subsequently.

Q7: Rationale for OT

In Appendix A.12, we compare three different alignment methods: OT, L2 distance, and no alignment. We present their results and conduct a case study. The OT alignment method performs global alignment rather than aligning each sample independently. Compared to other alignments, OT introduces structured information. This approach not only achieves better results but also produces sparser weights between instances and prototypes.

Q8: Efficiency Trade-off

A.4. has discussed this problem in details. We tested the wr distribution across months and its distribution does not change significantly, so it is reasonable that prototype is naturally fixed. If it changes in extreme cases, we just need to re-pull the data for training, as is always done in industry.

Q9: Robustness Across Scenario

This scenario does not occur in the WTP task, because according to our platform data analysis (its DAU is 400 millions), the users' watch-ratio always shows a multi-modal distribution. This is inherent in the WTP.

Q10: only two datasets

ITS NOT TRUE. In our paper it is three datasets and the results are reported averaged over five runs.

Q11. The interpretability of prototypes and their association with user behavior .

It has been discussed in A.3 in details. Due to space constraints, you can refer to A.3.

Q12: More baselines

The time of GR in arxiv is Sat, 28 Dec 2024, which is a same-time work and can be disregarded.

Q13: Figures modification

Due to space constraints, you can refer to Reviewer WSao Q1.

We sincerely thank you again for your feedback and hope that our responses can change your mind. We look forward to your reply and further discussion.

We sincerely thank you for recognizing the significance of our work and for your generous positive-score evaluation. We are very grateful for your valuable suggestions and constructive questions, which have helped us improve this paper. Below, we provide detailed responses to your queries:

Q1: modeling complexity(two-stage framework vs simple direct regression approaches)(refer to A.4)

Due to space constraints, please refer to the answer to Q3 in the rebuttal for Reviewer gDS8.

Q2: analysis of assign loss and compact loss

During Stage II, we adopt two losses to help with calibrating the sample space. In our method, we aims for the instance representations to cluster tightly around their corresponding prototypes. So the assign loss is designed to decrease the distance between samples and their corresponding prototypes. And since we hope for instances assigned to the same prototype to be closer together in the representation space, the compact loss is designed to encourage samples under the same prototype to cluster more closely in the representation space

By minimizing both losses, our method can not only help reduce instance representation confusion but also enhance the model’s ability to capture fine-grained features, ultimately improving prediction performance.

Q3: Model adjustment for changes in user behavior (refer to A.4)

We consider the following two aspects:

(1) The stability of user behavior

HVQ-VAE is completely independent, and its spatio-temporal complexity does not affect the training and inference time of ProWTP. When the distribution of watch ratios is sufficiently large, the resulting prototype distribution is stable. Furthermore, we observed that for a well-established video recommendation APP, the watch-ratio distribution remains largely unchanged and consistent across multiple months.

As shown in Figure 10, we randomly sampled 200,000 users from our APP (a short-video platform) and extracted their historical behavior on the 1st day of each month from January to November 2024. The data were divided into D=15 buckets based on video duration. We then computed the Wasserstein Distance between the watch-ratio probability density distributions of each month and November, as well as the Kolmogorov-Smirnov test with $p<0.05$ between their cumulative empirical distributions. The results indicated no significant distribution shifts across multiple months.

(2) Model adjustments driven by user behavior changes

Even in extreme scenarios where user behavior undergoes notable adjustments, we only need to resample the watch-ratio distributions for each duration buckets, perform offline retraining, and update the weights of ProWTP. This process incurs minimal computational overhead.

Q4: more findings from the online A/B test

According to the results, we observed two key findings:

1. ProWTP demonstrated consistent performance improvements over D2Q across all duration buckets.
2. When examining different duration segments, we found that medium and long-form videos (>60s) exhibited significantly more pronounced watch time benefits compared to shorter videos. This pattern also aligns with the results observed in our offline dataset evaluations.

Q5: alignment visualization

We visualized the relationship between errors and instance representations in Appendix A.1 (Figures 4 and 5). By comparing these visualizations, we observed that:

1. Prediction error ∆ is positively correlated with the degree of confusion.
2. TR exhibits a significantly higher level of confusion, while ProWTP effectively mitigates this confusion by reducing the distance between instances and reliable prototypes.
3. Compared to TR, ProWTP shows significantly fewer points with large errors.

This demonstrates that the root cause of reducing prediction errors lies in learning better instance representations.

Q6: Symbol list

Thank you for your suggestion. Due to space constraints of rebuttal, we will add a list of symbols in the final paper.

Thank you once again for your valuable suggestions and feedback. If you have any further questions, we would be happy to continue the discussion with you. We are looking forward to your reply.

Dear Reviewer gDS8,

We sincerely thank you for recognizing the significance of our work and for your generous positive-score evaluation. We are very grateful for your valuable suggestions and constructive questions, which have helped us improve this paper. Below, we provide detailed responses to your queries:

Q1: Lack of detailed analysis regarding the sensitivity to hyperparameters

In our method, we have two hyperparameters: K (number of prototypes) and β (weight of the compact loss). In Section 5.2, we have already provided the experimental results and analysis for hyperparameter K. Due to space constraints, here we only supplement the hyperparameter β experiments on the KuaiRand-Pure dataset.

Q2: Minor language and grammatical errors

Thank you for pointing out the writing issues, we will correct it in the final paper.

Q3: Complexity analysis and inference efficiency

Unlike simple direct regression approaches, our method incorporates an additional prototype generation process. But importantly, this prototype generation is completely independent, and its spatio-temporal complexity does not affect the training and inference time of ProWTP. When the distribution of watch ratios is sufficiently large, the resulting prototype distribution remains stable, which means that Stage I does not need to be performed frequently.

In Stage II, during the inference phase, the OT module is removed. The final value is computed as a linear combination of similarities to each prototype, which is then input into the regressor. The time complexity of this process is only O(CK), where C and K are small constants, ensuring that the time overhead remains negligible.

Q4: online experiments environments details

Thank you for your suggestion. Industrial recommendation systems typically employ a cascading architecture consisting of four stages: recall, pre-ranking, ranking, and re-ranking—a structure designed to efficiently recommend items to users from massive pools of video candidates. Since the recall stage operates without stringent real-time requirements, we strategically deployed ProWTP in this layer of an online short-video recommendation system, where it functions as one of multiple recall paths.

To validate its effectiveness in real-world applications, we used D2Q as our baseline model for comparison. The experimental results convincingly demonstrate the significant performance improvements our method delivers in actual business scenarios.

Q5: deal with imbalanced data distributions

Thanks for your question. In fact, video recommendation scenarios inherently face imbalanced data distributions in terms of video duration. Most video datasets, including WeChat and KuaiRand, exhibit left-skewed distributions in durations. Our experimental results clearly demonstrate that ProWTP outperforms competing models on the WTP task, showing particularly strong performance in watch time prediction under imbalanced data distribution conditions.

Thank you once again for your valuable suggestions and feedback. If you have any further questions, we would be happy to continue the discussion with you. We are looking forward to your reply.

Dear reviewer WSao,

We greatly appreciate your valuable questions and suggestions, which have helped us improve this paper significantly. We summarize your concerns below and provide detailed responses.

Q1: Details of Figures 1 and 2, and subsequent modifications

• Figure 1(a) shows the watch-ratio distributions of three different video duration buckets (short/mid/long) in the Wechat dataset.
• Figure 1(b) provides a dimensionality-reduced visualization of the instance representations $f(x)$ generated by the TR model on the Wechat dataset, using samples from four different video duration buckets to illustrate “instance representation confusion.” Since this figure does not fully convey the core issue of instance representation confusion, we plan to replace it with Figure 4(4) (from the Appendix) in the revised paper to present a clearer illustration in the Introduction.
• Figure 2 depicts the two-stage training process of ProWTP:
  (1) In the first stage, the watch-ratio distribution $W$ is used as input and reconstructed via the HVQ-VAE. We then extract the parameter $P$ (the codebook) obtained in this process to serve as the prototype parameter for the second stage.
  (2) In the second stage, sample features $X$ are used as input, and the output is the predicted $y$.

We will enhance Figure 2 with more symbols and annotations in our next revision for better clarity.

---

Q2: Experimental improvements and their significance in real-world scenarios

• Table 1 reports the average performance over five runs for each model. Generally, for ranking metrics like AUC, an improvement of 0.001 is considered significant. The improvement of 0.01 in RMSE and MAE metrics is significant, which is consistent with previous studies [1].
• Our online experiments indicate that different video duration buckets exhibit varying levels of watch-time gain, with medium-to-long videos seeing the greatest benefits. This outcome suggests that ProWTP can deliver higher gains in practice by recalibrating samples that are more prone to large prediction errors (i.e., medium and long videos).

[1] CREAD: A Classification-Restoration Framework with Error Adaptive Discretization for Watch Time Prediction in Video Recommender Systems.

---

Q3: Lack of prototype visualization and error analysis

• We observed that the prototypes learned by ProWTP form distinct “striped” clusters in high-dimensional space, and we will add a prototype visualization in the revised version.
• As mentioned in the previous section, ProWTP is particularly effective for medium-to-long videos, which inherently have larger prediction errors. By using prototypes as anchors and bringing samples with large errors closer, the overall performance is improved. Below is RMSE comparisons between TR and ProWTP for different video-duration buckets in the Wechat dataset to illustrate this improvement. According to the proof of A.1, the larger the prediction error, the further its f (x) is from the ideal center, then the larger ASSIGNMENT LOSS will be, and ProWTP will focus on optimizing such samples.

---

Q4: Explanation in A.1.1 regarding why only a small portion of training data falls into the non-activation region of ReLU

• We tested all three datasets; for each sample $(x, y)$, $y$ is the ground truth, and $ReLU(model(x))$ is the prediction. We found that the number of samples with $model(x) < 0$ (i.e., the activation output being zero) accounted for only about 1–2% of the total data in each dataset.
• This suggests that very few samples lie in the non-activation region of ReLU, supporting our assumption about instance confusion in Appendix A.1. We will include a scatter plot in subsequent revisions to further verify this claim.

---

Q5: Revisions to the main text and Appendix linkage to improve readability

We appreciate your suggestions and plan to reference Proposition A.1 in Section 4.3. At the same time, we will revise Figure 1(b) to more clearly explain the concept of instance representation confusion. By doing so, we hope to guide readers to the relevant sections of the Appendix and ensure they can easily find the in-depth explanations presented there.

---

We sincerely thank you again for your feedback and hope that our responses can change your mind. We look forward to your reply and further discussion.