Identify Then Recommend: Towards Unsupervised Group Recommendation

Response to Reviewer x7i3

Thanks for your valuable and constructive reviews. We appreciate your insights and suggestions, as they will undoubtedly contribute to improving the quality of our paper. In response to your concerns, we provide answers to the questions as follows in order.

Dimensions of Matrix

Thanks for your careful check! We have correct the dimension of the matrix $\mathbf{Q} \in \mathbb{R}^{k \times m}$ in the revised paper.

Existing Community Detection

Thanks for your question. The clustering/community detection methods, which need the pre-defined number of cluster centers, such as k-means, spectral clustering/community detection, cannot be applied in this scenario. For other clustering/community detection methods, which do not require the pre-defined number of cluster centers, such as DBSCAN, is hard to deal with our scenario well since 1) the data distribution of our application change dynamically, therefore it’s hard to determine the parameters regarding to the density, such the radius, and the threshold. But our proposed group identification module can automatically estimate the density and discover the groups via the heuristic merge-and-split strategy, thus can be easily applied in the real-time large-scale unsupervised recommendation system. 2) They can not be integrated into our framework without our proposed pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. We have added these claims in the revised paper.

Experimental Details

Thanks for your suggestion. For the A/B testing experiments, the evaluation metrics include two parts, including the click metrics and the trade metrics. For the click metrics, there are for metrics, including uv (user view), uvctr (user view click through rate), pv (page view), pvctr (page view click through rate). And for the trade metrics, there are two metrics, including including uv (user view) and pv (page view). For the data scale, the application contains about 130 million page view and 50 million user view per day. And the baseline model is the MMOE model for recommendation. We merely compared the baseline model with our strategy and the baseline model without our strategy to demonstrate the effectiveness of our proposed method. The details can be found in the application background in Section 6.4.1.

Detailed Introduction of the Application

We apply the propose ITR method in the livestreaming recommendation scenario. In a livestreaming room, the users naturally form several user groups based on their interests. For instance, in a sports goods livestreaming room, the users with basketball interests will form a user group, while the users with badminton interests will form another user group. These user groups are important information to describe the profile of the livestreaming room. For example, when this livestreaming room begins to sell basketball-related goods, the basketball user group will be activated and be willing to click, comment, or buy merchandise. Therefore, we should guide more users with basketball interests into this livestreaming room. Following this principle, we aim to group the users in the livestreaming room into different groups and utilize this group information to assist the recommendation. However, the number of user groups is always unknown and dynamic. Besides, the different livestreaming rooms have different group numbers, e.g., the gold livestreaming room has less group number than the groceries livestreaming room since user interests in the gold livestreaming room are more concentrated. In addition, the group annotation information, such as user-group distribution and group-item interactions, is also missing or needs extensive human annotation costs in this scenario. To solve these problems, we present ITR model in this paper. The proposed GIM can identify the user groups in the livestreaming rooms, and the proposed SGRM can assist and promote the recommendation. Furthermore, upon acquiring the identified group embeddings, we can also utilize them to enhance the traffic mechanism within real-time livestreaming recommendations. In practical terms, we initially designate the primary user groups within a livestreaming room as the room's profile. Additionally, we devise and calculate the matching score between the primary user groups and potential users, subsequently leveraging these matching scores to dynamically bolster the ranking score of relevant users, therefore improving the recommendation performance and the interactive environments in the livestreaming room.

Occasional Group Recommendation

Thanks for your suggestion. Although the occasional group recommendation is similar to our method, they are still essentially different. The setting of our experiments is purely unsupervised, and we are not merely solving the occasional group recommendation problem. But we are still glad to survey and discuss them [1-4] in the revised paper since the motivation of occasional group recommendation is partly similar to ours.

[1] Sankar et al. GroupIM: A Mutual Information Maximization Framework for Neural Group Recommendation. In SIGIR, 2020.

[2] Chen et al. Thinking Inside The Box: Learning Hypercube Representations for Group Recommendation. In SIGIR, 2022.

[3] Zhang et al. GBERT: Pre-training User Representations for Ephemeral Group Recommendation. In CIKM, 2022.

[4] Li et al. Self-Supervised Group Graph Collaborative Filtering for Group Recommendation. In WSDM, 2023.

Response to Reviewer x7i3 [2/3]

Technical Details During Inference

Thanks for your question. For our proposed ITR, the embeddings of the groups are generated as shown in Eq. (6) and are optimized by minimizing Eq. (8) and Eq. (11). Concretely, the adaptive density estimation, and the heuristic merge-and-split method will discover the group assignments, and the group embeddings will be formed with the average of the assigned user embeddings. During the training stage, the group embeddings are also optimized with the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. For the inference, during the testing stage, the group embeddings are learned, and user assignments are fixed. Therefore, we can calculate the recommended score between groups and the items (for the group recommendation task) and the recommended score between users and the items (for the user recommendation task). For calculating recommendation scores, we follow ConsRec and adopt the dot product or the neural mapping method. For applying ConsRec to the scenario without annotations, we just remove the interaction labels between the groups and the items, namely removing the group loss in Eq. (5) in the original paper of ConsRec.

Explanation of “base” & Utilization of ITR

Thanks for your question. The baseline model is the convention MMOE-based recommendation model. And we aim to adopt our proposed method at the re-rank stage, i.e., modified the predicted score of the recommendation model by considering the user groups. Next, we assume you have carefully read the background of this application and introduce the details of adopting our method into the re-rank stage. For different livestreaming rooms, the number of user groups will be different. For example, the gold livestreaming room has less group number than the groceries livestreaming room since user interests in the gold livestreaming room are more concentrated. Therefore, we need an unsupervised group recommendation method to discover the user groups in different livestreaming rooms. Here, we adopt our ITR method to discover the user groups and then produce the group embeddings. Then upon acquiring the identified group embeddings, we can also utilize them to enhance the traffic mechanism within real-time livestreaming recommendations. In practical terms, we initially designate the primary user groups within a livestreaming room as the room's profile. Additionally, we devise and calculate the matching score between the primary user groups and potential users, subsequently leveraging these matching scores to dynamically bolster the ranking score of relevant users, therefore improving the recommendation performance and the interactive environments in the livestreaming room. And our proposed ITR will not be used with the group annotations since if the group annotations are easy to obtain, we don’t need to develop an unsupervised learning-based group recommendation method.

Experiments for Occasional Group Recommendation

Thanks for your suggestions. In our paper, the setting of the experiments is purely unsupervised. However, the occasional group recommendation methods still rely on the annotation of the groups. Although they are essentially different, we are still glad to survey, discuss, and test them in the revised paper. Actually, for GroupIM [1] and CubeRec [2], we have already surveyed, discussed, and tested in our original paper. Please carefully check the related work part and the comparison experiment part in the original paper. For GBERT [3] and SGGCF [4], thanks for your suggestion and we will briefly introduce them and compare them with our method. GBERT [3] is proposed to solve the data sparsity and cold-start problems by the pre-training and fine-tuning techniques on BERT. SGGCF [4] aims to solve the data sparsity and high-order interaction problems by the heterogenous graph and the self-supervised learning. For GBERT [3], the code is not available online. Therefore, it’s hard to reproduce their results. And for SGGCF, we have started the experiments. Due to the limited rebuttal time, the experiments are not finished yet, and once they are finished, we will post the experimental results during the discussion period.

[1] Sankar et al. GroupIM: A Mutual Information Maximization Framework for Neural Group Recommendation. In SIGIR, 2020.

[2] Chen et al. Thinking Inside The Box: Learning Hypercube Representations for Group Recommendation. In SIGIR, 2022.

[3] Zhang et al. GBERT: Pre-training User Representations for Ephemeral Group Recommendation. In CIKM, 2022.

[4] Li et al. Self-Supervised Group Graph Collaborative Filtering for Group Recommendation. In WSDM, 2023.

to be continue...

Response to Reviewer x7i3 [3/3]

Effectiveness of GIM

Thanks for your suggestion. The group identification module (GIM) aims to discover the user groups based the embeddings in the pure unsupervised setting. In the pure unsupervised setting, the GIM is a must-used module to discover the user groups. And the effectiveness can be verified indirectly by the performance improvement of downstream tasks. To directly demonstrate the effectiveness of the proposed group identification module, we follow your suggestion and verify the quality of the discovered user group. Concretely, on the open benchmarks, we adopt the unsupervised clustering metric Silhouette Coefficient (SC) to evaluate the quality of the discovered user groups. SC is an important indicator for evaluating clustering quality, which can help evaluate the rationality of clustering results. It combines the cohesion and separation of each data point to quantify the clustering effect, and its value range is from -1 to 1. Specifically, the larger the SC, the better the clustering effect of the data points. On the average of the four datasets used in our paper, the discovered user groups achieve 0.879 SC, demonstrating the promising clustering performance of our proposed GIM. On the industrial data, due to the labels are not available and the data size being large, we conducted case studies to demonstrate the effectiveness of GIM. Specifically, we select some hot livestream rooms for the case studies, e.g., the gold-selling livestream room, movie-ticket-selling livestream room, and the face-tissue-selling livestream room. Then, we check the user group distribution and find the major group. Subsequently, we use some tags of the user interests to provide the labels of the major groups. In this manner, we can determine the interest of the major groups in the studies livestream room. We discuss the results as follows. For the gold-selling livestream room, the interests of the major groups include jewelry & gold and financial management. Besides, for the movie-ticket-selling livestream, the interests of the major groups include the movie and entertainment. Differently, for the face-tissue-selling livestream room, the interests of the major groups include food and hygiene products. These case studies can demonstrate that our proposed GIM method can effectively group the similar users to one group and separate the different groups. Also, the captured information of the major group can also be used for the refined recommendation.

Complexity Analyses & Convergence

Thanks for your suggestion. Following your suggestion, we conduct complexity analyses of our proposed ITR method. First of all, we define the number of the users, the number of groups, the average size of groups, as $n$, $k’$, $n/k’$, respectively. In the process of the adaptive density estimation, the time complexity and space complexity of calculating radius proposal for one group is $\mathcal{O}(1)$, $\mathcal{O}({k’}^{2})$, respectively, and for all groups is $\mathcal{O}(k’)$, $\mathcal{O}({k’}^{2})$, respectively. Then, the time complexity and space complexity of density estimation for all groups is $k’ \times n/k’$, $n \times k’$, respectively. Subsequently, at the heuristic merge-and-split strategy stage, for the explore step, it takes $\mathcal{O}(k’)$ time complexity, and $\mathcal{O}(1)$ space complexity, respectively. And for the exploit step, it takes $\mathcal{O}(k’ \times n/k’)$ time complexity, and $\mathcal{O}(nk’)$ space complexity, respectively. Besides, for the proposed pseudo recommendation pre-text task, the time complexity and space complexity is $\mathcal{O}(n \times k’)$, and $\mathcal{O}(n \times k’)$, respectively. In addition, for the proposed pull-and-repulsion pre-text task, the time complexity and space complexity is $\mathcal{O}(nk’+{k’}^{2})$, and $\mathcal{O}(nk’)$, respectively. Moreover, for the BPR loss, the time complexity and space complexity is $\mathcal{O}(n)$, and $\mathcal{O}(n)$, respectively. Overall, the time complexity and space complexity of our proposed ITR method is $\mathcal{O}(k’+ k’ \times n/k’+ k’+ k’ \times n/k’+ n \times k’+ nk’+{k’}^{2}+n) \rightarrow \mathcal{O}(nk'+{k’}^{2})$and $\mathcal{O}({k’}^{2}+ n \times k’+1+ nk’+ n \times k’+ nk’+n)\rightarrow \mathcal{O}(nk'+{k’}^{2})$, respectively. Therefore, our proposed method will not bring large memory and time costs since the complexity of our method is linear to the number of users.

For convergence, we checked the loss and performance of our method on all datasets and found that they converge well, i.e., the loss decreases and gradually converges to a low value, while the performance increases and gradually converges to a high value.

Dear Reviewer x7i3,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors

Dear Reviewer x7i3,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors

Response to Reviewer pdQ8T

Thanks for your valuable and constructive reviews. We appreciate your insights and suggestions, as they will undoubtedly contribute to improving the quality of our paper. In response to your concerns, we provide answers to the questions as follows in order.

Demonstration Figure & Code Ideas & Primary Designs

Thanks for your suggestion. We will add the demonstrate figures and the core ideas and primary designs before delving into the method part.

Advantages of ITR

Thanks for your question. The clustering methods, which need the pre-defined number of cluster centers, such as k-means, spectral clustering, cannot be applied in this scenario. For other clustering methods, which do not require the pre-defined number of cluster centers, such as DBSCAN, is hard to deal with our scenario well since 1) the data distribution of our application change dynamically, therefore it’s hard to determine the parameters regarding to the density, such the radius, and the threshold. But our proposed group identification module can automatically estimate the density and discover the groups via the heuristic merge-and-split strategy, thus can be easily applied in the real-time large-scale unsupervised recommendation system. 2) They can not be integrated into our framework without our proposed pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. We have added these claims in the revised paper.

Scalability

Thanks for your question. In this paper, we aim to solve the practical problems in the real-time large-scale industrial recommendation system. Therefore, we first design our method and conduct quick experiments on the toy open benchmarks. Then, we conduct extensive experiments on the real-time large-scale data in the application (with about 130 million page views / 50 million user views per day). We admit the scalability of the open benchmarks is limited, but we think it is reasonable for quick trials, and our final aim is to deploy the method in real-world applications. The details regarding the application can be found in Section 6.4.

Hyper-parameter Experiments

Thanks. We conduct the hyper-parameter experiments in the Section 6.3. The hyper-parameters mainly include the trade-off parameters $a$ and $b$. During the density estimation and merge-and-split stage, there are no introduced hyper-parameters.

Response to Reviewer dQ8T [2/2]

Core Ideas & Primary Designs

Thanks for your suggestion. Following your suggestion, we briefly introduce the core idea and the primary designs of our proposed method before introducing the method part. Concretely, we aim to develop an unsupervised group recommendation method since we find the promising performance of existing state-of-the-art methods relies on the annotations of the groups. The experimental evidences can be found in Figure 1. However, in the real-world scenario, the annotations regarding the group-item interactions and the user-group assignments are always not available. And labeling them in the real-time recommendation system is expensive and even impossible. To this end, we develop a pure unsupervised group recommendation method, which can automatically discover the user groups and provide the precise recommendation for them. Therefore, the core ideas of our methods are twofold, including the group identification and the group recommendation. For the group identification, our initial solution is to adopt some existing clustering or community detection methods, which do not require the number of clusters, e.g., DBSCAN, DeepDPM, etc. However, these methods can not automatically discover the user groups since they need other hyper-parameters such as the radius and calculation of the density. Therefore, we design an adaptive density estimation method, which can automatically estimate the density of the samples. Then, for the group discovery, we propose a heuristic merge-and-split strategy to merge the similar users into the group and split different user groups in the large group. Besides, the group embeddings are set as the learnable neural parameters and can be optimized during the learning process. Moreover, for the group recommendation part, the existing method can not deal with it in the unsupervised setting. And we propose two pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. The pull-and-repulsion pre-text task aims to optimize the group embeddings via separating the different groups and pushing the samples together to the corresponding groups. Besides, for the pseudo group recommendation pre-text task, it generates the pseudo labels for the group recommendation task and guide the network to conduct group recommendation even without the precise annotations. By these designs, our proposed ITR is able to automatically discover the user groups and then provide the precise group recommendation for them. Therefore, it can be applied in the real-time large-scale recommendation system. The details can be found in Section 6.4. We have added these insights at the method Section in the revised paper.

Cluster Number

Thanks for your question. In this paper, the setting of the user/group recommendation is pure unsupervised and the cluster number is not pre-given in both the open dataset and the industrial data. Therefore, during the training process, the cluster number will dynamically change on the open benchmarks since the status of the user embeddings change. Our proposed GIM can automatically discover and optimize the group embedding and the group number. After training, the number of clusters and the status of the group embeddings are restored, and then they will be used to conduct the testing of the group recommendation and the user recommendation.

Besides, on the real-time large-scale data, we first train the recommendation model using the past data. During this process, the number of clusters and the status of the group embeddings are optimized dynamically. After the training stage, they are restored to the database for serving the downstream tasks. Then, when the training data update, the recommendation model will be continue trained and the number of clusters also change dynamically.

Imbalance Problem

Thanks for your question. The imbalance problem does an essential factor in our scenario. We conduct some case studies in our application. Concretely, we select some hot livestream room for the case studies, e.g., the gold-selling livestream room, the grocery-selling livestream room. For the gold-selling livestream room, the imbalance problem exists since the most users in this room are interested in the gold and will form one large cluster, but the other users in this room are interested in other items and form several small clusters. Differently, for the grocery-selling livestream room, the imbalance problem is not very serious, since there are so many small user groups and the large user group may not easily form. For the downstream task, in our scenario, the imbalance groups may not be a bad phenomenon since we can use this property to find the major user group in the livestream room and provide more precise recommendation for them. We have added these claim and discussion at the application Section in the revised paper.

Dear Reviewer dQ8T,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors

Dear Reviewer dQ8T,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors

Response to Reviewer hAA1

Thanks for your valuable and constructive reviews. We appreciate your insights and suggestions, as they will undoubtedly contribute to improving the quality of our paper. In response to your concerns, we provide answers to the questions as follows in order.

Previous Related Methods

Thanks for your concern. Actually, we have surveyed and discussed two of these methods, i.e., COM [1] and MoSAN [2] in our paper. And please note that, in our paper, we claimed that two critical problems limit the applicability of the recent state-of-the-art methods are twofold. Firstly, the promising performance of these methods relies on a pre-defined and fixed number of user groups. Therefore, they can not handle group recommendations without giving the number of user groups, and unfortunately, the number of groups is usually unknown and dynamic in real-time industrial data. Secondly, the supervised training schema of existing GR methods requires extensive human annotations for user-group distribution and group-item interaction, easily leading to significant costs. The previous methods either rely on the pre-defined cluster number or the user-group / group-item annotations. Note that the setting of our experiments is purely unsupervised, and we are not merely solving the occasional group recommendation problem. For MoSAN [3], in Section 3.1 of its original paper, we find that it still needs the ad-hoc group for the training. Therefore, it is not a pure, unsupervised group recommendation method. And for the COM [2], it also user the group assignment information. Besides, for PIT [1], thanks for your suggestion; we missed this paper in the survey process since it seems old. It solves the group recommendation problem via the personal impact topic modeling. And following your suggestion, we will add it to the related work part in the revised paper. For the additional comparison experiments, we think it is hard to produce the results since all these methods are close-resourced. Besides, they are also not new papers and may not achieve very promising performance, especially in the pure, unsupervised learning setting. We have added these claims and explanations in the revised paper.

[1] Exploring personal impact for group recommendation. CIKM 2012.

[2] COM: a generative model for group recommendation. KDD 2014

[3] Interact and Decide: Medley of Sub-Attention Networks for Effective Group Recommendation. SIGIR 2019.

Large Group Size

Thanks for your question. In this paper, we aim to solve the practical problems in the real-time large-scale industrial recommendation system. Therefore, we first design our method and conduct quick experiments on the toy open benchmarks. Then, we conduct extensive experiments on the real-time large-scale data in the application (with about 130 million page views / 50 million user views per day). We admit the scalability of the open benchmarks is limited, but we think it is reasonable for quick trials, and our final aim is to deploy the method in real-world applications. The details regarding the application can be found in Section 6.4.

Similar Observations

Thanks. We believe the similar observations are reasonable since the phenomenon will appear in both the occasional group recommendation setting and the pure unsupervised group recommendation setting. But they are essentially different and unsupervised group recommendation is more challenging than the occasional group recommendation.

HR@10 for CAMRa2011 Dataset

Thanks for your question. For the performance of our proposed method on the CAMRa2011 dataset, we consider it as a corner case since our proposed method can beat all the baselines with different metrics. And we want to give a reasonable explanation here. For the results, we can observe that our method can beat GroupIM with HR@5 but can not beat it with HR@10. And HR@5 is a more precise metric than HR@10 since it requires the model to rank correctly in the top 5 items. Therefore, we suspect that the ranking ability of GroupIM may not be strong and robust since it can achieve very promising performance when ranking in the top 10 items but can not beat our method when ranking in the top 5 items. We have added this explanation in the revised paper.

Dear Reviewer hAA1,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors

Dear Reviewer hAA1,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors

Thanks for your response. I slightly increased the score due to the efforts of addressing all the comments (from other reviewers as well), but it cannot pass my acceptance threshold due to two reasons:

1. 'Mafengwo and CAMRa2011 have very limited number of members in a group.' And I do not see using only those datasets could really convince the effect of this model. I don't see the authors address this comment as well.
2. As far as I understand, ad-hoc groups [1, 2, 3] are groups that form just for one-off events; and it's still comparable with the proposed method.

Thank you.

Thanks for the response.

I did see the Large Group Size section. But I still disagree with the goal is to perform quick experiments on well-known open benchmarks before applying the method to real-time as:

1. [1, 2, 3] and other papers (like CubeRec you pointed out) did test on other group recommendation datasets with various group sizes and statistics reported. I do not think using Mafengo and CAMRa2011 are good enough because the group sizes are really small. CAMRa2011 even has the average group size of around 2. My opinion is we need more datasets with different group sizes to verify the robustness of the group recommendation model.

2. I do not see much details on the industrial datasets, details setup, deployment, pipeline components, etc. What I only see is Appendix 6.4.2 A/B Testing results and "the application contains 130 million page views and 50 million user views per day" in some of the responses. I did not find much details in the original paper (correct me if I'm wrong). Therefore, I cannot check with the 'internal' part and I only can mark the paper based on the 'external' (i.e., Mafengo and CAMRa2011).

I still want to keep my score at this point.

Response to Reviewer GTSo [1/2]

Thanks for your valuable and constructive reviews. We appreciate your insights and suggestions, as they will undoubtedly contribute to improving the quality of our paper. In response to your concerns, we provide answers to the questions as follows in order.

Alignment between Contribution and Results

Thanks for your suggestion. In our view, we think unsupervised learning is a setting that is more challenging compared with supervised learning. Correct me here if you have different opinions. Therefore, the unsupervised user/group recommendation is more challenge compared with the supervised ones (see the experimental results in Figure 1). In this paper, we aim to point out that most of the existing deep group recommendation methods are supervised, but in real-world scenarios, the labels are always missing. Therefore, we need to process the user/group recommendation in an unsupervised manner. In addition, with the unsupervised learning setting, we develop a novel group recommendation method by identify-then-recommend schema. Concretely, the designed group identification module aims to discover the user groups on the user embeddings. And the proposed pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task aim to produce high-quality group embeddings and conduct precise recommendations, respectively. Besides, thanks for your suggestion on the experiment. However, the variant of "known group annotation + GIM + PAR + PGR" may not be reasonable since if we already know the group annotation, we don’t need to conduct the group identification. Therefore, if the group annotation is missing (the settings of this paper), we must use the GIM and then conduct the ablation studies on the PAR and PGR. The GIM is a must-use module, and the effectiveness of GIM can be verified through the performance of the downstream tasks. The experimental results can be found in Table 3 and Table 4. We have added these claims in the revised paper.

Annotation Costs and Dynamic Change

Thanks for your question and suggestion. This paper aims to deploy the proposed method in the real-time large-scale industrial recommendation system. On the open benchmarks, we admit that the annotations of users and groups have already existed and in the experiments, we remove them for the unsupervised experimental setting. We also admit that annotating these toy datasets may not be very expensive. However, the group assignments must change dynamically during training, especially at the early stage. However, note that on real-time large-scale data, the annotation costs a lot, and the distribution will shift dynamically. For example, in our scenario, the application contains 130 million page views and 50 million user views per day. The group assignment and annotation will be changed daily since it is a real-time recommendation. In addition, this is a newly launched application. Therefore, the activities of users will shift drastically, e.g., from new users to old users. We believe it will lead to large annotation costs and distribution shifts, and we aim to develop a pure, unsupervised group identification method for the user/group recommendation. In this scenario, performing several runs of clustering methods (the one that needs a pre-defined number of clusters) is not applicable since the search space will become very large, especially since we don’t know the cluster number for the daily data. The current methods cannot deal with the data without a given number of clusters. For our proposed method, we just need one pass to determine the cluster number, which can fulfill the requirement of the daily data. The complexity of the multiple trials will be T times than our proposed method, where T denotes the time of trials. The details regarding the application can be found in Section 6.4. We have added these explanations in the revised paper.

Details about BPR Loss

Thanks for your question. BPR loss is a commonly used loss function in the recommendation. In our proposed method, we follow ConsRec for the BPR loss. And $\mathcal{L}]\_{\text{U2I}}$ is the same as the $\mathcal{L}\_{\text{user}}$ in the ConsRec. It is formulated as $\mathcal{L}\_{\text{user}} = -\sum\_{u\_s \in \mathcal{U}}{\frac{1}{|\mathcal{D}\_{u\_s}|}} \sum_{(j,j’)\in\mathcal{D}\_{u\_s}}{\text{ln}\sigma(\hat{r}\_{sj}-\hat{r}\_{sj’})}$ , where $\mathcal{D}\_{u\_s}$ is the user-item training set sample for user $u\_s$ and the $(j,j’)$ denotes the user $u\_s$ prefers observed item $i\_j$ over unobserved item $i\_{j’}$. For the sampling of positive and negative sample pairs, we also follow ConsRec, i.e., randomly sampling from missing data as negative instances to pair with each positive instance. For the number of negative samples, ConsRec conduct experiments and analyses in Figure 8 of their original paper. For the fairness, we keep the original setting of the ConsRec. We have added these details in the revised paper.

[1] Wu X, Xiong Y, Zhang Y, et al. Consrec: Learning consensus behind interactions for group recommendation[C]//Proceedings of the ACM Web Conference 2023. 2023: 240-250.

to be continue...

Response to Reviewer GTSo [2/3]

Unique about Current Clustering Method

Thanks for your question. The clustering methods, which need the pre-defined number of cluster centers, such as k-means, spectral clustering, cannot be applied in this scenario. For other clustering methods, which do not require the pre-defined number of cluster centers, such as DBSCAN, is hard to deal with our scenario well since 1) the data distribution of our application change dynamically, therefore it’s hard to determine the parameters regarding to the density, such the radius, and the threshold. But our proposed group identification module can automatically estimate the density and discover the groups via the heuristic merge-and-split strategy, thus can be easily applied in the real-time large-scale unsupervised recommendation system. 2) They can not be integrated into our framework without our proposed pre-text tasks, including the pseudo group recommendation pre-text task and the pull-and-repulsion pre-text task. We have added these claims in the revised paper.

Evaluation Protocols

Thanks for your question. For the evaluation, we follow ConsRec and adopt four metrics to evaluate the ranking capability of the methods, including Hit Rate @ {5, 10} and Normalized Discounted Cumulative Gain @ {5, 10}.

to be continue...

Dear Reviewer GTSo,

Thank you very much for your precious time and valuable comments. We hope our responses have addressed your concerns. Please let us know if you have any further questions. We are happy to discuss them further. Thank you.

In addition, we have attached the revised paper, if necessary, you can check it: https://anonymous.4open.science/r/NeurIPS-7269-ITR-revised-7D83/NeurIPS-7269-ITR-revised.pdf.

Best regards,

Authors

Response to Reviewer GTSo [3/3]

Complexity Analyses

Thanks for your suggestion. Following your suggestion, we conduct complexity analyses of our proposed ITR method. First of all, we define the number of the users, the number of groups, the average size of groups, as $n$, $k’$, $n/k’$, respectively. In the process of the adaptive density estimation, the time complexity and space complexity of calculating radius proposal for one group is $\mathcal{O}(1)$, $\mathcal{O}({k’}^{2})$, respectively, and for all groups is $\mathcal{O}(k’)$, $\mathcal{O}({k’}^{2})$, respectively. Then, the time complexity and space complexity of density estimation for all groups is $k’ \times n/k’$, $n \times k’$, respectively. Subsequently, at the heuristic merge-and-split strategy stage, for the explore step, it takes $\mathcal{O}(k’)$ time complexity, and $\mathcal{O}(1)$ space complexity, respectively. And for the exploit step, it takes $\mathcal{O}(k’ \times n/k’)$ time complexity, and $\mathcal{O}(nk’)$ space complexity, respectively. Besides, for the proposed pseudo recommendation pre-text task, the time complexity and space complexity is $\mathcal{O}(n \times k’)$, and $\mathcal{O}(n \times k’)$, respectively. In addition, for the proposed pull-and-repulsion pre-text task, the time complexity and space complexity is $\mathcal{O}(nk’+{k’}^{2})$, and $\mathcal{O}(nk’)$, respectively. Moreover, for the BPR loss, the time complexity and space complexity is $\mathcal{O}(n)$, and $\mathcal{O}(n)$, respectively. Overall, the time complexity and space complexity of our proposed ITR method is $\mathcal{O}(k’+ k’ \times n/k’+ k’+ k’ \times n/k’+ n \times k’+ nk’+{k’}^{2}+n) \rightarrow \mathcal{O}(nk'+{k’}^{2})$and $\mathcal{O}({k’}^{2}+ n \times k’+1+ nk’+ n \times k’+ nk’+n)\rightarrow \mathcal{O}(nk'+{k’}^{2})$, respectively. Therefore, our proposed method will not bring large memory and time costs since the complexity of our method is linear to the number of users.

Dear Reviewer GTSo,

We highly appreciate your valuable and insightful reviews. We hope the above response has addressed your concerns. If you have any other suggestions or questions, feel free to discuss them. We are very willing to discuss them with you in this period. If your concerns have been addressed, would you please consider raising the score? It is very important for us and this research. Thanks again for your professional comments and valuable time!

Best wishes,

Authors

Thank you for preparing the rebuttal which has mostly addressed my concerns. I decide to raise my rating.

To: AC

Dear AC

Greetings! We highly appreciate your efforts and contributions to NeurIPS 2024 and this submission. The discussion period has begun, but none of the reviewers have started discussing the responses. We are very eager to communicate with reviewers during the discussion period to further enhance the quality of this research, as the NeurIPS is an open, prestigious, and high-caliber conference. Could you help us ask the reviewers to engage in the discussion and whether their concerns are solved or if there are any additional questions? We sincerely thank you for your kind help.

With best wishes,

Authors of Submission 7269

Dear (Senior) Area Chairs,

As the discussion deadline approaches, we kindly request your assistance in reminding the reviewers to provide their post-rebuttal responses. We have made every effort to address their concerns thoroughly.

Currently, Reviewer hAA1 has not yet responded to the authors. The reviewer mentioned, "Please refer to my questions above. I may have more questions during the discussion phase," but we have not received any further feedback. We wonder if our responses have adequately addressed hAA1's concerns. Should Reviewer hAA1 have any additional questions or concerns, we are fully committed to addressing them promptly and thoroughly.

Besides, Reviewers GTSo and dQ8T have acknowledged that our responses have addressed their questions well and are willing to raise the scores (4->5 and 6->7) for the manuscript. Reviewer x7i3 points out some remaining concerns, and we actively provide detailed replies and are now awaiting the experimental results and feedback.

Thank you for your assistance!

Best regards,

Authors of Submission 7269