Federated Graph Learning for Cross-Domain Recommendation

We would like to sincerely thank you for positive evaluations and valuable comments for improvement.

W1: The use of the terms "extended" and "expansion" appears to be inconsistent.

Response: We agree and we will use expand/expansion uniformly in paper’s updated version.

W2: The generalization performance of the framework has not been sufficiently validated.

Response: We agree that it is important to verify the generalization performance of the model on other datasets. Thus, we conduct new experiments on Douban dataset to validate the generalization capability of the proposed method. As shown in Table u2 in Global Rebuttal pdf file, our method yields both optimal and sub-optimal results on almost all metrics. We will add these new results in the updated version. For details, please refer to the Global Rebuttal with pdf attachment.

Q1: What are the privacy innovations in this paper?

Response: We provide a full range of privacy both intra-domain and inter-domain. In our work, we include two horizontal federation phases and DP (line 177, Privacy-preserving knowledge extraction) to fully guarantee intra-domain privacy and inter-domain privacy. In previous work, those focusing on firm-level privacy tend to protect inter-domain privacy through privacy techniques (DP, projection methods, etc.); those focusing on user-level privacy tend to consider only intra-domain privacy (embeddings from different domains are passed in plaintext). In the Appendix of paper's previous version, we had provided a theoretical proof of privacy preservation (Appendix A) and experimentally verify the model performance under different privacy budgets (Appendix D.3). Our approach considers the full range of privacy protection, which distinguishes us from previous approaches, and outperforms all baselines.

We sincerely thank you for your valuable comments and suggestions. We hope our response addresses your concerns.

W1: Why addresses the broader-domain CDR involving more than three domains.

Response: Because solving the cross-domain recommendation problem with more than three participating domains is very important for better mining user preferences. In existing work in the area of cross-domain recommendation, researchers tend to focus on only two domains. These works often do not generalize well to multi-domain tasks. There are also some studies that begin to focus on multi-domain knowledge transfer, but they are also limited to only three domains [1]. We believe that these studies do not fully reflect the nature of multi-domain, while paying insufficient attention to privacy and negative transfer challenges. These two challenges become more severe with the number of domains, in line…. We therefore emphasize a number of domains greater than three in the article to distinguish our method from previous works. As shown in Table 1 and Table 2 of paper’s previous version, our methods (FedGCDR, FedGCDR-DP) outperform all baselines under the settings of 4 domains, 8 domains and 16 domains.

[1] Liu W, Chen C, Liao X, et al. Federated Probabilistic Preference Distribution Modelling with Compactness Co-Clustering for Privacy-Preserving Multi-Domain Recommendation[C]//IJCAI. 2023: 2206-2214.

W2: Why choose the special social term in Soreg.

Response: We use the social term in Equation 9 because it better exploits the user's interests. In Soreg, the authors propose two innovative social terms, as follows:

$(1)\sum_{i=1}^m || U_i -\frac{\sum_{f \in \mathcal{F}^+(i) } Sim(i,f) \times U_f}{\sum_{f \in \mathcal{F}^+(i) } Sim(i,f)}||^2_F$

$(2)\sum^m_{i=1} \sum_{f \in \mathcal{F}^+(i)}Sim(i,f)||U_i-U_f||^2_F$

where, according to Soreg, the second formula: “is insensitive to users with different tastes” [2]. For the same user's behavior on different domains, we consider that it shows different dimensions of user interests, which is why we combine as many domains as possible to fully explore user interests. So the same user may exhibit very different tastes on different domains, so we take the more appropriate first social term.

[2] Ma H, Zhou D, Liu C, et al. Recommender systems with social regularization[C]//Proceedings of the fourth ACM international conference on Web search and data mining. 2011: 287-296.

Q1: Whether the proposed framework can be extended to more realistic company-level applications.

Response: Yes, our approach fits real-world scenarios and can be modified to better fit company-level scenarios. In real applications, there are many similar setups, e.g., between different recommendation scenarios on online platforms, where their users are partially overlapping and the items are often different. Our assumptions are consistent with the real scenarios, while some modifications to our approach are needed to accommodate company-level applications. In specific, it is necessary to change the intra-domain GAT training and fine-tuning phase of the horizontal FL setting to a centralized training process undertaken by a domain server (e.g., a company). Knowledge transfer between domains (company to company) is still well protected with DP.

Q2: Why cannot we adopt stop-transfer method to avoid such complex attention generation progress?

Response: Because it's hard to determine whether to stop the transfer when there are multiple source domains. For simpler cross-domain or transfer learning setting, the approach you propose is also really a way to avoid negative transfer.

However, during CDR training, especially in our hypothetical BD-CDR scenario (more than three domains with multiple source domains and one target domain) setup, it would be difficult to determine which source domain or domains lead to negative transfer. Also we need to consider that the time a domain is joined affects whether it produces negative transfer or not. For example, assuming that the knowledge transfer from domain A to domain B is positive, while the re-transfer of knowledge from domain A to domain B, which aggregates the knowledge of domains C and D, is negative. Positive transfer indicates that there is some information or pattern in domain A that is beneficial to domain B. The later negative transfer suggests that after combining the knowledge of domains C and D, the negative knowledge in domain A dominates the transfer process. This shows that it is very arbitrary to rely only on the positive and negative transfer to judge whether to transfer or not, and the correct approach should be for the target domain to actively filter the negative information, so as to fully utilize the knowledge of each domain. Our method filters out potential harmful or conflicting knowledge from source domains, and mitigates the issue of negative transfer.

We sincerely thank you for your valuable comments and suggestions. We hope our response addresses your concerns.

W1&Q1: Writing errors.

Response: Thank you. We agree and will correct any errors you have raised and scrutinize the paper and fix any typos during the revision process.

W2: How well does the model generalize to other datasets or real-world scenarios with different characteristics?

Response: We agree that it is important to verify the generalization performance of the model on other datasets. Thus, we conduct new experiments on Douban dataset to validate the generalization capability of the proposed method. As shown in Table u2 in Global Rebuttal pdf file, our method yields both optimal and sub-optimal results on almost all metrics. We will add these new results in the updated version. For details, please refer to the Global Rebuttal with pdf attachment.

W3&Q3: The experimental section is completely devoid of any discussion on privacy preservation.

Response: In paper’s previous version, we did not detail privacy protection related content in the experimental section due to space constraints, instead we added this part in the Appendix of paper’s previous version. In accordance with previous work on federated recommendation, we have made full analysis and discussion on privacy protection in both theoretical and experimental aspects. For details, please refer to Appendix A Privacy analysis for theoretical proof and Appendix D.3 Privacy budget for the experiments on privacy budget in paper’s previous version.

Q2: How do non-overlapping users perform graph expansion in the target domain?

Response: Non-overlapping users perform graph expansion by a set of Gaussian noise matrices. In lines 197 through 198 of paper’s previous version, we had explained the case where there are no corresponding users in one of the source domains in the graph expansion strategy: “It is worth noting that for source domains where a user has no rating, $X_{S_i}$ is a Gaussian noise matrix.” Therefore, for non-overlapping users in the target domain, we conduct the graph expansion with $M$ Gaussian noise matrices (assuming $M$ source domains).

Q4: Does FedGCDR-DP in Tables 1 and 2 refer to another variant method? What is the difference with the proposed FedGCDR?

Response: Yes, FedGCDR and FedCDR-DP are two variants of our proposed method, where FedGCDR does not add Gaussian noise to the knowledge transfer module, while FedGCDR-DP is a complete implementation of the method integrating Gaussian noise. By introducing these two variants it is possible to compare the loss of accuracy due to the addition of inter-domain privacy protection. Thanks for this valuable comment, and we will add these descriptions to the experimental section in paper’s updated version.

Q5: Domain selection strategy.

Response: The basis of our domain selection strategy is the amount of data before performing data filtering. Thus, we sorted the domains contained in the Amazon dataset based on the amount of data in descending order and selected the top 16 domains. Similarly, Amazon-4 and Amazon-8 were selected accordingly. The only exception is that we prioritized the Movie domain, which has a relatively small amount of source data, based on popularity. We will add this content to the appendix in paper’s updated version.

We thank the reviewer for the constructive review and valuable questions that have helped us improve our work.

W1: The definitions provided for the intra-domain privacy and inter-domain privacy appear to be inconsistent.

Response: We agree and we will give a unified and complete definition in the introduction section of paper’s updated version.

W2: Why emphasize the concept of ‘Broader-Domain CDR’.

Responese: Because solving the cross-domain recommendation problem with more than three participating domains is very important for better mining user preferences. In existing work in the area of cross-domain recommendation, researchers tend to focus on only two domains. These works often do not generalize well to multi-domain tasks. There are also some studies that begin to focus on multi-domain knowledge transfer, but they are also limited to only three domains [1]. We believe that these studies do not fully reflect the nature of multi-domain, while paying insufficient attention to privacy and negative transfer challenges. These two challenges become more severe with the number of domains (lines 44 through 55). We therefore emphasize a number of domains greater than three in the article to distinguish our method from previous works. As shown in Table 2 of paper’s previous version, our methods (FedGCDR, FedGCDR-DP) outperform all baselines under the settings of 4 domains, 8 domains and 16 domains.

[1] Liu W, Chen C, Liao X, et al. Federated Probabilistic Preference Distribution Modelling with Compactness Co-Clustering for Privacy-Preserving Multi-Domain Recommendation[C]//IJCAI. 2023: 2206-2214.

Q1: Heterogeneity of embedding dimension exists between different domains.

Response: For solving the problem of heterogeneity of embedding dimensions, we have two possible approaches. 1. Aligning the attention layers of different domains. Since our method does not directly transmit the original embedding, each domain can choose the appropriate embedding dimension according to its own characteristics. Intermediate embeddings of the same dimension can be obtained by aligning the weight matrices of the attention layers, thus realizing knowledge transfer. 2. Aligning dimensions by mapping functions. Before knowledge transfer, we will first map the knowledge vectors of the current domain to the feature space of the target domain through MLP. The embedding dimensions of the source and target domains can be aligned by changing the input dimension of the MLP.