Understanding and Improving Adversarial Collaborative Filtering for Robust Recommendation

Thanks for your positive feedback. We deeply value your recognition of our work's contributions, particularly its importance, clear explanations, novel theoretical insights, rigorous mathematical proofs, extensive experimental evaluations, and the effective combination of theoretical and empirical analyses. Below, we provide detailed responses to each of your questions, offering clarification and additional support.

---

Q1: To extend theoretical understandings from the simple single-item CF scenario to practical multi-item scenarios, Corollary 1 is restricted to the dot-product-based loss function.

A1: Thank you for your question. The core of our recommendation scenario centers on the dot-product interaction between users and items. While Corollary 1 focuses on this foundational setup, exploring general formulations rather than specific modeling approaches can enhance theoretical applicability.

• To assess the robustness of Corollary 1 beyond dot-product scenarios, we will include experimental results using NeurMF with non-dot-product-based loss functions.

• We have found that PamaCF achieves state-of-the-art performance, even in non-dot-product-based loss functions.

• Recommendation Performance: The table can be seen in the pdf (Table 3) in the global response.

• Robustness against target items promotion: The table can be seen in the pdf (Table 4) in the global response.

---

Q2: The experiment only involves two CF backbones, namely MF and LightGCN. The results will be more convincing by adding more CF methods.

A2: Thank you for your suggestion. We appreciate your feedback, and in response, we have expanded our experimental evaluation to include results from NeurMF, as shown in the tables referenced in Q1. We have found that Our findings demonstrate that PamaCF achieves state-of-the-art performance within the NeurMF. This addition broadens the scope of collaborative filtering methods examined in our study.

---

We sincerely appreciate your time, effort, and valuable suggestions during the review process. We trust that these clarifications adequately address your concerns.

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have sufficiently addressed your concerns. Please let us know if there are any further questions or issues that you would like us to address. We look forward to hearing from you soon.

Thank you once again for your time and effort.

Sincerely,
The Authors

We sincerely appreciate your positive comments and recognition of the significance of our paper, including its theoretical contributions, novel methodology, and comprehensive experiments. In response to your feedback, we address your concerns and clarify some misunderstandings as follows:

---

Q1: The strategy for initializing embeddings is seldom adopted in existing RS.

A1: Thanks for your feedback. The Gaussian-based initialization method is chosen to derive upper and lower bounds and does not alter our core findings: (1) ACF outperforms traditional CF; (2) User-adaptive magnitude further enhances the effectiveness of ACF.

• For instance, the conclusion in Theorem 1 is already demonstrated in Appendix D1.1 at line 667, but incorporating the properties of Gaussian distribution allows for a more accurate formulation.
• In Appendix D2.1 at line 706, the expression already indicates the need for different magnitudes; introducing Gaussian distribution properties results in more accurate and readable upper and lower bounds.

---

Q2: The optimization in practical RS typically uses BCE, BPR, or softmax loss, rather than the loss function described in Eq. 2.

A2: Section 3's simplified pointwise loss ensures derivation clarity and simplicity, as in [1].

• To broaden the applicability of our findings, Corollary 1 in Section 4 extends this to complex losses (Appendix D.3).
• Moreover, in the implementation of PamaCF (Eq. 6 in the main text), we adopt the extended and widely used pairwise loss, i.e., PamaCF-BPR loss, detailed in Appendix B.

---

Q3: Preference prediction usually does not employ the sign function.

A3: The sign function aids recommendation error definition without affecting our derivations. In Appendices D.1, D.2, and D.3, we demonstrate that $\mathbb{P}[\mathrm{sgn}(\langle u, v \rangle) \neq r] = \mathbb{P}[r \cdot \langle u, v \rangle < 0]$, effectively addressing any potential impact of the sign function on our results.

---

Q4: Recommendation is often framed as a ranking problem, rather than being defined as in 'Definition 2' for analyzing recommendation error.

A4: Thanks for your question. Our approach in Section 3 simplifies analysis by focusing on item impact via a pointwise framework. This approach provides theoretical clarity by avoiding the complexities introduced by ranking tasks. Section 4's Corollary 1 extends these insights beyond the constraints of Definitions 1-3, as detailed in Appendix D.3.

---

Q5: It is counter-intuitive that the theoretical bound does not depend on the item embeddings.

A5: Theoretical bounds indeed depends on the item embeddings.

• When fixing a particular user, the item embeddings $v_{(0)}$ is equivalent to sampling from $\mathcal{N}(\bar{u}, \frac{\sigma^2}{n-1}I)$ (as mentioned in lines 120-123).
• When deriving upper and lower bounds, we map $v_{(t)}$ to $v_{(0)}$ and then take expectations, which is why $v_{(t)}$ does not explicitly appear in the expressions. For details, refer to lines 710 or 733 in Appendix D.2.

---

Q6: Are the relations $v = \frac{1}{n} \sum_i r_i u_i$ always held?

A6: No, this relation holds at epoch = 0. However, with a given learning rate $\eta$, our derivation shows that $v_{(t)} = M(t, \eta) \frac{1}{n} \sum r u_{(0)}$, where $M(t, \eta)$ is a mapping function detailed in Proposition 1 (lines 625-630).

---

Q7: It is interesting to discover that CF has a unique advantage such that the adversarial mechanism works even for clean data, unlike in basic classification tasks. Why is collaborative filtering so special?

A7: Thanks for acknowledging our work. The key distinction may lie in the nature of the tasks, particularly in the parameter search space. Consider the following simplification (one-layer classifier and MF):

• Traditional classification tasks are typically formulated as follows:

  • Given a set of samples $(x_i, y_i)_{i=0}^n$, where $x \in \mathbb{R}^{d_1}$ and $y \in \mathbb{R}^{d_2}$, the goal is to train a classifier $f(x_i) = w^Tx_i + b$.
  • The trainable component is the classifier $w \in \mathbb{R}^{d_1 \times d_2}$, and adversarial perturbations compel $w$ to locate a decision boundary within $\mathbb{R}^{d_1 \times d_2}$ that satisfies adversarial losses.

• For recommender systems, where $f(u_i, v_i) = u_i^T v_i$, involving $M$ users and $N$ items:

  • The application of adversarial perturbations allows for a broader parameter search space, specifically $MN \times \mathbb{R}^d \times \mathbb{R}^d$.

Traditional classification tasks require $w$ to satisfy all instances and their adversarial counterparts, whereas in recommendation tasks, adjustments to user and item representations can satisfy each other's adversarial counterparts.

---

Q8: I recommend that the authors provide intuitive explanations for the theorems and a brief outline of the proof procedure.

A8: Thank you for your suggestion. We will enhance the clarity by providing intuitive explanations and outlining the proofs. Due to space constraints on the responses, we will further supplement these in subsequent versions.

---

Q9: The paper should include more experimental details.

A9: Thanks for your suggestion. To address the need for additional experimental details, we have included them in Appendix C.1:

• Detailed discussions on attack and defense setups are provided from lines 589 to 600.
• As you mentioned, our backbone model follows LightGCN. We will emphasize this aspect more prominently in future revisions.

---

We appreciate your thorough feedback and recognition of the significance of our research. We believe these clarifications adequately address your concerns and look forward to your reconsideration and reevaluation of our work.

[1] Fight Fire with Fire: Towards Robust Recommender Systems via Adversarial Poisoning Training. SIGIR'21

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have sufficiently addressed your concerns. Please let us know if there are any further questions or issues that you would like us to address. We look forward to hearing from you soon.

Thank you once again for your time and effort.

Sincerely,
The Authors

Thanks for your detailed feedback and for acknowledging the readability and theoretical clarity of our paper. Below, we address each of your questions to clarify misunderstandings and address concerns. We hope this clarifies the contributions of our work and look forward your reconsideration.

---

W1: Part of the experiments are missing and incomplete.

W-A1: Due to space constraints of main text, we focused on presenting results that strongly support our claims. Other reviewers also praised the thoroughness and persuasiveness of our experiments. To address your concerns comprehensively, detailed explanations and additional support will be provided in A1-A3, corresponding to Q1-Q3.

---

Q1: What is the performance of different ACF methods on LightGCN?

A1: Thanks for your suggestion. In response, experimental results for LightGCN will be included. Our findings show improved recommendation performance for PamaCF integrated with LightGCN. Refer to Table 1 in the PDF in global response for details.

---

Q2: Why does the paper use HR@50 and NDCG@50 in Table 1 to present recommendation performance, while Table 2 uses T-HR@50 and T-NDCG@50 to evaluate performance against attacks?

A2: Thanks for your question. In Table 1, we actually used HR@20 and NDCG@20, not HR@50 and NDCG@50. Your query addresses two points: (1) the choice of metrics and (2) the difference in k values. Here are our responses:

1. HR@k and NDCG@k evaluate recommendation performance, while T-HR@k and T-NDCG@k assess system robustness against attacks. We provide detailed explanations in the Evaluation Metrics section (lines 256-269).
2. Our selection of k values follows standard practices in the field, detailed in the Implementation Details section (lines 589-592). This approach ensures consistency with research that distinguishes between metrics for recommendation and defense strategies [1], underscoring the reliability of our defense methods.
   • HR@20 and NDCG@20 focus on the top 20 recommendations, typical in collaborative filtering assessments [2].
   • T-HR@50 and T-NDCG@50 evaluate defense effectiveness against attacks, commonly using a top 50 ranking [3][4].

---

Q3: Is preventing target item promotions the only attack PamaCF can mitigate?

A3: PamaCF is capable of defending against various poisoning attacks. These attacks typically aim to promote specific items or degrade system performance. Item promotion attacks are well-documented [3][4], while performance degradation attacks involve harmful item embeddings mainly in federated recommender systems [5].

• Theoretical analysis (Section 3) of ACF considers the impact of poisoning attacks on recommendation errors, without restricting attack objectives. Theoretically, PamaCF can mitigate poisoning attacks aimed at different objectives.
• Empirical validation shows PamaCF not only counters item promotion attacks but also enhances overall recommendation performance.
  • To further validate PamaCF's robustness, we simulate performance degradation attacks with random user behaviors, confirming its effectiveness.
  • PamaCF demonstrates strong defense capabilities even against performance degradation attacks.
  • Refer to Table 2 in the PDF in global response for details.

---

Q4 (W2): The loss function of Gaussian Recsys in Eq. 2 does not fully correspond to [6]. So is the same regarding Eq. 3 and [7]

A4: The loss function used in Gaussian Recsys does correspond to [6]. [6] covers various loss functions, addressing any confusion about the specific one referenced. We refer to the loss function detailed in [6]'s "DETAILED PROOFS" section. The mention of "introducing the adversarial loss [7]" (line 112) in our paper refers to adopting the adversarial training paradigm (as described in Eq. 1 in the Preliminary section), where adversarial perturbations are added to the original loss (Eq. 2), rather than using the APR loss function from [7].

• For instance, the adversarial component discussed in [6] involves $\Delta_{v} = \arg\min_{\Delta, \Vert \Delta \Vert \le \epsilon} \langle ru, v + \Delta_{v} \rangle$, where the original loss is $\mathcal{L}(\Theta+\Delta) = \langle r(u+ \Delta_{u}), v + \Delta_{v} \rangle$. This aligns precisely with our employed loss function in Section 3.

---

Q5: All previous works are based on pairwise loss, whereas the paper is based on pointwise loss.

A5: It's worth noting that many methods also adopt pointwise loss [8][9]. Besides, in Section 3, our theoretical results leverage pointwise loss for clarity and simplicity. Section 4's Corollary 1 extends the discussed pointwise loss to encompass more complex functions, detailed in Appendix D.3.

---

Thanks sincerely for your thorough review of our paper. We have addressed each of your concerns and clarified misunderstandings. Based on these clarifications, we kindly request your reconsideration of the overall assessment of this paper. We firmly believe that our work contributes valuable insights and advances the state-of-the-art in robust recommender systems, making a significant contribution to the research community.

[1] LoRec: Combating Poisons with Large Language Model for Robust Sequential Recommendation. SIGIR'24
[2] LightGCN - Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR'20
[3] Revisiting Injective Attacks on Recommender Systems. NeurIPS'22
[4] Revisiting Adversarially Learned Injection Attacks Against Recommender Systems. RecSys'20
[5] UA-FedRec: untargeted attack on federated news recommendation. SIGKDD'23
[6] Fight Fire with Fire: Towards Robust Recommender Systems via Adversarial Poisoning Training. SIGIR'21
[7] Adversarial Personalized Ranking for Recommendation. SIGIR'18
[8] Denoising Implicit Feedback for Recommendation. WSDM'21
[9] BSL: Understanding and improving softmax loss for recommendation. ICDE'24

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have sufficiently addressed your concerns. Please let us know if there are any further questions or issues that you would like us to address. We look forward to hearing from you soon.

Thank you once again for your time and effort.

Sincerely,
The Authors

Thank you once again for dedicating your time and effort to reviewing our paper. We are following up on our message sent yesterday regarding the discussion period between authors and reviewers, which concludes on August 13.

We understand that you may have a busy schedule and truly appreciate the time and effort you have already invested in reviewing our work. We have carefully considered your comments and provided detailed clarifications and support for each concern in our rebuttal.

Specifically:

• For W1: Other reviewers (gsZx, 6kCb, MDNE) also praised the thoroughness and persuasiveness of our experiments. We understand your concerns and have provided detailed explanations and additional support in A1-A3, corresponding to Q1-Q3.
• For W2: We clarified that the loss terms in Section 3 indeed align with the cited papers, as discussed in A4 corresponding to Q4.

We are eager to learn whether our response has sufficiently addressed your concerns and if there are any further questions or suggestions. Please accept our apologies for reaching out again so soon; however, we are committed to meeting the deadline and ensuring the timely progression of our paper through the discussion process. If you require any additional information or clarification, please do not hesitate to reach out.

Once again, we express our gratitude for your feedback and your continued assistance in this process.

Sincerely,
The Authors

Thanks for your positive feedback. We sincerely appreciate your recognition of the importance, novelty, reasonable motivation, sound theoretical results, and the solid evaluations presented in our work. Below, we address each of your questions to resolve your concerns:

---

Q1: Can authors discuss the feasibility of relaxing the assumption in theorems such as the Gaussian recommender system? In addition, is the extension to multiple items natural?

A1: We appreciate your question. The assumption in the Gaussian recommender system serves primarily for clarity and simplicity of theoretical results, as also adapted in [1]. To relax this assumption, one approach involves considering more complex loss functions or scenarios involving interactions with multiple items.

• We elaborate on these scenarios in Corollary 1, with detailed derivations available in Appendix D.3.
• Specifically, for any dot-product-based loss function involving multiple items, we ascertain the range of perturbation magnitude for each user.
• While the formulations in Appendix D.3 are complex, they are crucial for identifying the relationship between perturbation magnitudes and user embeddings.

---

Q2: It will be better if the magnitude of user latent vectors is provided in Figure 1. Thus we can align the empirical findings and theoretical findings more straightforwardly.

A2: Thank you for this excellent suggestion. We will incorporate the magnitude of user latent vectors into Figure 1. This addition will enhance the alignment between empirical findings and theoretical insights. The table below presents $\Vert u \Vert^2$ for the five users in Figure 1, which will be included in the revised paper. The revised image can be seen in the pdf (Figure 1) in the global response.

---

Q3: How can we choose $\rho$ in practice? Is $\rho$ learnable or is it only a hyper-parameter?

A3: $\rho$ is considered a hyper-parameter used to adjust the overall magnitude in practical applications. In Experiment 5.4, we extensively studied the impact of $\rho$ on method performance, revealing that:

• Even with a small $\rho$, noticeable improvements are observed.
• Optimal values for $\rho$ typically fall within the range of 0.1 to 1.0 across diverse datasets. In practice, we typically search for $\rho$ within this range at intervals of 0.1.

---

Q4: Is there a more reasonable way to design the form of $c(u,t)$ and equation 5?

A4: Our paper introduces a specific design pattern for $c(u,t)$ that aligns with the criteria outlined in Corollary 1. The term $\rho \cdot c(u,t)$ allows for flexible adjustment of overall magnitude. Moving forward, we aim to explore more intricate design patterns aimed at better leveraging the properties identified in Corollary 1. For instance, we will investigate alternative mapping functions to replace the sigmoid function, which may better capture the relationships outlined in Corollary 1. These efforts are aimed at enhancing the effectiveness and adaptability of our approach in practical applications.

---

Q5: Some presentation issues.

A5: Thank you for your valuable suggestions aimed at enhancing the presentation of our paper. We will take the following actions to address these concerns:

• Ensure accurate representation by verifying the correct usage of bold symbols.
• Improve readability by reducing the number of digits in the tables.
• Enhance visibility by increasing the text size in Figures 2 and 3.

---

We appreciate your detailed comments and constructive feedback, as well as your recognition of the importance of our work. We believe these clarifications will effectively address your concerns. We eagerly await your reconsideration and reevaluation of our work.

[1] Fight Fire with Fire: Towards Robust Recommender Systems via Adversarial Poisoning Training. SIGIR'21

Thank you for taking the time and effort to review our paper. We have carefully considered your comments and responded to each point with detailed clarifications and supporting evidence. As the deadline for the discussion period between authors and reviewers is approaching, we kindly ask whether our responses have sufficiently addressed your concerns. Please let us know if there are any further questions or issues that you would like us to address. We look forward to hearing from you soon.

Thank you once again for your time and effort.

Sincerely,
The Authors