Data-Free Model Extraction for Black-box Recommender Systems via Graph Convolutions

Thank you for your review and for acknowledging the novelty and relevance of our work. Below, we provide detailed responses to clarify our method and address your questions:

Q1: What is meant by "member data rate" in what's a "data free" setting? I would have thought only queries were impossible and that data is not available, though I assume this is a misunderstanding on my part
A1: Member data, i.e., the data used by companies to train their recommender models, is typically inaccessible, as it is considered proprietary and commercially sensitive. The "data-free" setting refers to the assumption that attackers do not have access to such internal training data. Correspondingly, we haven't use the any member data in our method.

Q2: The member data and query rates seem not realistic from an attack perspective. Can you describe a realistic attack setting in which the rates presented here are reasonable? How does the method work for very small rates?
A2: Thank you for raising this concern. We provide the following clarifications:
[Clarification on member data and query rates] We believe this question refers to the investigation study section. In that part, we perform a systematic analysis to examine how varying the member data rate (when accessible) and the query budget affect the performance of model extraction. At the data level, this helps to understand the dependency of extraction effectiveness on information availability; at the model level, we also study how different model architectures respond to these factors.
[which the reasonable rates] Accessing to member data is generally unrealistic in real-world attack settings. That’s why our proposed method focuses on the data-free scenario. The member data/query rate is used purely for comparative and analytical purposes.

Q3: The datasets look fairly small. What is the scalability profile of the method on larger datasets?
A3: Thank you for the suggestion. We have added a large datasets Steam (with 334730 users and 13047 items) to our experiments. The results further demonstrate the effectiveness and robustness of our proposed method.

[1] Wang Y, Su J, Chen C, et al. Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(12): 12766-1277.
[2] Yue Z, He Z, Zeng H, McAuley J. "Black-box attacks on sequential recommenders via data-free model extraction." Proceedings of the 15th ACM conference on recommender systems. 2021.4.

Thank you very much for your thoughtful and constructive feedback. We appreciate your recognition of our method's motivation and technical design. We have carefully responsed to each of them in point-by-point.

Q1: More investigation study experiments:
A1: Thank you for your suggestion. We have reached this conclusion after extensive experiments and combined with theory. Specifically:
[Empirical Evidence] We have conducted comprehensive cross-model extraction experiments on BPR, NCF, GCMC, NGCF, and LightGCN across several datasets. Due to the char limitation, wo provide the PTD results with 50% member data rate on ML-100K. The results clearly show that graph convolution-based models demonstrate superior performance.
[Theoretical Analysis] As discussed on Page 4, Lines 150–156, BPR, NCF, and LightGCN represent a progressively enhanced modeling hierarchy. BPR is a classical matrix factorization model; NCF builds on it by incorporating MLPs to capture nonlinear user-item interactions; and LightGCN further introduces graph convolutional layers, enabling the model to better exploit high-order collaborative signals. This progression not only reflects increasing modeling capacity but also empirically supports the superiority of graph convolution-based recommender models in model extraction tasks.

Q2: Why top-k recommendation list:
A2: For this important point, our setting is both theoretically grounded and practically feasible:
[Theoretical basis] The top-k feedback setting is consistent with existing model extraction studies [1,2].
[Pratical case analysis] In real-world recommendation platforms, such as social recommendation systems (e.g., X) or multimedia platforms like YouTube, users can easily access hundreds or even thousands of recommendation items, whether by entering a search query or browsing the homepage. Therefore, obtaining top-k recommendation lists is both practical and realistic in real deployment scenarios.

Q3: How is the query budget selected:
A3: As this work is the first to study data-free black-box model extraction in recommender systems that rely solely on user and item identifiers as input features, we adopt a stringent and practical definition: each new user history interaction list (not each new user) submitted to the victim model is counted as a separate query. Correspondingly, we empirically select a query budget that balances extraction performance and budgets. Detailed analysis is provided in our Sec 5.3.

Q4: Minor: Yelp in Line 321 should be ML-1M?
A4: Thanks you for pointing it. It is a typo.

Q5: More experiments on other datasets.
A5:Thank you for the suggestion.
[Experiment results] Due to page limits, we presents part of representative results on additional datasets. It is easy to observed that the conclusions of the ablation studies and sensitivity analyses on the remaining datasets are consistent with those reported in the main paper.
[Revision plan] Considering the paper length and readability during submission, we included results on a subset of datasets in the main text. We will incorporate the necessary experimental results for the remaining datasets into the final Appendix to to provide a more comprehensive presentation.

[Ablation Study]

[Budget Analysis] Yelp:

Gowalla:

Q6: Comparison with Sim4Rec(AAAI2025)
A6: Thank you for the suggestion. We have conducted a comparison with Sim4Rec. The results further demonstrate the effectiveness of DBGRME. Although the Sim4Rec (published in Apirl 11th, 2025) belongs to the official NeurIPS (March 1st, 2025) defined “Contemporaneous Work”, we consider its topical relev7ance and plan to include the comparison results in the final Appendix for completeness.

[1] Yue Z, He Z, Zeng H, McAuley J. "Black-box attacks on sequential recommenders via data-free model extraction." Proceedings of the 15th ACM conference on recommender systems. 2021.4.
[2] Wang Y, Su J, Chen C, et al. Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation. Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(12): 12766-1277.

Thank you for your thoughtful review and constructive suggestions. We appreciate your recognition of our method's motivation. We have carefully addressed each of them in our point-by-point response below.

Q1: More comparison works:
A1: [Paper Review] We analyzed the three suggested papers from multiple perspectives, including task type and research domain, as shown in the table below. Among them, Sim4Rec is the most recent and closely related to our work; therefore, we included it in our comparative experiments.

[Comparison experiments] Based on the comparison results, it can be observed that Sim4Rec performs between DFME and DBGRME, further highlighting the superiority of DBGRME. Moreover, although the Sim4Rec (published in Apirl 11th, 2025) belongs to the official NeurIPS (March 1st, 2025) defined “Contemporaneous Work”, we consider its topical relevance and plan to include the comparison results in the final Appendix for completeness.

Q2: Query budget setting comparison:
A2:Thank you for your question. Compared to existing works, our query budget is actually relatively modest. Specifically:
[Budget definition] Firstly, we would like to clarify that there exists a difference between the query budget definitions in prior work (e.g., [1,4]) and ours (“per user/history interaction query”). Specifically, prior works typically count each user as one query, regardless of the update of interaction lists (refer to the description in Sec 4.1.1 of [4]). In contrast, we adopt a stricter definition: each new interaction list, even from the same user, is counted as a separate query. This leads to a more conservative and realistic budget estimation.
[Comparison with existing work] Under our strict definition, recommendation model extraction methods such as [4] incur a budget of up to 1M on ML-1M (refer to the description in Sec 4.1.1 and Sec 5.2.1 of [4], i.e., 5000 budgets × 200 sequence length), which significantly exceeds our defined budget of 400k.

Q3: Other common datasets
A3: Thanks for your advice. To further strengthen the evaluation, we compared various public datasets by interaction scale and sparsity. Based on this comparison and time permitting, we conduct experiments on the Steam dataset, which contains significantly more users and interactions. And the results of another dataset will be post in the discussion stage. We will include the corresponding results in the final Appendix.

Q4: Discussions about possible defense:
A4: Thank you for the suggestion. Due to space constraints, we briefly discussed potential defense strategies on Page 9, Lines 360–364. Following your valuable feedback, we will expand this discussion from three perspectives (detection, model-level defense, and legal & ethical considerations), and include a more comprehensive analysis in the Appendix.

Here, we discuss the future extension of potential countermeasures from three perspectives: detection-based defense, model-level defense, and legal & ethical considerations.
[Detection-based defense] Recent attacks [1,4] often rely on autoregressive querying or generator-driven interaction synthesis. Monitoring query frequency, interaction entropy, and behavioral anomalies can help detect suspicious users. Platforms may then apply query rate limiting or introduce noise to hinder extraction.
[Model-level defense] This approach involves injecting perturbations into the model's representation or output space to mislead attackers. However, in hard-label recommendation settings, such defenses require careful calibration to maintain model accuracy while degrading extraction effectiveness.
[Legal & ethical considerations] In addition to technical defenses, platform operators should implement responsible deployment practices such as strict API access control, policy enforcement, and legal safeguards. Raising public awareness and establishing regulatory frameworks can further mitigate the misuse of recommendation models.
Moreover, this study is conducted on benchmark datasets with the aim of improving system robustness and promoting awareness of potential vulnerabilities.

[1] Wang Y, Su J, Chen C, et al. Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(12): 12766-12774.
[2] Wang C, Sun J, Dong Z, et al. Data-free knowledge distillation for reusing recommendation models[C]//Proceedings of the 17th ACM Conference on Recommender Systems. 2023: 386-395.
[3] Lin Z, Xu K, Fang C, et al. Quda: Query-limited data-free model extraction[C]//Proceedings of the 2023 ACM Asia Conference on Computer and Communications Security. 2023: 913-924.
[4] Yue Z, He Z, Zeng H, McAuley J. "Black-box attacks on sequential recommenders via data-free model extraction." Proceedings of the 15th ACM conference on recommender systems. 2021.4.

Thank you for your continued feedback. We would like to respond to your comments point by point as follows.

Q1: Could you provide the query budget compared to the existing works based on the strict definition and the previous definition?
A1: [Budget Definition and Calculation]

• Previous Definition: num_users
• Our Strict Definition: num_query * num_users * num_iters

Among them, num_users refers to the number of synthesized users, num_query denotes the number of queries per iteration, and num_iters indicates the total number of iterations involving querying. Hence, the comparison budget results (bracelets denoting attack success rate Agr@50) are:

Unlike the previous definition, which treats each query as a single user, our strict definition considers every user in the interaction list as an independent query, regardless of whether the queries originate from the same user.

Hence, we have key observations:

1. Under the previous definition, all methods are compared at a similar budget level (1k/5k). Even in this case, our DBGRME significantly outperforms others (0.6047 vs. 0.2845).
2. Under our strict definition, with same query budgets for Sim4Rec and our proposed DBGRME-200k, they both have 200k budgets, but our methods exceed it significantly.

Q2: Why is the performance improvement the most significant on LGN?
A2: We believe the underlying reason lies in the inherent vulnerability of the LGN architecture. The LGN model relies solely on basic graph convolution, without incorporating non-linear transformations or attention mechanisms. Compared to other architectures, it is shallower and more linear, making its recommendation behavior more transparent and easier to approximate. As a result, attackers can more effectively infer its internal mechanisms and parameter patterns from limited feedback. These factors contribute to the most significant performance improvement observed on LGN.

Moreover, we are truly grateful for your valuable and constructive feedback, which has significantly contributed to enhancing our work. Thank you again for your positive and engaged response. We sincerely hope that you will consider revisiting your score in light of our response and clarifications.

We sincerely thank you for your positive and encouraging feedback on our work. We truly appreciate your recognition of our scenario design, methodology, and experimental setting. Below, we would like to respond to your comments point by point.

Q1: Query budgets discussion.
A1: [Comparison to existing work and discussions] Firstly, prior studies typically count each user as one query, regardless of how many times their interaction list changes (see Sec 4.1.1 of [1]). In contrast, we adopt a stricter and more realistic approach: each new interaction list, even from the same user, is treated as a separate query. Under this definition, methods like [1] incur a budget up to 1M on ML-1M (5000 users × 200 interactions), which is significantly larger than our setting of 400k. Moreover, as the first attempt to explore data-free model extraction for recommender systems based on user-item free embeddings, while our current setup may not fully match industrial-scale deployment, we believe it provides a solid and conservative foundation for understanding query efficiency in data-free recommendation model extraction.

Q2: Lacks discussion of attack detectability and potential countermeasures:
A2: Thank you for the suggestion. Due to space constraints, we briefly discussed potential defense strategies on Page 9, Lines 360–364. Following your valuable feedback, we will expand this discussion from three perspectives (detection, model-level defense, and legal & ethical considerations), and include a more comprehensive analysis in the Appendix.

Here, we discuss the future extension of potential countermeasures from three perspectives: detection-based defense, model-level defense, and legal & ethical considerations.
[Detection-based defense] Recent attacks [1,2] often rely on autoregressive querying or generator-driven interaction synthesis. Monitoring query frequency, interaction entropy, and behavioral anomalies can help detect suspicious users. Platforms may then apply query rate limiting or introduce noise to hinder extraction.
[Model-level defense] This approach involves injecting perturbations into the model's representation or output space to mislead attackers. However, in hard-label recommendation settings, such defenses require careful calibration to maintain model accuracy while degrading extraction effectiveness.
[Legal & ethical considerations] In addition to technical defenses, platform operators should implement responsible deployment practices such as strict API access control, policy enforcement, and legal safeguards. Raising public awareness and establishing regulatory frameworks can further mitigate the misuse of recommendation models.
Moreover, this study is conducted on benchmark datasets with the aim of improving system robustness and promoting awareness of potential vulnerabilities.

Q3: Computation cost about generator:
A3: Thank you for raising this point. Regarding mentioned “100 iterations with 1,000 synthetic users per loop”., in our setting, the total budget is at most 10^5 forward passes, not 10^8. While if each full batch of 1,000 synthetic users is considered as one forward pass, then the total number is only 100 forward passes. Therefore, we believe the overall computational cost of the generator remains manageable in practice.

Q4: High-rank agreement:
A4: Thanks for your advice. We have re-evaluated the high-rank agreement (Agr@10) between the trained surrogate model and the victim model. The results further confirm the effectiveness of DBGRME. We would like to add this to the final Appendix.

[1] Yue Z, He Z, Zeng H, McAuley J. "Black-box attacks on sequential recommenders via data-free model extraction." Proceedings of the 15th ACM conference on recommender systems. 2021.4.
[2] Wang Y, Su J, Chen C, et al. Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(12): 12766-12774.