We appreciate Reviewer y2uu for your constructive feedback. We hope our response effectively addresses your concerns and questions.

W1: About novelty

As discussed in the manuscript, our work is motivated by a key limitation of existing data-poisoning approaches: they require access to all of a subject's images to be effective. This is often impractical, as controlling every image of a subject on the internet is impossible. As shown in Table 1 of our main paper, the presence of even a single unprotected image can cause the protection to fail.
PersGuard operates under the same assumption of requiring full data access for its attack, thus sharing the same critical limitation as previous data-poisoning approaches. Furthermore, the authors of PersGuard note that their method's effectiveness is reduced by simple prompt variations (their "Gray-box setting"), whereas APDM demonstrates robustness not only to prompt variations but also to the presence of clean images and even entirely unseen images (as shown in Tab. R1 in our response W2).

W2 & W4: Clarification of anti-personalization process and results on the unseen data

(Process Clarification) Our work is grounded in a practical scenario where a service provider offers a diffusion-based personalization service to users. Within this system, a malicious user would naturally attempt to personalize our protected APDM using the provided by the service—which we model using the powerful and widely-adopted DreamBooth. In addition, we also conducted with other personalization methods, Custom Diffusion, and represent the results in Table 3 in the supplementary.
(Robustness to Unseen Data) In our paper, the images used for personalization are the same as those used for our protection process, i.e., negative samples in DPO. However, APDM also demonstrates robustness regardless of the number of unseen images. As shown in the Tab. R1, APDM can protect the personalization with 4-12 unseen images, which contain the protected target, i.e., not seen in the training.

Table R1. Personalization with unseen data.

(Robustness to Extended Fine-Tuning) Additionally, we test APDM on a larger number of personalization iterations (Tab. R2). As seen below, we found that APDM is still robust to prolonged fine-tuning.

Table R2. Personalization with larger iterations.

W3: Threat Model and Assumptions

(Knowledge and Capabilities of Users)

• Malicious User: We assume a strong adversary. They can attempt personalization using either clean images (if leaked/off‑platform) or perturbed images after bypass transforms (e.g., flip, blur). During inference, they have full freedom to vary prompts and can use any identifier (e.g., t@t), not just the default sks token.
• Protector (Service Provider): The provider's capability is demonstrated by applying protection effectively across various models (e.g., SD v1.5, SD v2.1) and against multiple personalization methods (e.g., DreamBooth, Custom Diffusion). The negative samples used for protection are proprietary to the provider and are not exposed to any user.

(Defender's Control Conditions) The core condition under which the defender has control is a provider-controlled deployment. In this model, a user requests protection for their identity, and the provider updates its model to a protected version. The provider maintains full control over the model's lifecycle.

(Realism of this Assumption) We argue that this assumption is highly realistic in many practical scenarios. Service providers inherently control their own models and deployment environments. They are responsible for the service they offer, but not for user activity outside their platform. This setting is a practical expansion of the "uncontrolled" setting considered in Anti-DreamBooth.
Furthermore, in the rebuttal, we also add studies on variant class names, multi‑subject protection, and unseen‑data protection; these results highlight APDM’s robustness, and we will include them in the final version.

W5: About theoretical analysis

The goal of our theoretical justification is to provide a formal, rigorous proof that moves beyond intuition, and we would like to elaborate on its non-trivial nature. The reviewer's intuition seems to be that since $L_{per}$ converges, $L_{adv}$ ($L_{protect}$) must naturally fail. However, this high-level view oversimplifies the problem and overlooks the core of our analysis.
The convergence behavior of $L_{adv}$ is not self-evident for a critical reason: $L_{adv}$ is not simply the negative of $L_{per}$. The key difference lies in the shared perceptual loss term, $L_{ppl}$. The presence of the positive $L_{ppl}$ term in $L_{adv}$ creates a complex optimization dynamic. It acts as a regularizer pulling the optimization in a different direction from the $-L_{simple}^{per}$. Therefore, it is not immediately obvious whether the optimization will:

• fail to converge and oscillate,
• converge to a trivial solution where $L_{ppl}$ dominates, or
• find some other unintended local minimum.

Our proof provides the answer to this ambiguity. It is not trivial because it mathematically analyzes this internal conflict. We prove that the necessary conditions for convergence (derived in Proposition 1) lead to mutually contradictory requirements on the gradients (Theorem 1, Eqs. 11-12). This specific mathematical deadlock is the non-obvious reason for its failure. Furthermore, this naive approach is a well-motivated and relevant baseline. Similar gradient ascent techniques are widely adopted in fields like privacy protection to remove specific knowledge from models. Our analysis is therefore valuable as it provides a formal explanation for the limitations of such intuitive approaches.

Thus, our analysis is essential. It moves beyond the simplistic "it must fail" intuition to provide a formal, rigorous proof of why and how it fails, justifying the need for a more principled approach like ours.

W6: Protection when the identifier (class name) changes

Table R3. Variation of identifier (class name).

We investigate the effectiveness of APDM when the identifiers (class name) are different between the protection and personalization. Note that, for APDM, the identifier "person" is used for protection. As shown in the Tab. R3, APDM shows its robustness to identifier changes (third row), as well as to changes on the unique identifier (Table 5 in the supplementary).

W7: Additional qualitative results

Due to the policy, we cannot attach a visual comparison in this rebuttal. Nevertheless, we argue that APDM can still produce high-quality and text-aligned images compared to the pre-trained Stable Diffusion. We will attach a visual comparison in the camera-ready version.

W8: Multiple identities scenario

We completely agree that deploying a separate model for each identity is impractical, and a multi-identity solution is vital for any real-world application. While we designated this as a key area for future work (as mentioned in Supplementary G), we were motivated by the reviewer's comment to conduct a new preliminary experiment. Our goal was to assess the inherent robustness of our current, unmodified APDM in a multi-identity scenario. To this end, we constructed a challenging negative set ($x^-_0$) with four distinct subjects (cat, sneaker, glasses, clock.) and applied our method without any tailoring. For the purpose of comparison, the results of PAP are also presented, for which the evaluation was performed on the perturbed data (the "# of clean images: $0$" setting in our main paper).

Table R4. Multi-subject protection.

These results highlight two critical points regarding scalability:
(Effective Multi-Identity Protection) Despite the increased complexity of protecting four subjects simultaneously, APDM still outperforms the comparison. The resulting DINO score of 0.3924 is significantly better than the unprotected baseline (0.5833) or PAP (0.5699), demonstrating that our core protective mechanism is not limited to a single identity.
(Inherent Robustness and Scalability) Crucially, this performance was achieved without any specific modifications to our algorithm, even in step (i.e., same training time). This showcases the inherent robustness of our approach, suggesting it is not a fragile, single-purpose solution but a generalizable framework with the potential to scale.

In summary, while a fully-optimized multi-identity model remains an important future direction, this experiment confirms that APDM's fundamental design is not inherently limited to a single identity. It directly addresses the reviewer's concern about practicality by demonstrating the robustness and scalability of our core method. This provides a strong foundation for the future work we have planned.

We appreciate Reviewer gqhK for the positive feedback. We are glad the reviewer recognized both the novelty of our practical approach and the strength of its empirical results. We address gqhK's comments and questions below.

W1: Using 100+ images for personalization

To directly address this concern, we conducted an additional experiment using a much larger dataset of 100 images for personalization. For this experiment, we manually collected a dataset of 100 images of a public figure from various online sources$^{[1]}$. We then evaluated APDM against DreamBooth and PAP in this setting. The results for PAP were generated using all 100 perturbed images, which corresponds to the most favorable configuration for PAP (the "# of clean images: $0$" setting in our main paper).

Table R1. Personalization with 100 images.

The results clearly demonstrate that APDM remains highly effective even in this more challenging "100-image" scenario. APDM's DINO score (0.3747) indicates significantly stronger protection compared to both unprotected DreamBooth (0.7983) and PAP (0.7811). Furthermore, the performance gap over PAP is even more pronounced in the BRISQUE (35.01 vs. 5.96). This confirms the superior robustness of our method.

[1]: To mitigate potential privacy and copyright concerns, this collected set was used exclusively for this rebuttal experiment. The data was handled securely and has been permanently deleted after use.

W2: Personalization on the same categories

To answer this question, we represent results for personalizing subjects from the same categories (person → person). In this experiment, we protected one female subject and personalized three other distinct female subjects (ID1-3). We report quantitative results following the evaluation protocol of DreamBooth.

(Training ID and ID1-3 are white-female persons, but the identities are different.)

Table R2. Personalization on the same subject.

As shown in the Tab. R2, APDM effectively maintains high personalization performance even for subjects within the same category (female → female). The DINO and CLIP scores are comparable. This demonstrates that our method successfully preserves the model's utility for personalizng new, similar subjects while protecting the target identity.

Thanks to the authors' detailed responses and extended results. The author has addressed some of my concerns, and I'll keep my score as it already indicates acceptance.

We appreciate Reviewer qkFB's positive evaluation about our motivation novel, our methodology effective, and our experimental results. We hope our response addresses your concerns and questions.

W1: Discussion about protection efficiency

To address this concern, we evaluate our approach against baselines based on both theoretical complexity and practical cost.
(Theorectical Complexity) The primary distinction lies in their scaling properties. APDM requires a one-time, fixed training cost (9 GPU-hours), making its computational complexity $O(1)$ as it is independent of the number of images to be protected ($N$). In contrast, data poisoning methods require per-image processing, leading to an $O(N)$ complexity where the total cost grows linearly with $N$.
(Practical cost) The real-world implications of this difference are significant. The Tab. R1 shows the processing times of existing methods:

Table R1. Time consumption.

As the Tab. R1 illustrates, they require approximately 5 minutes to poison the data. This allows for a direct cost-benefit analysis: APDM becomes cost-effective once the number of images ($N$) exceeds 110. In real-world applications driven by user-generated content like social media, where image sets are not only large but continually expanding, the $O(1)$ scalability of our approach offers a substantial advantage in overall efficiency. We will add a detailed discussion of this complexity analysis to the final manuscript.

W2: Additional comparisons

As requested, we conducted additional experiments comparing APDM with the suggested methods, Mist and SDS. We report the results on the containing a single perturbed image setting (the "# of clean images: $N-1$" setting in main table).

Table R2. Additional comparisons.

Our findings reveal two crucial points:

(Shared Vulnerability) In this setting, the new baselines are also highly vulnerable. This confirms that the presence of clean images is a critical challenge for existing data-level protection methods.
(Superior Robustness) APDM continues to significantly outperform all baselines, demonstrating its superior robustness again.

W3: Multiple identities scenario

We agree with the reviewer that protecting multiple identities is an important direction, and we have noted this as future work in Supplementary G. However, to proactively explore this, we conducted a preliminary experiment to assess the robustness of APDM in such a scenario. For this, we constructed a challenging multi-concept negative set ($x^-_0$) with four distinct subjects (cat, sneaker, glasses, clock) and applied our method without modifications. For comparison, we evaluated PAP using its most favorable configuration (the "# of clean images: $0$" setting in our main paper).

Table R3. Multi-subject protection.

The results demonstrate two key aspects of our method:
(Effective Protection) APDM maintains substantial protective capabilities, achieving a DINO score of 0.3924—far superior to both unprotected DreamBooth (0.5833) and data poisoning approach PAP (0.5699).
(Remarkable Robustness) This strong performance was achieved with an unmodified APDM, showcasing its inherent ability to generalize its protective mechanism to a task of greater complexity.

Therefore, this experiment not only confirms the strong potential for multi-identity protection but also validates the robustness of its core design. We believe this serves as an excellent starting point for developing a dedicated multi-concept protection method in our future work.

We appreciate Reviewer 3bmf for your constructive feedback. We hope our response effectively addresses your concerns and questions.

W1: Necessity of model-level protection

It is important to clarify who is responsible for this privacy risk. Under regulations such as GDPR, the service provider—not end users—must guarantee protection for their service model. Crucially, providers already control the life‑cycle of the model and they can control the risk by updating their model. In this circumstance, the model-level protection is a highly practical solution.

Even if your suggestion (i.e., watermarking) can solve the user burden, it still has some limitations: (i) newly uploaded content from other sources remains unprotected, and (ii) simple transforms can weaken the watermark, reviving the privacy risk (Bypass issue).

The concerns in Supplementary F (and illustrated in Figure 1) are motivated by the above problems. In those part, we identify four key problems, and service-provider-side solutions like watermarking fail to address three of them. As we mentioned above, watermarking has a limitation about (i) and (ii). Furthermore, another critical problem also arises in the service phase: the provider has no way of knowing which user-provided data requires protection (conflict with regulation). In other words, if malicious users use clean images, the watermarking approach cannot handle this. For this reason, with watermarking, the service provider cannot fulfill their responsibility to protect sensitive data.

This analysis underscores the necessity of a model-level approach like APDM. However, this does not render data perturbation methods obsolete. As you mentioned, methods like watermarking are effective in out-of-service cases. However, they cannot fully address the protection challenges, such as the new content and bypass problems, which APDM is designed to solve. Therefore, a complementary deployment of data‑level and model‑level protections is the most efficient and robust strategy.

W2: Discussion about computational cost

To provide a solid analysis of the computational expense, we compare our approach with existing methods by addressing the fundamental difference in their computational complexity and providing quantitative results.

The core difference lies in the scaling properties. APDM requires only a fixed cost (9 GPU-hours) for protection, which is independent of the number of images to be protected. Therefore, its complexity is $O(1)$. In contrast, poisoning approaches require processing each image individually. While the per-image cost is relatively low, the total cost scales linearly with the number of images ($N$), resulting in an $O(N)$ complexity.

Table R1. Time consumption.

As shown in the Tab. R1, existing poisoning approaches need nearly 5 minutes to process their methods. This allows for a direct computational cost comparison. APDM becomes more cost-effective when the number of images ($N$) exceeds 110. In real-world applications, such as social media platforms or cloud services where users continuously upload new photos, $N$ is not only large but also constantly growing. In such scenarios, APDM is significantly more efficient in terms of overall cost. We will add a detailed discussion on this computational complexity analysis to the final manuscript to clarify this point.

W3: Multi-concept protection

Table R2. Multi-subject protection.

As noted in Supplementary G, we aim to extend our work to multi‑concept protection scenarios. Motivated by your comment, we conducted a preliminary experiment to test the robustness of our current APDM on a multi-concept task. For this experiment, we constructed a multi-concept negative set ($x^-_0$) by combining samples from four distinct subjects (cat, sneaker, glasses, clock) and applied our APDM without any modification, treating them as a single conceptual group. For comparison, we reported the results of PAP using all perturbed data (the "# of clean images: $0$" setting in our main table) for the evaluation. As shown in Tab. R2, APDM still provides substantial protection in the multi-concept setting compared to PAP. This demonstrates the inherent robustness of our approach, even for a task it was not originally designed for, and highlights the potential of APDM for future work.

W4: Eq. (2) and (3)

We thank you for this suggestion. While we initially adopted the notation from the original DreamBooth paper for consistency, we agree that Eq. (3) can be improved. We will revise Eq. (3) in the final version for better clarity by explicitly representing the personalized text token ($c_{per}$).

W5: Additional comparisons

Table R3. Additional comparisons.

We present additional results in Tab. R3. We report the results for the setting that contains a single perturbed image (the "# of clean images: $N-1$" setting in our main table). The results show that APDM still outperforms these stronger competing methods.

W6: Response to perturbed data

Table R4. Personalization with perturbed data.

To answer this question, we conducted an additional experiment. We generated perturbed data using the Anti‑DreamBooth method. As shown in the Tab. R4, APDM is robust to input data, even when the data are perturbed. Furthermore, APDM with perturbed data still outperforms the original Anti-DreamBooth with a large margin, even when Anti-DreamBooth utilized all perturbed images (the "# of clean images: $0$" setting in our main table). This demonstrates the powerful protective performance of APDM.

W7: Noise Budget

As mentioned in Line 225 (Experimental Setup) in our main paper, the perturbation intensity (noise budget) was set to 5e-2. We selected this value based on the Anti-DreamBooth paper, and for a fair comparison, this value was applied to all baselines.

W8: Qualitative results in Table 3

Table R5. Additional quantitative results.

Unfortunately, we are unable to provide qualitative figures due to the rebuttal policy. However, we argue that Table 3 in our main paper already provides strong evidence to address this concern. The DINO score is a key metric that directly evaluates the success of personalization. The fact that APDM outperforms DreamBooth on this score indicates that APDM both (i) successfully personalizes the unprotected subjects and (ii) avoids over‑fitting to the protected target. Because APDM achieves a higher score, any potential degradation in quality is minimized. Furthermore, as shown in the Tab. R5, APDM also achieved competitive scores on both CLIP‑I and CLIP‑T, indicating that its outputs remain well-aligned in both the image and text domains. We hope this explanation resolves your concern.

W9: About Recent works (Writing)

Thank you for the suggestions. We will update the camera-ready version to include these recent related works as follows:

• [1] investigates the vulnerability of existing protection methods. We will cite this work to provide stronger support for our problem definition, particularly for the Bypass problem.
• [2] proposes a targeted attack approach. As we have already demonstrated in Tab. R3 of this rebuttal, this method's effectiveness is still limited in our scenario. We will incorporate [2] as an advanced baseline in our comparisons to further highlight the robustness of our method.

[1] Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI - Robert Hönig, Javier Rando, Nicholas Carlini, Florian Tramèr - ICLR 2025
[2] Targeted Attack Improves Protection against Unauthorized Diffusion Customization - Boyang Zheng, Chumeng Liang, Xiaoyu Wu - ICLR 2025

Thank the authors for the detailed reply to all the questions raised. In view of the rebuttal, the model level protection is intriguing, find myself more positive about the approach and would update final justification after finer consideration of the points raised by other reviewers.

If time permits(apologize for the last minute request), would appreciate if the authors could answer this follow-up question on reply W7: While results are obtained at noise budget 5e-2 based on Anti-Dreambooth, more recent work like ACE achieve good results at ~2e-2 noise budget. Will it be possible to provide minimum results (ACE and APDM) at the noise level 4/255 (similar to W5).