ObCLIP: Oblivious CLoud-Device Hybrid Image Generation with Privacy Preservation

We sincerelly thank the reviewer pbWf for the positive feedback on "Techniques to leverage redundancy ... (some of which appear to be novel) also make sense and appear effective" and helpful reviews. We address the concerns raised and respectfully hope that our clarifications will help prompt a positive rating of the paper.

Q1: Explanation for the comparision to cryptographic methods.

Apologies for the misunderstanding. We first clarify that the comparison with HE-diffusion is presented in Section 5.2 on page 7. We report only the latency because the original HE-Diffusion paper (in Table 2$\sim$3) only provides runtime numbers for efficiency and implicit similarity metrics for utility (to measure error between plaintext and encrypted generations), but lacks full utility evaluation. Due to the considerable computation overhead of HE-Diffusion (one denoising step takes over 100 seconds), we were unable to conduct a complete evaluation. During the rebuttal, we found a recent HE-based paper[1] that uses plaintext to approximate the utility. We adopt this methodology, which should provide the utility upperbound for these two HE-based works as it avoids the approximation errors introduced by encrypted inference. We provide the supplementary evaluation results in the table below. We take FID score reported in the paper of [1] due to lack of source code. MPC-based approaches do offer strong privacy and somewhat comparable utility with delicate approximation methods. However, their efficiency is typically orders of magnitude slower (due to heavy cryptographic computations). For example, as shown in the Table below, HE-Diffusion is over $100\times$ slower and [1] is even over $1000\times$ slower than ObCLIP. Such drawback greatly hinders the practicality of MPC-based approaches. In contrast, ObCLIP achieves significantly better efficiency with comparable utility.

Regarding the qualitative comparison in Table 1, to better highlight the trade-offs, we update the table with four levels of performance: High, Medium, Low, and Very Low. Specifically, MPCViT focused on secure inference of vision transformer, which is different from our approach (U-Net based diffusion). Therefore, we do not include quantitative comparison to this work. Note that, due to LaTeX syntax incompatibility, we here use textual indicators instead of circular symbols. We hope this resolves your concern and promise to update the table in the final version.

[1]: Secure Diffusion Model Unlocked: Efficient Inference via Score Distillation

Q2: Clarification on the novel contributions.

We argue that in caching-based acceleration, the core challenge lies in devising a caching strategy that effectively balances efficiency and utility. Specifically in this paper, we incorporate both temporal- and batch- redundancy-based accelerations designed for the client-server hybrid generation paradigm. For temporal redundancy, we first analyze server-side redundancy across self-attention, cross-attention, and block outputs, as shown in Figure 6. Guided by these insights, we develop a multi-granularity acceleration strategy that further reduces overhead compared to prior work. For example, different from T-Gate[1], we cache both the self-attention and cross-attention in the server-side early denoising (T-Gate only caches self-attention). In addition, we propose a novel batch-redundancy-based acceleration technique that reduces the total attention computation by a factor of $1/N$. By jointly leveraging both temporal and batch redundancy, our approach achieves a better efficiency–utility trade-off, which is thus non-trivial. We hope this approach offers useful insights for future work, particularly in real-world high-concurrency inference services, where similar forms of redundancy across different user requests could be exploited.

We also appreciate the reviewer’s positive feedback that "the approach is sensible." We believe that, despite its simplicity, our method offers an effective and practical solution for private image generation while maintaining good utility.

[1]: Faster Diffusion via Temporal Attention Decomposition. 2025

Q3: Clarification on the obfuscation approach.

Thanks for your careful reading. We provide the detailed design of obfuscation mechanism in Algorithm 1 in the Appendix B. We clarify that it mainly involves two steps: (i) detecting sensitive tokens, and (ii) constructing the corresponding candidate set. The first step, also known as named entity recognition, are used as an existing technique[1] in this work. One limitation is that we depend on the underlying model to detect sensitive tokens. We argue that this could be of independent interest. For example, there are recent works that use LLM for NER detection[2], which yields much better performance.

Regarding Fig. 8, we apologize for the insufficient elaboration due to page limit. In this figure, the parentheses indicate an example age (e.g., "18-year-old") to represent the "young" category. For general tasks that require age as input, we discretize age into three categories: young (18–35), middle-aged (36–59), and old (60+). We hope this resolves your concern and will include a more detailed explanation in the final version.

[1] Isotonic. Distilbert fine-tuned for ai4privacy task, 2025.

[2] UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition. 2023

Q4: Examples in Fig. 3b

We acknowledge that as shown in the figure, even server-only generation yields obvious defects. However, server-first hybrid generation is clearly more influenced by server-side guidance. This supports our design choice to offload the early denoising stages to the server, enabling stronger semantic alignment and improved generation quality.

We thank the reviewer 7DQm for the positive feedback on "ObCLIP can effectively reduce the privacy leakage risks of user prompts" and thoughtful reviews, which is quite encouraging for us. We hereby answer the specific questions below and hope the explanations resolve your concerns.

Q1: Experiments for multi - dimensional attributes

We clarify that our experiments actually encompass multi-dimensional attributes. To ensure a comprehensive evaluation, we include experiments involving two-attribute combinations (gender × age) and three-attribute combinations (gender × age × ethnicity), presented respectively in Table 2 and Table 6 in the Appendix. The results consistently demonstrate the effectiveness of ObCLIP across these attribute combinations.

We summarize the utility data in Table below, with ObCLIP yielding comparable performance to the no-private baselines. Beisdes, compared to the parivate baseline vanilla OG, our efficiency is about $7\times$ higher (and even orders of magnitude higher than cryptographic methods like HE-Diffusion). For healthcare scenario, attributes such as treatment records can be pre-discretized similarly (e.g., headache, cancer). The generation process can then be conducted as described in this paper by leveraging different attribute combinations.

Q2: Insufficient innovation in core privacy mechanism.

We clarify that our method obfuscates all sensitive attributes within the real prompt, rather than only single-dimensional attributes. This design explicitly addresses potential attribute correlations that could otherwise lead to information leakage, thereby ensuring perfect indistinguishability. This approach is applicable to realistic scenarios involving diverse and inherently multi-dimensional sensitive attributes. Despite its simplicity, our method offers an effective and practical solution for private image generation while maintaining good utility.

Moreover, as discussed in the Conclusion section, the obfuscation mechanism could be further optimized by incorporating differential privacy-based private subset sampling, which offers a better privacy-utility trade-off. Notably, this direction is orthogonal to our current approach and fully compatible with it. We would leave it as future work.

Q3: Limited generative model generalization

We argue that this paper focuses on U-Net based diffusion models, which is consistent across all compared baselines. Furthermore, we have conducted extensive experiments on a diverse set of stable diffusion models, including SD-v1.4, Realistic Vision, SDXL-1.0, and LCM-SDXL, to ensure a comprehensive evaluation. We believe these results sufficiently demonstrate the effectiveness of our proposed approach.

Regarding the extension to other Transformer-based diffusion models, we provide a theoretical discussion as follows. The primary difference lies in the denoiser backbone—Transformer-based models replace the U-Net architecture. Nevertheless, both architectures rely heavily on self-attention and cross-attention modules. Our cache-based acceleration primarily targets these attention modules, and therefore, the proposed method remains potentially applicable to Transformer-based models, provided that temporal and batch-wise redundancies still exist. Notably, temporal redundancy has already been observed in prior work[1], supporting the feasibility of our approach in this broader context. This is actually orthogonal to this work and we leave a comprehensive study into Transformer-based architectures as future work.

[1]: Faster diffusion via temporal attention decomposition.

We thank the reviewer YVsw for the positive feedback on "presents a novel and well-motivated cloud-device collaborative framework" and quite insightful reviews, which is quite encouraging for us. We hereby answer the specific questions below and hope the explanations resolve your concerns.

Q1: Trade-off for large candidate prompt set

Thanks for your clear understanding of our proposal. We briefly discuss such efficiency limitation in the Conclusion section. The computation cost indeed grows with the candidate prompt set size $N$. To handle large $N$ case, we propose to use a subset of candidate prompts (of cardinality $N'$, with $N' < N$) to greatly lower the computation cost. This can be achieved by differential privacy-based top-$N'$ selection. By choosing appropriate privacy budget $\epsilon$, we can achieve measurable privacy, guaranteed by differential privacy theory, while providing a much better efficiency. Such optimization is orthogonal to the method in this paper and we leave it as future work.

Q2: Extension of oblivious prompt transformation to general scenarios

We clarify that the oblivious transformation can be seamlessly extended to general scenarios. This transformation basically involves two steps: (i) detecting private tokens, and (ii) constructing the corresponding candidate set. For general applications, only the first step requires adaptation, as the definition of privacy is highly context-dependent. For instance, in location- or occupation-sensitive scenarios, one can employ regex-based rules or NER models to identify such tokens. The subsequent transformation process remains unchanged. We will include the discussion of this extension to general scenarios in the final version to enhance clarity.

Q3: Detailed diffusion process between the device and the cloud in general scenarios

We define switch point $k$ to divide the denoising steps between the cloud and the device. Specifically, the early $k$ steps are conducted on the cloud, with candidate prompt set as input. Then the remaining $T-k$ steps are conducted on the device, with the cloud-generated intermediate noisy latent as input. In the paper, we tried $k \in \{5, 10\}$ for comprehensive evaluations. A larger $k$ means more steps are conducted on the cloud, which can improve the quality of the generated images.

Q4: Devices in experiments.

We clarify that we adopt a standard client-server scheme. Therefore, only one device is used in the experiments. Besides, since we focus on the image generation (i.e., inference), with no fine-tuning of the diffusion model, the distribution of client data should not have significant impact on the performance. As evidenced by the experiment results, ObCLIP yield comparable performance to the server-side model on both MSCOCO and MJHQ datasets.