Differentially Private Latent Diffusion Models

• Low rank approximation: We thank the reviewer for this suggestion. Our current submission studies the trade-off between the final accuracy based on which layer’s attention modules to be fine-tuned, and the privacy guarantee we provide. Once we include LoRA or other parameter-efficient LLM training schemes, this fore-mentioned trade-off takes an extra dimension. Namely, once we approximate the parameters by a few low-rank matrices, we reduce the dimension of the parameters to fine-tune with DP-SGD (which is a good thing). This, however, introduces a loss of information about the parameters (which is a bad thing). As a result, now we need to study the trade-off among the three: (a) which layer’s attention modules to fine-tune (so far fine-tuning the layers in out block is preferred) ; (b) how accurate the approximation gets (which prefers a higher rank); and (c) how the resulting model performs in the presence of noise due to DP-SGD (which prefers a lower rank). This trade-off might exacerbate when there is a larger gap between the public and private datasets we consider. We are currently looking into the possibility of plugging in the LoRA package into our code base to test this trade-off. Please check our updated results at https://openreview.net/forum?id=FLOxzCa6DS&noteId=7Huv2tefxz

• Table 7: As the reviewer observed, DPDM outperformed our method when tested on MNIST. This result, however, is not transferable to other more complex datasets, where pre-training a larger model (LDM) and fine-tuning a small number of selected parameters made a larger accuracy increase than training the entire DM with DP-SGD (see Table 13, CelebA32). Here is some more comparison to DPDM, where the numbers we took from their Appendix (https://openreview.net/pdf?id=pX21pH4CsNB) for CelebA64 and CIFAR10: | FID | DP-LDM(ours) | DPDM | |-------------------------------|------------------|------| | CelebA64 (eps=10, delta=1e-6) | 14.2 | 78.3 | | CIFAR10 (eps=10, delta=1e-5) | 8.4 | 97.7 |

• Minor issue: The main document should only contain a 9-page main body and references, while the authors also attach the appendix. Thanks for brining this up. We double checked the author guideline at https://iclr.cc/Conferences/2024/AuthorGuide, it is said that either way is allowed.

We would like to thank the reviewer for the feedback and suggestions for further improvement. We will add them to our final version of paper.

• About Novelty: We would like to reiterate the main contributions of this work. Diffusion models are currently the state-of-the-art method in image generation. There are currently two published methods that attempt to make diffusion models differentially private. DP-DM performs well on simple datasets such as MNIST and FashionMNIST, but does not perform well on more complex datasets such as CIFAR10 and CelebA32 and 64. This method also requires a long training time as a diffusion model is trained from scratch. DP-Diffusion attempts to overcome this issue by pretraining a diffusion model on public data and then fine-tuning on private data. However, this fine-tuning is done on all of the parameters of the diffusion model and is still computationally costly. Our method incorporates the benefits of all these attempts. In particular, we also use diffusion models, as DMs model the complex image distributions very well. We also use public data to reduce the training time (which leads to a smaller privacy loss). On top of these ingredients, we incorporate the latent diffusion models that reduce the dimension of diffusion process and selectively fine-tune attention modules in the LDMs to further reduce the fine-tuning time and dimensionality of the parameters to be fine-tuned. The resulting algorithm achieves state-of-the-art results in image generation with differential privacy. There is no other work that achieves this level of accuracy using diffusion models. Thanks to our particular model choice, we can furthermore generate realistic images at the scale of 256x256 and text-to-image generation with differential privacy, which was not explored before.
• Comprehensive study on the influence of network architecture selection on DP-SGD: We have these results in our appendix already, where we tested a different number of layers for LDM, different downsampling ratio (f) and different number of channels for auto-encoders, and ablation studies given in several sections in our Appendix. These are all standard things to search when using LDM, which we diligently delivered already. If there are any other things we should consider doing, please let us know.
• General instruction for fine-tuning strategy: For this, we illustrated the accuracy versus which layer’s attention modules to fine-tune, and also how to pick a good public data given a private dataset. Overall, we found that fine-tuning the layers in Out Blocks were more useful than those in Input Blocks, at a fixed privacy budget. This outcome was consistent through all datasets we tested. So this is one strategy we will suggest for future use of this work in a new task. We can use FID (or a privatized FID, see our comment to c8Le) as a proxy to judge if a particular candidate public data is more useful than others, which is another strategy for users. If there are other things the reviewer thinks we should add other than these strategies, please let us know.
• Given that public data is assumed, the author may consider how to fully exploit to refine the DP-SGD with less noise: In our method, we pre-train an auto-encoder (VQ-GAN based), which provides the embedding of high-dimensional inputs and creates the diffusion process in this latent space to reduce the amount of noise to add in DP-SGD, compared to running DP-SGD on the high-dimensional input space. One can additionally use gradient embedding as the reviewer suggests to further reduce the noise in DP-SGD. In fact, any of the numerous papers that aim to improve the performance of DP-SGD (moving average, data augmentation, sparsifying gradients, DP-ADAM, etc) can all be applied to our work to further improve the performance of DP-SGD.

We thank the reviewer for the feedback. Here are the answers of each point :

• Novelty: We clearly mentioned in our introduction that the use of DP-SGD to fine-tune a pre-trained model itself could be viewed unremarkable as a method. However, please pay attention to what impact this seemingly ordinary method can bring. In fact, the potential impact is well appreciated and described by other reviewers: Reviewer tmmk wrote “The paper reads incremental, but it does a decent job in finding the correct hammer, which does lead to promising results.”, and reviewer c8Le wrote “Although the proposed approach lacks novelty, DP latent diffusion models could be of great interest to the community and the results look promising. “. Notation on encoder and decoder: we’ve used E and D for referring to other things in the paper, so we settled with a non-standard notation for encoder/decoder. We can certainly change these to others. We’re open to suggestions, if the reviewer wants to suggest a better notation.

• "encoder “ Δ and decoder ∇ ": non-standard notations: We understand that E and D are commonly used for encoders and decoders. Unfortunately, we have used D to note dataset to define differential privacy. We are considering changing encoder Δ and decoder ∇ to be Enc and Dec as used in [1,2,3]. But if the reviewer has a suggestion on which notation we should use for encoders and decoders, we’re open to it.

• Table 2: There is NO synthetic data involved in the 88.3% accuracy. Showing this accuracy (evaluated on real test data) is quite standard in DP data generation literature, as it is used as a top accuracy synthetic data could achieve given a dataset (implying that no synthetic data can be better than real data).

• For FID using private data: please see https://openreview.net/forum?id=FLOxzCa6DS&noteId=BmOgRtVX6z.

• Conditioning embedder: Conditioning embedder refers to both class conditioning and other types of conditioning such as conditioning on prompts, blurred variants of input image, semantic maps, layouts, etc. For more information about conditioning in LDM, refer to [4]. When there’s a difference between the classes in private and public datasets, e.g., in imagenet32 with 1000 classes and cifar10 with 10 classes, we freeze the embedder corresponding to 9900 classes, while fine-tuning the pre-trained embedder for the first 10 classes. Then, we use only these fine-tuned components when generating class-conditioned images for CIFAR10.

[1] ABL Larsen et al, Autoencoding beyond pixels using a learned similarity metric, PMLR, 2015

[2] Liu, Yihong et al, On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss, ACL, 2023

[3] Sprinks, Graham, and Moens, Marie-Francine, Generating Continuous Representations of Medical Texts, NAACL, 2018

[4] Rombach et al, High-Resolution Image Synthesis with Latent Diffusion Models, CVPR, 2022

• FID using private data: Whether it’s necessary to add noise to FID depends on whether we release these intermediate FID scores for choosing the public dataset. In our submission, we did not consider privatizing FID, as we had no intention of releasing these FID scores. One might argue that even if the intermediate FID scores are not released, the selection itself is already embedded in the pre-trained model, which is then used for private fine-tuning. So, to be thorough, we can consider using a DP mechanism to privatize the selection step. Recall that FID is calculated by two statistics, mean and covariance of features from a pre-trained inception network, so using a Gaussian mechanism to privatize the mean and covariance of the private data before computing FID can be used [7]. A similar idea was used in DP-MEPF[8], which uses the mean feature difference between private and public data to determine early stopping. Considering the privatized mean difference only makes sense, as the noise scale for mean perturbation is O(d), while that for covariance perturbation is O(d^2). As shown below, the privatized mean alone, and both mean and covariance do not alter the rank of the final FID scores significantly. This stems from the fact that the sensitivity of the mean and covariance is extremely small for datasets we consider, i.e., on the order of 1 over the number of training data. So privatizing or not privatizing FID does not change the conclusion about which dataset is more useful to use as public data for pre-training. We will include this point in our final version.

Here is the FID when we privatize both mean and covariance, the eps is the total value of eps for mean and eps for covariance, basically we set eps for mean = eps for covariance.

Here is the mean difference when we privatize mean only:

[7] Park, Mijung et al, DP-EM: Differentially Private Expectation Maximization, AISTATS, 2017

[8] Harder, Frederik et al, Pre-trained perceptual features improve differentially private image generation, TMLR, 2023

We thank the reviewer for the positive feedback and detailed suggestion. Thanks for pointing out the typos and the parts that need to be clarified. We have fixed these and will add further improvement according to the review's feedback to our final version.

Below are the answers to each point:

• Using LoRA in our model: Thanks for the suggestion. Our current submission studies the trade-off between the final accuracy based on which layer’s attention modules to be fine-tuned, and the privacy guarantee we provide. Once we include LoRA or other parameter-efficient LLM training schemes, this fore-mentioned trade-off takes an extra dimension. Namely, once we approximate the parameters by a few low-rank matrices, we reduce the dimension of the parameters to fine-tune with DP-SGD (which is a good thing). This, however, introduces a loss of information about the parameters (which is a bad thing). As a result, now we need to study the trade-off among the three: (a) which layer’s attention modules to fine-tune (so far fine-tuning the layers in out block is preferred) ; (b) how accurate the approximation gets (which prefers a higher rank); and (c) how the resulting model performs in the presence of noise due to DP-SGD (which prefers a lower rank). This trade-off might exacerbate when there is a larger gap between the public and private datasets we consider. We are currently looking into the possibility of plugging in the LoRA package into our code base to test this trade-off. We will update this rebuttal once we get the results.

• Clarification for the GPU hours spent: Sorry for the confusion. Table 7 does not contain the pretraining hours. There are three steps in our method, first pretraining an autoencoder takes 8 hours as listed in Table 14, second pretraining the LDM takes 6 hours as listed in Table 15, third fine-tuning the LDM to make it DP takes 10 hours, which is listed in Table 7. The DP-DM method does not contain any pretraining, they trained from scratch, which takes 192 hours in total.

• Privacy definition: Yes, we use the inclusion/exclusion definition of DP, as this is what Opacus uses. We will make this clear in our paper.

• DPSGD scaling issues: We’re aware of [1,2] that the reviewer cited. The finding of these papers is that when the gradients can be characterized by a few principal components (this happens in the case of fine-tuning LLMs), the scaling issue of DP-SGD is lifted. However, in general, this issue is present in most settings of training deep neural network models.

• For Table 4, best result is the highest we achieved, avg result is the avg we have by taking three runs with different seeds.

• Table 8, we did the hyperparmeter search for each dataset, "Best CNN accuracy" is the highest score we achieved for individual dataset, which shows out choice of EMNIST will achieve best performance among those three.

• Why fine-tuning 9-16 layers is the best and provide recommendations : Our results suggest that empirically finetuning attention modules in out_blocks outperform finetuning those input_blocks. This is consistent with the results of [6]. We sould recommend this strategy to users if their privacy budget is limited. We will add this discussion to our final version.

[6] You, Fuming and Zhao, Zhou, Transferring Pretrained Diffusion Probabilistic Models, https://openreview.net/forum?id=8u9eXwu5GAb, 2023

As requested, we performed additional experiments after incorporating LoRA into our model. Our results for unconditional CelebA generation at 64x64 resolution (trained for 50 epochs) are summarized in the following table of FID values. Each FID value is computed as the average of three runs. The final column in the table contains results using DP-LDM for comparison, though we note that the FIDs are slightly higher (worse) than in tables 4 and 5 in our main paper because these models were trained for 50 epochs instead of 70.

As shown in the table, when we allow for small privacy loss (epsilon=1), LoRA achieves the best results at rank 8. As we allow for larger privacy loss (epsilon=10), the accuracy of LoRA at higher rank (64) performs better. However, in all cases, the performance with LoRA is still lower than that with the full parameters fine-tuned in DP-LDM.

This phenomenon is also observed in fine-tuning large language models. As a concurrent work with [2], Li et al [1] showed parameter-efficient adaptation methods (which reduce the dimensionality of updates) do not necessarily outperform a baseline method that fine-tunes all model parameters. The large batch size improves the signal-to-noise ratio (shown in Fig 3 [1]), which significantly helps improve the performance of the full-parameter fine-tuned model. We find that this is analogous to our results. When we fine-tune the parameters in the attention modules at their original ranks, we lose less information. Due to the large batch size we use (bs=8192), we maintain a relatively good signal-to-noise ratio in the gradients perturbed by DP-SGD, which leads to better performance than LoRA at the same privacy level.

It is an intriguing question, whether the intrinsic dimensionality of the gradients are truly low, when fine-tuning DP-LDMs. Perhaps we could use the metric used in [3] to empirically evaluate the intrinsic dimension. This could be a direction to pursue for future work. We thank the reviewer for raising this question.

[1] LARGE LANGUAGE MODELS CAN BE STRONG DIFFERENTIALLY PRIVATE LEARNERS, X. Li, F. Tramer, P. Liang, T. Hashimoto, ICLR 2022.

[2] Differentially Private Fine-tuning of Language Models, D. Yu, S. Naik, A. Backurs, S. Gopi, H. Inan, G. Kamath, J.Kulkarni, Y. T. Lee, A. Manoel, L. Wutschitz, S. Yekhanin, and H. Zhang., ICLR 2022.

[3] Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning Aghajanyan et al., ACL-IJCNLP 2021.

Training our LDM with LoRA for 50 epochs took 36 hours on a single NVIDIA V100. This was the same across all epsilon levels and ranks ($r \in \{1, 2, 4, 8, 64\}$). For comparison, our original LDM training completed 50 epochs in 32 hours on a single NVIDIA V100. Note that after incorporating LoRA into our model, the training time increased. We believe this is because LoRA introduces new weight matrices into the model, increasing the amount of computation required in the backward pass. Despite fewer tunable parameters in the model (see the updated table in https://openreview.net/forum?id=FLOxzCa6DS&noteId=7Huv2tefxz), the same gradients that needed to be computed for training an LDM without LoRA still need to be computed in order to propagate backwards. The new weight matrices also need gradients, which represents an overhead that is captured by longer training times.

We thank the reviewer for the positive and constructive feedback on our paper. Thanks for pointing out the missing references and further improvements we can make to make the paper clearer; we will add those to our final version.

Regarding extra evaluations suggested by the reviewer:

1. We’re currently working on implementing the experimental setups for testing classification accuracy of the synthetic CelebA64 dataset. We will upload the results here once it is done.
2. We agree with the reviewer that evaluating our method on a scenario with an extreme domain gap between the public and private datasets. Hence, we are working on testing our method on the Camelyon17 dataset.

Answers to each point below:

• The baseline methods do not come with a cite. Is DP-diffusion [6] or [8]? Appendix A.2 suggests [6], but the caption of Fig. 8 suggests [8], which is confusing. : Thanks! We have corrected the typo.

• "inserted into the layers of the underlying UNet backbone": exactly where? The original Unet does not contain attention modules, we followed the way of LDM to insert attention modules into layers as the way illustrated in Figure 1 (marked in red). For input_blocks, attention modules are added to the Resnet block except the first in block and downsampling ones; For middle block, attention module is added between two Resnet blocks; For out_blocks, attention modules are added except the out block.

• "It modifies stochastic gradient descent (SGD) by adding an appropriate amount of noise by employing the Gaussian mechanism to the gradients" We have clarified this sentence.

• "However, Lin et al. (2023) do privatize diffusion" -> do not: Thanks, we have fixed this.

• "encoder “ Δ and decoder ∇ ": non-standard notations: We understand that E and D are commonly used for encoders and decoders. Unfortunately, we have used D to note dataset to define differential privacy. We are considering changing encoder Δ and decoder ∇ to be Enc and Dec as used in [9, 10, 11]. But if the reviewer has a suggestion on which notation we should use for encoders and decoders, we’re open to it.

[9] ABL Larsen et al, Autoencoding beyond pixels using a learned similarity metric, PMLR, 2015

[10] Liu, Yihong et al, On the Copying Problem of Unsupervised NMT: A Training Schedule with a Language Discriminator Loss, ACL, 2023

[11] Sprinks, Graham, and Moens, Marie-Francine, Generating Continuous Representations of Medical Texts, NAACL, 2018

As requested, we applied our model to conditional CelebA generation at 64x64 resolution. We began with an LDM pretrained on conditional ImageNet at the same resolution (64x64), and then fine-tuned it on CelebA where the (binary) class labels were given by the “Male” attribute. A ResNet-9 [2] classifier was trained on 50000 synthetic images, and then a test accuracy was computed on the test split of the real data. Note that these test set images were not seen in the training of the LDM or the classifier. As a baseline for comparison, we also ran publicly available DP-MEPF [1] code and computed test accuracies in a similar manner. Our results are summarized in the table below, which compares downstream test accuracies between DP-LDM and DP-MEPF [1] at several epsilon levels. All accuracies are computed as the average of three runs with standard deviations provided in parentheses. We see that our method outperforms DP-MEPF [1] at each tested epsilon level.

[1] Harder, Frederik et al, Pre-trained perceptual features improve differentially private image generation, TMLR, 2023

[2] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778, doi: 10.1109/CVPR.2016.90.