Characterizing the Training Dynamics of Private Fine-tuning with Langevin Diffusion

Thanks for your positive feedback!

Weakness

“In practice, trainings are usually done in a discrete manner (say t=1,2,...), but Lagenvin diffusion is a continuous approximation for $t>0$. This gap may hamper the generality of this work's findings. For example, if $\Delta t$ in Theorem 3.3 belongs to [0,1], then feature distortion might not be an issue, because we start from t=1. Therefore, it would be helpful if authors could provide a short discussion of the value of $\Delta t$, or name some driving factors that may significantly affect the value of $\Delta t$”.

Response:

Thank you for highlighting the distinction between continuous Langevin diffusion and the discrete time steps used in practical training. This is an important point, as it helps clarify how theoretical findings based on continuous dynamics translate to real-world implementations. We will add a short discussion in our paper on this issue. To approximate continuous Langevin diffusion in discrete time steps, one can use the Euler–Maruyama method, a standard numerical approach for solving stochastic differential equations. This method discretizes the continuous process by mapping the time interval $\Delta t$ into discrete steps of size $\eta$, allowing us to approximate the dynamics as follows: $\theta_{t+1}=\theta_t-\eta\nabla\mathcal{L}(\theta_t)+\sqrt{2\eta\sigma^2}\xi_t$, where $\eta$ is the discrete step size, $\sigma$ is the noise scale, and $\xi_t$ represents Gaussian noise at each step. By choosing an appropriate $\eta$, we approximate the continuous process well over discrete steps, allowing theoretical insights into the continuous process to inform practical implementations. As for the value of $\Delta t$ in Theorem 3.3, it represents the initial time interval during which feature distortion occurs. In practical terms, the rate and degree of feature distortion can vary based on factors such as:

• The initialization scale of model parameters, particularly the linear head,
• The learning rate in DP-SGD, which affects the stability and alignment of features in early training stages,
• The noise scale $\sigma$, as higher noise may delay alignment or lead to greater initial feature distortion. We also provide a detailed description in remark F.2, Appendix F in our updated paper.

Question

“Typos in Equation (10), Line 1148”.

Response:

Thank you for catching that error. Yes, the time range in Equation (10) is indeed a typo. We appreciate your careful reading, and we will correct Equation (10) and Line 1148 in the revised version of our paper.

Question:

“Public pre-training and private fine-tuning approaches have been seriously questioned recently. I would appreciate the author's thoughts on the recent position paper by Tramèr et al. (2024).”

Response:

We generally agree with the points raised by Tramèr et al as guiding principles. That being said:

1. On the utility critique, there are use cases where there exists public data (that can be responsibly curated) that is representative of the private domain. For example, natural language on public web forums has similarities to speech patterns on phone messaging apps (while not being identical). While their observations are generally true (e.g., pretraining on ImageNet won’t help a Sarcoma classifier), there are still important use cases where pre-training on public data can help.
2. On the privacy critique of public pretraining, we feel the points brought up by Tramèr et al are not just a weakness of pretraining on public data, but of differential privacy as a whole, which is not nuanced enough to capture contextual integrity. Indeed, there has been work (Cummings et al, CCS 2021) showing that users do not understand or appreciate the privacy affordances of differential privacy. This does not mean that DP as a whole should not be studied, but it broadens the scope of interesting privacy questions to include alternative metrics and techniques that are more aligned with user expectations.
3. More generally, we do not think that topics should be considered “not worth studying” simply because they are out of fashion or controversial. There are responsible and irresponsible ways of deploying DP finetuning, as with any technology. We are trying to understand the foundations of a problem, which is separate from the (very important) problem of understanding how to deploy technologies responsibly.
4. There are some potential solutions to the problems Tramer et al. mentioned:

   • Use synthetic data (e.g. the random priors used in Tang et al. 2023).

     “Learning to See by Looking at Noise” by Baradad et al. (NeurIPS 2021).

   • Do differentially private pre-training.

     “ViP: A Differentially Private Foundation Model for Computer Vision” by Yu et al. (ICML 2024).

Question:

“How realistic are Assumptions 3.1 and 3.2? Were they experimentally validated?”

Response:

We discussed these assumptions in our response to Weakness-3. Additionally, empirical results from Hancheng Min et al. (2024) align with Assumption 3.1, providing practical evidence that supports its validity.

Your question highlights a fundamental challenge in deep learning theory—the gap between theoretical assumptions and empirical realities. Developing strong theories often requires certain idealized assumptions on the data that may not align with real datasets like ImageNet. In deep learning theory, there are typically two main approaches to making assumptions on data:

1. Statistical Assumptions: These assume distributional properties of data, often unrelated to real datasets but motivated by theoretical goals. For instance, Lee et al. (NeurIPS 2022) in “Convergence for Score-Based Generative Modeling with Polynomial Complexity” assume the data distribution satisfies a log-Sobolev inequality to leverage stochastic differential equation theory.
2. Geometric Assumptions: These assume certain geometric patterns in data points, which is common in studying training dynamics of multi-layer networks. For example:
   • Kumar et al. (ICLR 2022) assume that out-of-distribution (OOD) data points are perpendicular to training data. (https://openreview.net/forum?id=UYneFzXSJWh)
   • Phuong and Lampert (ICLR 2021) assume orthogonal separability.
   • Wang and Pilanci (ICLR 2022) and Min et al. (ICLR 2024) also use geometric assumptions relevant to neural network convergence and alignment.
3. Implicit assumptions: assume that a family of abstract loss functions that have certain properties used in standard optimization theory, such as convexity, PL condition, and smoothness. These assumptions restrict the properties of loss landscapes and thus implicitly restrict properties of the training data.
   • Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M. Kakade, Michael I. Jordan. “How to Escape Saddle Points Efficiently”. ICML 2017.

Question:

“Is there a reason why Langevin diffusion is defined differently from Ganesh et al. (2023b)?”

Response:

Our definition of Langevin diffusion is equivalent to that of Ganesh et al. (2023b), with a difference in notation. In our work, we use the noise scale $\sigma$, which directly corresponds to the noise multiplier in DP-SGD, while Ganesh et al. use $\frac{1}{\beta}$ for the same term. We chose $\sigma$ for consistency with DP-SGD terminology. When we apply the Euler–Maruyama method on our Langevin diffusion, we obtain a discrete update that corresponds to DP-SGD. For our analysis with clipping, please refer to “Appendix F: Theory with Clipping”.

It’s also worth noting that there are two equivalent definitions of Langevin diffusion commonly used in the literature:

• “On the universality of Langevin diffusion” (https://arxiv.org/abs/2204.01585v3) by Ganesh et al.
• “Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks” (https://proceedings.neurips.cc/paper_files/paper/2023/hash/1165af8b913fb836c6280b42d6e0084f-Abstract-Conference.html) by Ye et al.

Both definitions provide similar privacy guarantees, as shown in Lemma 2.1 of the first paper and Theorem 3.1 of the second.

1. Tilde notation: The notation (x,y) ∼ (x’,y’) means that y=y’. We will remove this notation and just write $y=y’$ in our paper.
2. Gaussian initialization: We make this assumption based on the following papers:
   • Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. ICLR 2022. In their experiments, they initialize the linear head with zero-mean gaussian distribution.
   • Xinyu Tang, Ashwinee Panda, Vikash Sehwag, Prateek Mittal. Differentially Private Image Classification by Learning Priors from Random Processes. NeurIPS 2023. In their experiments, they initialize the linear head of the WideResNets with zero-mean gaussian distribution.
3. “Optimality”: Thanks for pointing out the ambiguity of “optimality”. It simply means that $w_j$ perfectly aligns with the mean data direction of a certain label.
4. Ganesh’s paper: The final version of Ganesh’s paper, published in Conference on Learning Theory, is based on the third version of their arxiv paper. Follow this link: https://arxiv.org/abs/2204.01585v3. Our theorem 4.1 is based on their Lemma 2.1. Similar results can be found in Theorem 3.1 of ”Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks” by Ye et al. (NeurIPS 2023).
5. Explanation of Theorem 4.2 and 4.3:
   • Explanation of Theorem 4.2:
     • Interpretation of constants: $A_1$ and $A_2$ respectively measures the maximum/minimum alignment between the pre-trained features $w_j$ and the data points $x_i$. $B_1$ and $B_2$ are the noise scale multiplied by the norms of the pre-trained features.
     • Limit behavior: If we take limit $t\rightarrow\infty$, then we get a lower bound limit $\frac{B_1}{A_1}$ and an upper limit $\frac{B_2}{A_2}$. When there is only one feature vector and one data point, the upper limit and lower limit are equal and we obtain an exact dynamics for the loss.
     • Effect of noise: If we only increase the noise scale $\sigma$ and fix other initial settings, $B_1,B_2$ are larger. As a result, the expectation of the loss would decrease faster but cannot get close to zero because of the large noise. The faster convergence induced by noise is caused by the curvature of the loss landscapes. The second term $\sigma^2\mathrm{tr}(H_{\theta}\mathcal{L})$ of Ito’s Lemma (Equation 2) introduced the curvature property of the loss function $\mathcal{L}$ into our analysis.
     • Effect of pre-training: If we use a bad pre-trained encoder (i.e. decrease the alignment between the pre-trained features and the data points), the loss lower bound is further away from zero. This agrees with the intuition that given a bad backbone, linear probing cannot achieve good performance.
     • Order of convergence: the linear probing setup gives linear and convex properties. So we obtain exponential convergence, which agrees with the standard results in strongly convex settings.
   • Explanation of Theorem 4.3:
     • Limit behavior: If we take limit $t\rightarrow\infty$, then we get a lower bound limit $\frac{C_l}{B_l}=\frac{\sigma^2(1+\mu^2)}{4}$ and an upper limit $\frac{C_u}{B_u}=\frac{\sigma^2}{2\mu^2}$.
     • Effect of noise: If we only increase the noise scale $\sigma$ and fix other initial settings, the lower limit and upper limit both increase. This indicates that the loss cannot get close to zero because of the large noise.
     • Effect of data separability: If we only increase the metric $\mu\in[0,1]$ of data separability, i.e. let the training data to be more separable, the upper limit $\frac{\sigma^2}{2\mu^2}$ decreases. Meanwhile, the upper limit and lower limit get closer. This indicates that the loss is expected to converge to a smaller value in the end as the training task becomes easier.
     • Effect of pre-training: the limit of the upper and lower bounds does not explicitly depend on the pre-training conditions. The intuition is that for our simplified setting, with different pre-trained backbones, full fine-tuning the whole model would finally produce similar performance conditioned on the training data and the noise.
6. According to the proof of Theorem 5.1, when we fix the total training time $T$, the necessary time $t_{lp}$ for DP-LP increases as we increase the noise scale $\sigma$. So the time proportion $\frac{t_{lp}}{T}$ of DP-LP increases throughout the whole fine-tuning process. According to Theorem 4.1, the privacy budget proportion of DP-LP depends on the time proportion of DP-LP. Therefore, a greater proportion of the privacy budget should be allocated to DP-LP when the total privacy budget is smaller.
7. Comments on Theorem 5.1: Sorry for the confusion. In this theorem, we use r as a high-probability upper bound for the privacy budget necessary to do linear probing. $\rho$ denotes the probability commonly used in inequalities derived by concentration bounds.

Review:

“The experimental setup lacks details on the clipping threshold, a key parameter influencing DP-SGD performance and necessary for reproducibility.”

Response:

Thank you for highlighting the need for clarity on the clipping threshold. In our experiments, we used clipping thresholds 𝐶 = 0.1 C=0.1 and 𝐶 = 1 C=1. For the vision benchmarks in our paper, these values are based on established empirical studies that explore optimal clipping thresholds for DP-SGD. In particular, Appendix B.1 of Unlocking High-Accuracy Differentially Private Image Classification through Scale by Soham De et al. (2022, DeepMind) provides an in-depth analysis of clipping norms, concluding with the choice of 𝐶 = 1 C=1 for their primary experiments.

Our experimental settings also draw from the methodologies outlined in:

• Unlocking High-Accuracy Differentially Private Image Classification through Scale by Soham De et al., 2022.
• Differentially Private Image Classification by Learning Priors from Random Processes by Tang et al. (NeurIPS 2023).

We hope this additional context clarifies our choice of parameters and provides the reproducibility details necessary.

Review:

"Assumptions 3.1 and 3.2 are unconventional and strongly restrictive. Only one previous work referenced these assumptions, making it difficult to accept them as standard or practical in the context of simple binary classification."

Response:

We appreciate the reviewer’s concerns about the unconventionality of Assumptions 3.1 and 3.2. However, these assumptions are rooted in a well-established line of research within neural alignment, developed over multiple studies focused on multi-layer network dynamics. Neural alignment has been studied in prior work for orthogonally separable data (Assumption 3.1) and for orthogonal data.

1. Assumption 3.1, for example, draws from foundational work, including Phuong and Lampert (2021), Wang and Pilanci (2022), and Min et al. (2024). Min et al. provided a detailed review of papers using Assumption 3.1 in Section 3.2 of the paper “Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization” (ICLR 2024).

   • Mary Phuong and Christoph H Lampert. “The inductive bias of ReLU networks on orthogonally separable data.” ICLR 2021.
   • Yifei Wang and Mert Pilanci. “The convex geometry of backpropagation: Neural network gradient flows converge to extreme points of the dual convex program.” ICLR 2022.
   • Hancheng Min, Enrique Mallada, and René Vidal. “Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization.” ICLR 2024.

2. Regarding Assumption 3.2, we assume that a “clustering” behavior emerges in the pre-trained features, which allows the features to work well in transfer learning (Galanti et al., 2022). This phenomenon is well-documented empirically in the neural collapse literature (Kothapalli, 2023), suggesting that pre-trained features $w_j$ tend to converge around the mean direction for data in class $c(j)$. Assumption 3.2 says that data with positive label (resp. negative) only activates the $j$-th neuron if $j\in F_{+}$ (resp. $j\in F_-$). As a result, any positive data pair $(x,y)\sim(x,y')$ activate the same set of neurons. From a contrastive learning viewpoint, it makes the representations of them semantically similar (Saunshi et al., 2019). Namely, when the features $w_j$ and data inputs $x_i$ are normalized unit vectors, the difference between representations of a positive data pair is bounded by: $\| g(x) - g( x' ) \|_{ \infty } \le \max \cos(w_j,x_i)$ with $y_i=c(j)=y$, which represents the maximum cosine similarity between the features $w_j$ and the data points.

   In conclusion, the form of Assumption 3.2 is consistent with empirical observations from the neural collapse literature and contrastive learning theory. By aligning with these well-documented phenomena, the assumption provides a reasonable and interpretable foundation for our theoretical analysis.

We will clarify these points in our manuscript.

3. On the Zeroth-Order Approximation:

   Review:

   “In addition, the analysis is done for a non-standard simplified Langevin diffusion instead of DP-SGD. The benefits of such an approach seem questionable. The authors ‘Apply a zeroth-order asymptotic expansion’ which sounds very complicated but looks like a simple removal of Brownian motion representing Gaussian noise. This noise is the core component of DP training, and its removal makes method (1) equivalent to simple gradient flow.”

   Response:

   • Please note that our approximation does not remove the effect of noise, nor is the resulting model equivalent to gradient flow. The noise multiplier σ remains explicitly in our convergence bounds. We retain the key noise effects for the loss dynamics by keeping the second-order term from Ito’s lemma in Equation (2) and preserving the second-order terms associated with Brownian motion. This approach allows us to capture the essential stochastic characteristics of DP-SGD without modeling the full noise term directly on the parameters. In essence, this approximation enables us to analyze the expected behavior of parameter updates while preserving the noise-sensitive behavior of the loss itself. By isolating these core elements, we provide insights into the overall training dynamics under differential privacy without losing the major noise effects that influence convergence properties and feature alignment. To support our claim that this approximation does not introduce too much error, we have proved an error approximation guarantee (Theorem F.4 in the updated manuscript), which shows that our approximated model does not differ too much from the original Langevin diffusion model.
   • To complement our results under our approximation, we provided a rigorous theoretical analysis without approximation in Section 4.1.1. This non-approximated model relies on a 2-layer linear network, allowing us to maintain theoretical rigor without the complexities introduced by the approximation. This section presents a practical balance between analytical feasibility and capturing essential dynamics. The approximation allows us to obtain lower bounds for the DP-LP and DP-FFT loss, which are very challenging without the approximation in stochastic analysis.
   • Furthermore, we believe that our work represents a significant step in the theoretical analysis of non-linear activations and multi-layer architectures in DP-SGD—an advancement in the field. Future developments in this area, potentially building on our zeroth-order framework, can further enhance the community’s understanding of DP-SGD dynamics in more intricate and realistic settings.

4. Addressing Approximation Criticisms with Additional Analysis

   Review:

   “In addition, it ignores the crucial per-sample clipping operation, which makes DP deep learning feasible. Thus, the proposed approximation loses the main features of differentially private training. The authors say on line 164 ‘our modeling preserves the noisy behavior characteristic of DP-SGD’ which needs to be justified.”

   Response:

   • In the updated version of our paper (see attached), we provide analysis of Langevin diffusion with clipping. Note that this is the first analysis of clipped Langevin diffusion, and the zeroth order approximation is crucial to the analysis. We also provide the approximation error of the zeroth order approximation under clipping. See “Appendix F: Theory with Clipping”.

   • Theory with clipping and approximation

     • We have updated our paper in openreview and we provide analysis of Langevin diffusion with clipping. The additional theoretical results are: (1) The existence of a unique strong solution of the Langevin diffusion with clipping. (2) The zeroth order approximation error of Langevin diffusion with clipping. (3) A clipping version of Theorem 3.3 (feature distortion).

     We put all these additional results in “Appendix F: Theory with Clipping”.

We appreciate the reviewer’s points regarding the simplicity of our theoretical model and the approximation we use. Below, we clarify why these modeling choices are fundamental to our analysis and not limitations.

1. Exponential Convergence and Model Choice

   Review:

   “It is unclear how this model aligns with realistic applications, particularly since results in Section 4.1.1 suggest exponential convergence, implying that the underlying problem is no harder than strongly convex optimization (or that PL conditions hold for the loss).”

   Response: While it is true that our 2-layer neural network exhibits exponential convergence, the key goal here is not solely to establish convergence rates, but rather to preserve the model’s architecture in a way that captures layer-specific dynamics of DP fine-tuning. Abstracting the convergence as PL conditions on a general function would indeed obscure critical structural details of the two-layer setup, and would not allow us to capture the dynamics of adapting a pre-trained encoder with a linear head. Thus, our focus here is not on the order of convergence guarantees, but on preserving architectural features that are intrinsic to the phenomena we study.

   Additionally, note that exponential convergence is a common characteristic in simplified settings, as seen in Kumar et al.’s analysis of non-private transfer learning and Min et al.’s work on linear networks, yet these models continue to provide insights applicable to complex architectures.

2. Intuition from Simplified Models

   Review:

   “The theoretical model appears oversimplified. Using a 2-layer neural network with ReLU activations while enabling a decoupling of feature learning and classification seems far removed from the practical settings under consideration.”

   Response:

   Two-layer neural networks are widely accepted in the theoretical community as valuable tools for deriving intuition (examples below). Similar to the approach taken by Tian et al. in “Understanding Self-Supervised Learning Dynamics without Contrastive Pairs,” who used a 2-layer model to explore foundational questions in self-supervised learning, our goal here is to identify core principles of representation alignment and feature distortion in DP fine-tuning. Such simplified models allow for clearer insights and serve as a basis for studying more intricate behaviors in deep networks. The theoretical insights derived from these models are supported by empirical results on complex architectures (e.g., Figure 1, Figure 3), demonstrating their relevance to real-world applications. For instance, see below some recent distinguished ML theory papers that use 2-layer NNs for analysis, while studying phenomena that apply empirically to more complex architectures:

   1. (ICML 2021 Outstanding Paper Award Honorable Mention) Yuandong Tian, Xinlei Chen, Surya Ganguli. Understanding Self-supervised Learning Dynamics without Contrastive Pairs.

   2. (ICLR 2022 Oral) Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution.

   3. (NeurIPS 2022 Oral) Yuandong Tian. Understanding Deep Contrastive Learning via Coordinate-wise Optimization.

W1:

“The idea of combining FFT and LP is not entirely novel, as similar approaches were introduced and experimentally tested by Tang et al. (2023). A more detailed discussion of how this paper builds on or extends prior work is needed.”

Response: We acknowledge the reviewer’s point that the concept of combining LP and FFT methods has been explored in prior works, including Tang et al. (2023). Our paper makes distinct theoretical and empirical advancements over existing approaches:

1. Theoretical Foundation: To the best of our knowledge, our work is the first to provide a rigorous theoretical analysis of DP-LP-FFT under a continuous model. Previous work, including Tang et al., primarily focused on empirical evaluations without theoretical insights. By analyzing the dynamics of DP-SGD using a novel approximation technique based on Langevin diffusion, we bridge a significant gap, providing theoretical backing for the observed phenomena in DP fine-tuning across various privacy settings.

2. Broader Scope of Representation Alignment: Kumar et al. show that the out-of-distribution performance of pre-trained features can be distorted in non-private fine-tuning while they leave in-distribution feature distortion as an open question for both non-private and private settings. Unlike prior work, our analysis demonstrates that representation alignment in LP-FFT occurs universally across both private and non-private settings, thereby addressing unresolved theoretical questions posed by Kumar et al. (2021). By highlighting and formalizing this alignment trend as a universal phenomenon, we introduce the concept of "representation alignment" that holds even as data distribution shifts across domains.

   Reference: Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. ICLR 2022 (oral).

3. Empirical Generalization and Mechanistic Insight: Tang et al. investigated LP-FFT in a specific scenario involving synthetic pre-training data. In contrast, our experiments systematically confirm that the trade-offs observed by Tang et al. are generalizable across diverse datasets and architectures. We provide a deeper understanding of this phenomenon by identifying the mechanisms of feature distortion and alignment through visualizations and empirical validations. This approach provides a clearer understanding of how and why LP-FFT performs well in DP contexts, paving the way for a theoretical understanding of privacy-utility trade-offs in differentially private models. We also provided results of LP-LoRA in comparison to LP-FFT in our appendix.

For reference, the ICLR2022 oral paper “Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution” also analyzed LP-FFT. However, the novelty and importance of the theoretical insights provided by Kumar et al. were still recognized despite this observation. We hope that, similarly, our theoretical contributions here will be acknowledged for extending foundational insights into novel, underexplored domains.

Thank you for your initial review. We’ve provided a detailed response to your comments, and since the discussion period deadline is approaching, we’d greatly appreciate your feedback on our response.

Your input is important to further improving the work, and we look forward to hearing your thoughts!

Thank you again for your careful reading of our paper. We really appreciate your time and feedback. Responses to each point are included below.

Most of the theorems presented in the current form are unclear to me, and I suggest adding clarifications in the revision.

Thanks for the feedback. We will add our clarifications to Weakness3 and Weakness5 in the paper.

Numerous references to prior works mentioned in the response look solid but do not justify relevance to real-world applications to me. I would like to recommend adding more concrete, practical recommendations based on the empirical results obtained.

We appreciate the reviewer’s suggestion to emphasize the relevance of our work to real-world applications and to provide practical recommendations. We address this feedback by highlighting new practical insights derived from our empirical results that are different from [Tang et al. 2023]. Per your suggestion, we will emphasize these recommendations more strongly in our paper:

1. Recommendation on fine-tuning methods:

   a. Exploit Sparsity in DP-LoRA: DP-LoRA demonstrates resilience to representation distortion due to its sparse parameter updates (Table 2). DP-LoRA offers a practical solution for achieving privacy without compromising model expressiveness. We are the first to provide a head-to-head comparison of feature distortion between DP-LORA and other DP finetuning techniques, so this is a new practical recommendation.

   b. Leverage DP-LP for Distortion Mitigation: Based on our findings, practitioners dealing with privacy-preserving fine-tuning in settings sensitive to representation distortion (e.g., medical imaging or personalized recommendation) should consider DP-LP-FFT. This approach effectively mitigates the representation distortion observed in DP-FFT while maintaining robust performance.

2. Architecture-Specific Recommendations (Table 1):

   a. CNN-Based Models: Since architectures like MobileNet-v3, ResNet, and WideResNet are more prone to feature distortion (Table 1), practitioners using these models should carefully tune parameters such as noise scale and clipping thresholds to minimize degradation. Additionally, they should consider integrating DP-LP to counteract feature distortion in these networks.

   b. Transformer-Based Models: Transformers empirically exhibit greater robustness against feature distortion, making them a suitable choice for privacy-critical applications with complex feature representations, such as time-series analysis or natural language processing. Fine-tuning these models with DP-FFT or DP-LoRA can yield reliable performance even under stringent privacy constraints.

3. Visualization and Interpretation of Feature Distortion:

   a. Top-1 k-NN Accuracy as a Diagnostic Tool For Feature Distortion (Figure 1, Figure 3): Evaluating feature quality through top-1 k-NN accuracies and applying UMAP to backbone mappings provide a simple yet effective method to interpret feature distortion during fine-tuning. Engineers can use this metric to diagnose and compare the impact of different fine-tuning approaches on feature integrity.

   b. Real-time Monitoring During Fine-tuning (Figure 5): By integrating k-NN accuracy checks into the fine-tuning workflow, practitioners can monitor the model’s feature representation quality in real-time, enabling early intervention and adjustment of training parameters.

Proof of Theorem F.4. misses several steps in the first inequality. Because of this, I think that the result is incorrect as due to the sum over N, the upper bound is expected to scale linearly with N (assuming dataset size due to undefined notation), which makes the result poor from a Differential Privacy perspective.

Thanks for pointing out the missing term $n$ in Theorem F.4. We have updated the theorem statement and the proof. The order of time $t$ and noise scale $\sigma$ remain the same. Please note that the scaling with $n$ can be removed by writing the loss as a sample-wise average, as is commonly done in mini-batch gradient descent. We kept the additive (non-averaged) form for consistency with the main paper. Please also note that the listed scaling with $n$ is the same as in other papers analyzing DP-SGD with Langevin Diffusion, e.g. in (Ye et al., 2023), for the same additive loss expression—this is not an artifact of our model.

Corollary F.7 is claimed as the existence proof of a unique, strong solution for the clipped Langevin diffusion. However, it is not proved that per-sample loss function has finitely many discontinuities, and I think it is incorrect in the general case of non-smooth activations (like ReLU).

We can generalize the statement in Corollary F.7 from “finitely many discontinuities” to “a discontinuity set with Lebesgue measure zero”. Corollary F.7 is based on Theorem F.6. Theorem F.6 only requires the loss function to be measurable, which is a very weak condition and applies for most of the common ML architectures.

Moreover, currently, the basic fundamental properties of clipped Langevin diffusion are not understood, e.g., it is unclear whether it even converges (and if yes, to what) in contrast to standard Langevin diffusion.

The criterion for the existence of a stationary distribution for the SDE is provided in reference [1], offering a deeper insight into its fundamental properties. Importantly, the convergence results rely on continuity rather than differentiability, meaning the non-differentiability of functions such as ReLU and clipping does not affect the validity of the analysis.

• [1] (Theorem 2.1.1, Proposition 2.1.2, Theorem 2.2.1) Sandra Cerrai. Second Order PDE’s in Finite and Infinite Dimension: A Probabilistic Approach. Springer, Berlin, 2002.

Dear Reviewer avt2,

We would like to thank you again for your thoughtful feedback and for helping us improve the paper. Your comments were very helpful, and we appreciate your time. Since the discussion period is coming to a close, we were wondering if you had any thoughts on our latest response and updates?

Thanks so much.

Thanks for your positive feedback!

Weakness:

Review:

“There is a lack of comparison of feature distortion with DP-LP.”

Response:

In the DP-LP (Differentially Private Linear Probing) method, only the final linear layer, or “head,” is updated, while the pre-trained backbone features remain frozen. Since the backbone features are not fine-tuned, feature distortion does not occur under DP-LP. Thus, a comparison of feature distortion with DP-LP is unnecessary, as there is no risk of altering the backbone representations in this method.

For additional context on why freezing features prevents distortion, see:

• Kaiming He, Haoqi Fan, YuxinWu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Computer Vision and Pattern Recognition (CVPR), 2020.
• Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. ICLR 2022.

Question:

“In line 199 , why are subspaces separated by the two cones

Response:

The separation of subspaces follows directly from Assumption 3.1. This assumption leads to the convexity of cones $S_{+}$ and $S_{-}$, each containing all positive and negative data points, respectively. Hancheng Min et al. provide a rigorous proof of this convex cone structure in Appendix C of their paper:

• Hancheng Min, Enrique Mallada, and René Vidal. Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization. ICLR 2024.

Thank you again for your initial review. We’ve provided a detailed response to your comments, and since the discussion period deadline is approaching, we’d greatly appreciate your feedback on our response.

Your input is important to further improving the work, and we look forward to hearing your thoughts!