Federated Unlearning with Diffusion Models

It is not clear why minimizing the Frobenius norm of the attention map for the sensitive objective helps with unlearning.

Writing needs to be improved. Some sentences are difficult to parse. Some symbols or equations are inaccurate. For instance, in Eq (2), sum from 0 to N means there are N+1 numbers of clients in total. In Theorem 1, the right hand side of Eq. (9) is dependent on i, whereas the left hand side is the sum of losses across all clients. The format of citations also needs to be fixed throughout the paper.

The definition (or granularity) of unlearning needs to be clarified at the beginning of the paper to set up the context—whether it targets at unlearning at the client level, or local sample level, or some feature level.

I think Theorem 1 and 2 are observations, as opposed to theorems. Lemma 1 is a restatement of previous results without any citations in the main text.

It is not clear if the federated unlearning setup involves any personalization, i.e., if the goal is to learn a single global model that forgets about individual clients’ samples, or to learn personalized models where each client model forgets about that client’s unlearnt data. While in practice, personalization could be achieved simply by performing some local finetuning, the two problems still correspond to different techniques and reasonings.

• The theoretical part seems cannot support the claim such as "leverage traditional federated learning methods to address federated unlearning issues".
• It is unclear (the rationale behind) why considering the contrastive loss between the two attention maps, not directly adopting existing machine unlearning methods such as random labeling or assigning to the safe target.
• Current experiments are not enough to support the conclusion.

Please see Questions for details.

1- I am not sure from the novelty of the proposed idea compared with the federated unlearning algorithms in the literature. From my point of view, the paper is doing a similar technique of maximizing (locally) the KL divergence and then average the local model.

2- I am not convinced with the loss function in Eq(3), Eq(4), and Eq(8). Maybe the KL divergence of the distiribtuion $p_{\theta_p}(.|C_i)$ and $p_{\theta_p}(.|C_T)$ are already large. I think it make more sense to maximize the difference [D_KL(on C_i) - \alpha D_KL (C_T)] which means unlearning task C_i and learn the task C_T.

3- The Lemmas and theorems are trivial and don't provide any insights. For example, Theorem 2 can be proven in one line expansion of the KL divergence from Lemma 1. What is the upper bound in Lemma 1? There is always a trivial upper bound on the KL divergence. What are the implications and the importance of Theorem 1 and Theorem 2?

• The overall writing quality could be improved; there a few typos (e.g., line 053, an novel; line 219, it does not related), the paper does not use parenthesis for in-text citing, some figures are not commented (e.g., Figure 2 and Figure 3).
• The theoretical results and mathematical formulation are far from being sound:
  • Equation 2 is not well defined; what does it mean to solve an optimization for all $t \in \{1, \dots, T\}$? does it mean that we are solving $T$ optimization problem?
  • The optimization problem in (2) aims at maximizing the divergence between two conditional distributions, without having any further constraints. This formulation does not much the goal described in the paper, as it does not constrain the target distribution to have a good generative power on the non-private concepts.
  • Similar problems as above hold also for problem (3).
  • Theorem 1 is trivial.
  • I doubt the correctness of Lemma 1. For example, when $p_{\theta_p}(x_j)$ is the normal distribution, Lemma 1 implies that both $\theta_g$ and $\theta_g$ induce the same distribution. Further justification and explanation is needed.
  • In lines 308--310, the paper claims an equivalence between (2) and (3) without proving it.
  • nit: $\sum$ is usually reserved for discrete integration, while $\int$ is used in the non-discrete case.
  • The paper promises detailed proofs of the theoretical results, but (in my opinion) does not fulfill this promise.
• I do not really understand the motivation behind the clustered aggregation scheme in Section 3.3. For example, if all clusters have the same size, this aggregation would be equivalent to a normal aggregation.