Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity

1. Defend Adversarial Attack

Our framework effectively defends against Model Inversion Attacks (MIA) [1] by preventing the reconstruction of the target unlearn features. As shown in Tab 3 in main paper, the attack success rate with our framework is similar to that of the naive retrain method, indicating that unlearned features are protected. In contrast, the baseline model before unlearning has a 90% attack success rate, highlighting severe data privacy risks. For other baselines like FedCDP and FedRecovery, the success rate is about 80%. Our Ferrari framework reduces this rate to around 50%, significantly enhancing privacy. Tab 4 shows that our framework prevents reconstruction of the target unlearned feature (e.g., the mouth), whereas the baseline model does not.

Ferrari effectively defends against MIA [1, 2] by preventing the reconstruction of the target unlearn features. As shown in Tab 3 (main paper), the attack success rate with our framework is similar to that of the naive retrain method, indicating that unlearned features are protected. In contrast, the baseline model before unlearning has a 90% attack success rate, highlighting severe data privacy risks. For other baselines like FedCDP and FedRecovery, the success rate is about 80%. Ferrari reduces this rate to around 50%, significantly enhancing privacy. Tab 4 shows that our framework prevents reconstruction of the target unlearned feature (e.g., the mouth), whereas the baseline model does not.

Ref:

[1] Fredrikson et al, “Model inversion attacks that exploit confidence information and basic countermeasures,” CCS ’15, (New York, NY, USA), p. 1322–1333, Association for Computing Machinery, 2015

[2] Y. Zhang et al, “The secret revealer: Generative model inversion attacks against deep neural networks,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 250–258, 2020

2. Computational Complexity

We discussed time efficiency in Section 5.4 (main paper). Additionally, we present the computational complexity of our proposed Ferrari framework using FLOPs metrics, as shown in Fig 5 of the rebuttal PDF. Our Ferrari framework exhibits the lowest FLOPs, achieved by accessing only the local dataset of the unlearned client and minimizing feature sensitivity within a single epoch.

3. Theoretical Assumption

We explain more on Assumption 1-2.

• First, Assumption 1 elucidates that the utility loss associated with a perturbation norm less than $C$ is smaller than the utility loss when the perturbation norm is greater than $C$. This assumption is logical, as larger perturbations would naturally lead to greater utility loss.

• Second, we can also extend the Theorem 1 to the case of the non-zero training loss. This is because

$

\mathbb{E}_{(x,y)\in \mathcal{D}} \left( || f_{\theta^}(x) - y || + \lambda \mathbb{E}_{|| \delta_F || \geq \lambda} \frac{||f_\theta(x) - f_{\theta^} (x + \delta_{F})||_2}{||\delta_{F}||_2} \right) \

$

$

= \mathbb{E}_{(x,y)\in \mathcal{D}} \mathcal{l}|| f_{\theta^}(x) - y || + \lambda \mathbb{E}_{(x,y)\in \mathcal{D}} \mathbb{E}_{|| \delta_F || \geq \lambda} \frac{||f_\theta(x) - f_{\theta^} (x + \delta_{F})||_2}{||\delta_{F}||_2}

$

$

\leq \mathbb{E}_{(x,y)\in \mathcal{D}} \mathcal{l} || f_{\theta^}(x) - y || + \mathbb{E}_{(x,y)\in \mathcal{D}} \mathbb{E}_{|| \delta_F || \geq \lambda} ||f_\theta(x) - f_{\theta^} (x + \delta_{F})||_2

$

$

\leq \mathbb{E}_{(x,y)\in \mathcal{D}} \mathbb{E}_{|| \delta_F || \geq \lambda} ||y - f_{\theta^*} (x + \delta_{F})||_2

$

which obtain Eq. (13) in appendix (noted that $\|\delta_\mathcal{F}\| \geq \frac{1}{\lambda}$ in original appendix should be $\|\delta_\mathcal{F}\| \geq {\lambda}$).

4. Future Work

We identify key areas for future research on federated feature unlearning with Ferrari:

• Dataset Access: Explore methods that require less than 70% of the unlearning client's dataset for effectiveness.

• Model Types: Investigate the approach's applicability to non-classification models, such as generative models.

• Advanced Perturbation: Develop advanced perturbation techniques to improve unlearning effectiveness.

• Data and Model Diversity: Support a wider range of data types and models.

• Privacy Integration: Integrate federated feature unlearning with other privacy-preserving methods for enhanced data protection.

Future work should focus on minimizing dataset requirements, broadening model applicability, and improving robustness and privacy guarantees.

1. Challenge of paper

Thank you for your suggestions. We clarify the second challenge from the original manuscript as follows:

Firstly, previous feature unlearning methods [1, 2] in centralized settings are impractical for Federated Learning (FL), as they require access to all datasets and client participation. In FL, retained clients may refuse to participate due to privacy or computational concerns. Therefore, the most practical approach requires only the participation of the unlearn client.

Secondly, existing approaches to Federated Unlearning primarily focus on unlearning at the client, class, or sample level. However, these methods do not address the challenge of feature unlearning in federated settings. They are limited to isolated data points and cannot manage the removal of specific features across multiple data points. Our results highlight that benchmark Federated Unlearning frameworks like FedCDP [3] and FedRecovery [4] struggle with effective feature unlearning in FL environments.

Ref:

[1] Warnecke et. al, “Machine unlearning of features and labels,” in Proc. of the 30th Network and Distributed System Security (NDSS), 2023

[2] Guo et. al, "Efficient attribute unlearning: Towards selective removal of input attributes from feature representations." arXiv preprint arXiv:2202.13295 (2022).

[3] Wang et. al, “Federated unlearning via class-discriminative pruning,” in Proceedings of the ACM Web Conference 2022, WWW ’22, (New York, NY, USA)

[4] Zhang et. al, “Fedrecovery: Differentially private machine unlearning for federated learning frameworks,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 4732–4746, 2023

2. Unlearning Concern

Thanks for you comments. First, federated feature unlearning happens at the end of the federated training. When the federated training ends, unlearned client implements the unlearning operation to obtain the unlearned model $\theta_u$ finally the server replace the original model with $\theta_u$.

Second, in the case of IID data, it is valuable to unlearn features. For instance, a client (Alice) may want to remove the "marital-status" feature from adult data. Even if other clients have the same feature, Alice still needs to remove this information (marriage status).

Third, the experimental results in Tabs 2 and 3 (main paper) demonstrate that the proposed Ferrari can successfully remove the feature of one client, even in the IID setting.

Finally, beyond the IID scenario, our framework remains effective for feature unlearning in Non-IID scenarios, as illustrated in Fig 3 of the rebuttal PDF. The results show that the Ferrari significantly improves feature unlearning performance, with only a 0.2% drop in $D_u$ when $\gamma = 1$ compared to the IID scenario. Furthermore, considering the backdoor unlearning scenario, where backdoor data from one client has a different distribution compared to the normal data.

3. Text Dataset

Thank you for your suggestion. As suggested, we conducted text classification using the IMDB movie reviews dataset [1] with a BERT model to analyze feature unlearning by removing the influence of sensitive features, specifically celebrity names. Each client’s local dataset includes only a specific name of a celebrity. As to Table below, Ferrari framework proves highly effective, maintaining high model accuracy while achieving the lowest feature sensitivity and MIA attack success rate, thus preventing the reconstruction of sensitive features and enhancing data privacy.

• Model Utility Analysis:

• Feature Sensitivity Analysis:

• Model Inversion Attack Analysis:

Ref:

[1] https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews

5. Minor Issues

• The repetitive references have been removed in our manuscript, preventing confusion for the reader.

• Thank you for your suggestion. We will make the necessary changes to enhance the clarity of our work.

• Thank you for your suggestion. We will request permission from TPC to update the short title of this paper. If approve, we will update the title accordingly.

• Repeated phrases in line 105 has been removed to improve the readability.

• We corrected the unlearned feature set $F$ to $F_i$ in Equation 2 (main paper)

1. Global Model Response

We sincerely apologize for any confusion in our previous communication.

In scenarios with IID, unlearning a feature from a target client will inadvertently lead to unlearning similar features in other clients within the global model. For example, we use a face classification task to assess if unlearning 'mouth' feature for one client impacts the model's sensitivity to the 'mouth' feature in the other nine clients.

• (i) In Table 1, the feature sensitivity for unlearned client ($D_u$) and retained client ($D_r$) is significantly reduced (e.g., dropping below 0.11), compared to values > 0.95 before unlearning.

• (ii) In Table 2, the successful Membership Inference Attack (MIA) rate for features from unlearned client ($D_u$) and retained client ($D_r$) decreases to approx. 50%, whereas it was > 80% before unlearning.

Table 1: Feature sensitivity analysis in FL including 10 clients under IID, a small feature sensitivity indicates good unlearning effect | Dataset | Unlearn Feature | $\mathcal{D}_u$ (Before Unlearning) | $\mathcal{D}_r$ (Before Unlearning) | $\mathcal{D}_u$ (Retrain) | $\mathcal{D}_r$ (Retrain) | $\mathcal{D}_u$ (Ours) | $\mathcal{D}_r$ (Ours) | |-------------|---------------------|-----------------------------------------|-----------------------------------------|-------------------------------|-------------------------------|----------------------------|----------------------------| | CelebA | Mouth | 0.96 ± 1.41×10⁻² | 0.95 ± 1.63×10⁻² | 0.07 ± 8.06×10⁻⁴ | 0.08 ± 7.51×10⁻⁴ | 0.09 ± 3.04×10⁻⁴ | 0.11 ± 3.62×10⁻⁴ |

Table 2: Model inversion attack in FL including 10 clients under IID, a small MIA indicates good unlearning effect | Dataset | Unlearn Feature | $\mathcal{D}_u$ (Before Unlearning) | $\mathcal{D}_r$ (Before Unlearning) | $\mathcal{D}_u$ (Retrain) | $\mathcal{D}_r$ (Retrain) | $\mathcal{D}_u$ (Ours) | $\mathcal{D}_r$ (Ours) | |-------------|---------------------|-----------------------------------------|-----------------------------------------|-------------------------------|-------------------------------|----------------------------|----------------------------| | CelebA | Mouth | 84.36 ± 3.22 | 84.17 ± 2.93 | 47.52 ± 1.04 | 48.36 ± 1.39 | 51.28 ± 2.41 | 52.46 ± 2.76 |

2. Global Sensitivity Reduction

Thank you, we appreciate the opportunity to clarify further.

Firstly, under IID setting in federated learning (FL), we recognize the concern regarding the removal of sensitive features from the global model, especially when two clients have similar or identical features and only one requests unlearning. However, it is important to note that retaining such a sensitive feature in the global model may lead to potential violations of regulations such as GDPR. Thus, we argue that removing the feature from the global model is a reasonable and necessary step to ensure compliance. Also, as demonstrated in Table 1 (main paper), the removal of sensitive features for unlearned clients does not significantly diminish the model's utility for the remaining clients. Thus, under IID setting, our unlearning strategy offers a practical and acceptable solution that balances compliance with maintaining the effectiveness of the global model.

Secondly, unlearning in other scenarios within FL, such as Non-IID, backdoor unlearning, and bias unlearning, our proposal holds particular value. In Non-IID and backdoor, individual clients often possess personalized features that are unique to them, unlike the shared features in the IID setting. For bias unlearning, the shared objective across all clients is to remove bias from the global model. As shown in Sections 5.3.2 - 5.3.3 (main paper), Ferrari performs effectively in unlearning bias and backdoor features; as well as in Non-IID setting (please refer Fig. 3 in the rebuttal PDF).

3. Unseeing as unlearning

Thank you. The feature unlearning herein is equal to "unseeing" it during training instead of unfitting it. According to [1], exact feature unlearning can be similarly defined as: $\theta_u \sim \theta_r$, where $\theta_u$ is the unlearned model and $\theta_r$ is the retrained model using data $x+\delta_{\mathcal{F}}$. Our proposed aims to optimize:

$

\min_\theta \mathbb{E}_{(x,y) \in \mathcal{D}_u} \frac{1}{N} \sum_{i=1}^N \frac{||f_\theta(x) - f_\theta (x + \delta_{F,i})||_2}{||\delta_{F,i}||_2}

$

This objective drives the unlearned model to converge toward the retrained model, effectively making the unlearned model unsee the feature.

[1] Bourtoule et al., “Machine unlearning,” in 2021 IEEE SP.

1. Non-Lipschitz Analysis

We evaluate the Lipschitz loss function for optimizing feature sensitivity (Equation 6) and compare it with a variant lacking the denominator, termed the Non-Lipschitz loss (see Fig 1 of the rebuttal PDF). Results show that models using the Non-Lipschitz loss exhibit batch fluctuations and ineffective performance due to unbounded optimization, leading to catastrophic forgetting (Fig 7(c) in main paper). In contrast, models optimized with the Lipschitz loss achieve a steady reduction in feature sensitivity, effectively unlearning target features while maintaining model utility, as guaranteed theoretically (Section 4.3 of main paper)

2. Figure 7 Clarification

In Fig 7, the solid line represents the retain client dataset $D_r$ while the dotted line represents the unlearn client local dataset $D_u$ in Federated Learning scenario.

3. Figure 6 Clarification

The retrain model's runtime was measured by training the global model from scratch for 20-25 epochs until convergence, while the fine-tune model involved 5 epochs of fine-tuning the baseline model. In comparison, the Ferrari framework achieved local feature unlearning in just one epoch, resulting in significantly faster runtime than both the retrain and fine-tune methods.

4. Minor Issue

To address space constraints, we combined the ablation study and hyper-parameter analysis into Section 5.5, renamed "Ablation and Hyper-parameter Analysis." This section separately discusses the Non-Lipschitz component and the Gaussian noise level and number of unlearned datasets. Additionally, we will correct all single and double quotes to the proper format.

5. Figure 7b Clarification

Sorry for the confusion. Actually, Fig 7(b) intention is to demonstrate that our framework requires only at least 70% of the unlearned client's local dataset to achieve optimal performance comparable to using the entire dataset. This highlights the framework's robustness and practicality, as it remains nearly optimal even when only 70% of the dataset is available due to data loss.

We sincerely appreciate your thoughtful feedback and for upholding the 8 = Strong Accept score. Your insightful suggestions have greatly improved the clarity and overall quality of our work. Thank you for your valuable support.

1. Motivation

Thank you for raising this important question about the motivation for feature unlearning in Federated Learning (FL). In the case of IID data, where clients may share similar features, it is indeed valuable to consider the need for unlearning specific features. For example, a client may request the removal of the "marital-status" feature from their dataset. Even if other clients have the same feature, it remains essential for this client to remove any information related to their marital status. This underscores the significance of offering feature unlearning to address individual privacy concerns.

Furthermore, as demonstrated by the experimental results in Tabs 2 and 3 (main paper), our proposed approach, Ferrari, can successfully remove a specific feature for one client even in an IID setting. This capability highlights Ferrari's effectiveness in addressing individual feature unlearning requests without compromising the global model's integrity.

Beyond the IID scenario, we also consider the Non-IID case. One such example is addressed in response to your fourth question. Additionally, we explore the unlearning backdoor scenario, where backdoor data from one client has a different distribution compared to normal data. This further illustrates the necessity of feature unlearning to ensure model integrity and security in diverse data distributions.

Overall, while data erasure provides a high level of user safety, feature unlearning offers a tailored approach to meet individual privacy needs, making it a crucial aspect of Federated Learning.

2. Proof

According to [1], exact feature unlearning can be similarly defined as: $\theta_u \sim \theta_r$, where $\theta_u$ is the unlearned model and $\theta_r$ is the retrained model using data $x+\delta_{\mathcal{F}}$. The proposed Ferrari framework aims to optimize

$

\min_\theta \mathbb{E}_{(x,y) \in \mathcal{D}_u} \frac{1}{N} \sum_{i=1}^N \frac{||f_\theta(x) - f_\theta (x + \delta_{F,i})||_2}{||\delta_{F,i}||_2}

$

This objective leads the unlearned model to approach the retrained model. In particular, when this loss approaches zero, the proposed Ferrari framework achieves exact unlearning.

Ref:

[1] Bourtoule et. al, “Machine unlearning,” in 2021 IEEE Symposium on Security and Privacy (SP), pp. 141–159, 2021

3. Utility Loss

Our method is expected to outperform the "naive retrain" method in sensitive feature unlearning scenario. This is because adding Gaussian noise to the target unlearn feature region during retraining reduces model utility, as shown in Fig 1 (main paper). Equation (9) theoretically proves that our framework results in less utility loss than naive retraining, since larger loss leads to lower model performance.

4. Non-IID

Our proposed framework remains effective for feature unlearning in Non-IID scenarios, as illustrated in Fig 3 of the rebuttal PDF. The results show that the Ferrari significantly improves feature unlearning performance, with only a 0.2% drop in $D_u$ when $\gamma = 1$ compared to the IID scenario. Additionally, it maintains $D_r$ accuracy with only a slight 2% decrease compared to the Retrain method in the Non-IID scenario.

5. MIA Privacy

Ferrari effectively defends against Model Inversion Attacks (MIA) [1, 2, 3] by preventing the reconstruction of the target unlearn features. As shown in Tab 3 (main paper), the attack success rate with our framework is similar to that of the naive retrain method, indicating that unlearned features are protected. In contrast, the baseline model before unlearning has a 90% attack success rate, highlighting severe data privacy risks. For other baselines like FedCDP and FedRecovery, the success rate is about 80%. Ferrari reduces this rate to around 50%, significantly enhancing privacy. Tab 4 (main paper) shows that our framework prevents reconstruction of the target unlearned feature (e.g., the mouth), whereas the baseline model does not.

Ref:

[1] M. Fredrikson et al, “Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing,” Proceedings of the USENIX Security Symposium. UNIX Security Symposium, vol. 2014, pp. 17–32, 2014

[2] Fredrikson et al, “Model inversion attacks that exploit confidence information and basic countermeasures,” CCS ’15, (New York, NY, USA), p. 1322–1333, Association for Computing Machinery, 2015

[3] Y. Zhang et al, “The secret revealer: Generative model inversion attacks against deep neural networks,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 250–258, 2020

6. MIA Metrics

Thank you for your enquiry. First, Model Inversion Attack (MIA) reconstructs the data from the unlearned model. We use the attack success rate of MIA to indicate whether the reconstructed data exposes the feature, as used in [1, 2, 3]. Second, besides MIA evaluation, we also leverage backdoor accuracy, Feature Sensitivity, and visualization tools (GradCAM) to evaluate the unlearning effectiveness of different methods in Section 5.1 (main paper).

The references are the same as the references in question 5 above.

7. Baselines

Our study compares against benchmark baselines in federated unlearning, such as FedCDP for class unlearning and FedRecovery for client unlearning, due to the absence of comparable baselines for feature unlearning in Federated Learning. This highlights the novelty and significance of our work. Our results show that current methods like FedCDP and FedRecovery are ineffective for specific feature unlearning scenarios, emphasizing our study's contribution.

Thank you for your thoughtful feedback. We are pleased to hear that our rebuttal has addressed most of your concerns. We are truly grateful for your decision to revise and improve the rating based on our clarifications. Your willingness to acknowledge the revisions means a great deal to us, and we are committed to addressing any further concerns you may have. We are more than happy to engage in any additional dialogue necessary to ensure that our paper meets the highest standards of quality.

Once again, thank you for your invaluable insights and for your thoughtful consideration in upgrading the score.

1. Large-Scale Dataset

Thank you. As requested, we conducted additional experiments using the ImageNet dataset to assess the generalization ability of our proposed framework. Below are the tables summarizing the results. These results demonstrate that our framework performs well even in large-scale environments, confirming its robustness and ability to generalize effectively. Specifically, the results show low $D_u$ accuracy while maintaining high test accuracy, indicating high model utility.

• Model Utility Analysis:

• Unlearning Effectiveness Analysis:

2. Computational Overhead & Scability

Thank you. As requested, We present the computational complexity of Ferrari using FLOPs metrics, as shown in Fig 5 of the rebuttal PDF. Our Ferrari framework exhibits the lowest FLOPs, achieved by accessing only the local dataset of the unlearned client and minimizing feature sensitivity within a single epoch.

For scalability, Ferrari remains effective in large-scale FL environments, as shown in the client number analysis in Fig 4 of the rebuttal PDF. It exhibits less than 0.5% accuracy deterioration for the retain client dataset $D_r$ and less than 0.1% for the unlearn client dataset $D_u$.

3. Theoretical Analysis

Thank you and we will elaborate more on Assumption 1-2.

• First, Assumption 1 elucidates that the utility loss associated with a perturbation norm less than $C$ is smaller than the utility loss when the perturbation norm is greater than $C$. This assumption is logical, as larger perturbations would naturally lead to greater utility loss.

• Second, we could also extend the Theorem 1 to the case of the non-zero training loss. This is because

$

\mathbb{E}_{(x,y)\in \mathcal{D}} \left( || f_{\theta^}(x) - y || + \lambda \mathbb{E}_{|| \delta_F || \geq \lambda} \frac{||f_\theta(x) - f_{\theta^} (x + \delta_{F})||_2}{||\delta_{F}||_2} \right) \

$

$

= \mathbb{E}_{(x,y)\in \mathcal{D}} \mathcal{l}|| f_{\theta^}(x) - y || + \lambda \mathbb{E}_{(x,y)\in \mathcal{D}} \mathbb{E}_{|| \delta_F || \geq \lambda} \frac{||f_\theta(x) - f_{\theta^} (x + \delta_{F})||_2}{||\delta_{F}||_2}

$

$

\leq \mathbb{E}_{(x,y)\in \mathcal{D}} \mathcal{l} || f_{\theta^}(x) - y || + \mathbb{E}_{(x,y)\in \mathcal{D}} \mathbb{E}_{|| \delta_F || \geq \lambda} ||f_\theta(x) - f_{\theta^} (x + \delta_{F})||_2

$

$

\leq \mathbb{E}_{(x,y)\in \mathcal{D}} \mathbb{E}_{|| \delta_F || \geq \lambda} ||y - f_{\theta^*} (x + \delta_{F})||_2

$

which obtain Eq. (13) in appendix (noted that $\|\delta_\mathcal{F}\| \geq \frac{1}{\lambda}$ in original appendix should be $\|\delta_\mathcal{F}\| \geq {\lambda}$).

4. LR & Gaussian Noise

We considered the sensitivity of Gaussian noise levels in our proposed framework, as shown in Fig 7(a) (main paper). Within the range from 0.05 to 1.0, retain client dataset $D_r$ accuracy stays high and unlearn client dataset $D_u$ accuracy remains low, indicating a balance. Thus, we implement noise value within this range in our proposed framework for balanced accuracy across $D_r$ and $D_u$.

In Fig 2 of the rebuttal PDF, we analyze the learning rate hyper-parameter for Ferrari, ranging from 0.00001 to 0.01. Our findings show that a larger learning rate slightly decreases accuracy on both $D_r$ and $D_u$, while a lower learning rate slightly improves accuracy. A learning rate of 0.0001 provides a satisfactory balance, with less than a 2% change in accuracy for $D_r$ and less than a 0.2% change for $D_u$. Thus, we selected 0.0001 for all experiments in this study.

Therefore, the unlearning effectiveness of our proposed framework is not sensitive to the learning rate but is sensitive to the level of Gaussian noise.

5. Client Participation Affect

Thank you for your suggestion. We would like to clarify that our proposed framework remains effective even when clients drop out or join during training or unlearning. This robustness is due to our framework’s reliance solely on the participation of the unlearning client and their local dataset for local feature unlearning. As demonstrated in Fig 2 and Alg 1 of our main paper, our framework minimizes feature sensitivity, ensuring effective unlearning regardless of broader client participation. This adaptability is a key advantage of our proposal.

6. Minor Mistake

Thank you for pointing out the minor mistakes and typos in the paper. We have carefully reviewed the manuscript and corrected these issues.