Sparse-Guard: Sparse Coding-Based Defense against Model Inversion Attacks

Thank you for your detailed review and your comments that the paper offers an "interesting and novel research direction" with many possible extensions. Thank you also for your comments that it is "well motivated" and "well written."

We believe we address your remaining reservations below, and we would appreciate it if you would improve your score in light of our response. Specifically:

Re: limited experiments due to only 2 datasets (MNIST and Fashion MNIST) in datasets:

• We have now doubled the datasets to 4 including the high-res CIFAR-10 dataset and a dataset of medical images that represents a realistic attack setting. We also add a new attack and multiple new defenses as described below. In sum, we have quadrupled the number of experiments. Across all (quadrupled) experiments, Sparse-Guard now outperforms benchmarks by a factor of up to 704. It also has the best PSNR (the most important metric) across every single experiment. In one single experiment, it loses to the Wang et al. defense in terms of SSIM by a factor of 0.001, while significantly outperforming it in terms of PSNR and FID.

Re: evaluation setting is strange because we allow the adversary access to training samples to reconstruct instead of unseen test samples:

• This aspect of our setup is identical to the standard setups in the benchmarks we consider (Titcomb et al. 2021, Gong et al., 2023). Per Section 2, our goal is to consider settings that capture ‘worst-case’ black-box attacks with a powerful attacker. Specifically, We suppose that the attacker builds the inverted model using a held-out set of training data. Then, we suppose that the attacker has access to the training data's intermediate network outputs, which he can submit to his inverted model to reconstruct training examples. Of course, you're right that this setting supposes that the attacker has obtained some leaked data. The fact that Sparse-Guard is robust in this setting suggests that it is a good defense even in the `worst-case' with a powerful attacker. In real-world settings, an attacker may have to approximate the training data's intermediate outputs, but that would make an attack more difficult (and also allow for more subjective decisions in terms of how to set this up fairly for experiments).

Re: evaluating on new MIA's that use GAN's, and evaluating on new defenses:

• Thank you. We added the new Plug and Play attack that you and the other reviewers suggested (the newest of the attacks you suggest), and we also added two new defenses: the Wang et al. defense you suggest and the *new differential-privacy guaranteeing DP-SGD defense of Hayes et al. that is currently under development at Google DeepMind and Meta AI, and that is currently the only defense with provable guarantees for model inversion attacks. Appendix tables 8-13 show all of these new results.
• We run all experiments and all defenses on all 4 datasets (instead of 2) and then re-run them under the Plug-and-play attack. This new set of experiments on [4 datasets, 6 benchmarks, 2 attacks, 3 settings] is now significantly more comprehensive than other recent papers including the Plug and Play paper (which evaluates 3 datasets and 3 benchmarks).
• Across all (quadrupled) experiments, Sparse-Guard now outperforms benchmarks by a factor of up to 704. It also has the best PSNR (the most important metric) across every single experiment. In one single experiment, it loses to the Wang et al. defense in terms of SSIM by a factor of 0.001, while significantly outperforming it in terms of PSNR and FID.

Re: whether the technical contribution is rather low since the approach seems to be simply re-using the sparse coding layer framework of Rozell et al. (2008):

• We are using the new (2022) extension of Rozell sparse coding to convolutional networks, which modifies the the Rozell 2008 update rule (see [1]) to extend it to the convolutional setting. However, Rozell and subsequent sparse-coding architectures including [1] consider a single sparse coded layer. To our knowledge, our paper is the first time an architecture with intermittent sparse layers has been proposed. We compare the 'standard' new convolutional architecture of [1] with our intermittent convolutional architecture (the standard one is called `Sparse-Standard' in all of our tables), and we find that our intermittent architecture significantly outperforms it. We thank you for raising this issue and we have revised our paper to clarify (1) the fact that we are using the new convolution sparse-coding extension rather than the 2008 approach as well as (2) the novelty of our architecture compared to a standard sparse-coding architecture.

    [1] Teti et al., Lateral Competition Improves Robustness Against Corruption and Attack (ICML, 2022)
  

Re: the related work is comprehensive but...." [we should clarify settings/approaches and aspects like images vs. LLM's:

• Thank you for this suggestion -- we have revised this section.

Re: How much longer does it take to train a network using the Sparse-Guard architecture compared to a model without it?:

• Sparse-Guard is equal or faster than benchmarks on all experiments except the case of MNIST under the Plug-and-Play attack, so we include a table of the timings for this 'worst-case' experiment in Appendix table 14. In that 'worst-case' experiment, Sparse-Guard offers a significantly better defense than all benchmarks, and it is still faster than Gong et al.'s GAN-based defense, but its 11% slower than Wang et al.

Re: Why is the FID metric valid?

• FID is one of the standard metrics for the evaluation of inversion attacks, and it is used in (for example) the Plug and Play paper you suggest. It computes whether the reconstructed images are 'overall-distributionally-similar' to the originals by computing their distributional distance, rather than e.g. pixel-by-pixel distance. It also captures an element of how a reconstructed image 'generally looks like' a training image: in other settings, FID is also often used to evaluate whether fake images generated by GAN's 'look like' they could be real.

Re: space after Table 2 should be increased:

• Thank you -- we have fixed this. In general, we noticed many more spacing issues with this year's ICLR template compared to previous years (and also many more than ICML & NeurIPS templates). We are not sure what is causing this, but have looked through the paper to make sure there are no more compressed whitespaces.

Thank you for this update. Re: your comment "However, to accept the paper, I think there should be more empirical proof on more fine-grained datasets (e.g., CelebA)":

• We have re-uploaded our paper with even more experiments on the CelebA dataset (our 5th dataset, and our 3rd added since the original submission). Please see Tables 14 and 15. We note that this CelebA dataset is significantly larger in terms of resolution and image count, so compute times for all benchmarks are significantly greater (and the re-upload deadline is just hours away). Therefore, in the interest of time, we compare Sparse-Guard to the best of the benchmarks (Wang et al.) suggested by the reviewers, rather than all benchmarks. Sparse-Guard outperforms this best defense, and we also show that our performance advantage grows greater under the Plug and Play attack. We are adding remaining benchmarks as the finish computing, but we note that they tend to perform worse than the Wang et al. benchmark that we outperform here.

We ask that you consider improving your score in light of these new experiments, and we thank you again for your many helpful comments that have helped us to make the evaluations in this paper stronger. We strongly feel that a practically effective defense based on a completely novel research direction is of immediate interest both to other researchers and also to companies like Google and Facebook that are actively developing public products for sensitive application domains using available model inversion defenses.

Thank you for your constructive review---we believe we address your remaining reservations and would appreciate it if you would improve your score in light of our response.

The main concern and reason for your score of 3 seems to be whether Sparse-Guard's performance advantage holds against additional defense baselines, additional new attacks, and additional datasets. We now show that it does:

Appendix Tables 10, 11, & 13 add two additional state-of-the-art benchmark defenses:

• We added the Information Regularization defense of Wang et al. suggested by Reviewer ETXH;
• We also added very recent DP-SGD defense of Hayes et al. just released by Deepmind and Meta AI, which is currently the only defense with provable guarantees for model inversion attacks;

Appendix Tables 12 and 13 re-run all experiments using the Plug-and-Play attack you suggested. **Sparse-Guard outperforms benchmarks by a factor of up to 6.7 under this novel attack. In particular, it offers better PSNR (the most important metric in the literature) across all combinations of datasets and defenses.

We also doubled the number of datasets in our experiments. Specifically, [Appendix tables 8-13] rerun all defenses, attacks, and experiment settings with the original 2 datasets as well as:

• The hi-res CIFAR-10 dataset ;
• An additional dataset of medical images.

In sum, we have quadrupled our experiments, and we now show that Sparse-Guard outperforms alternatives across multipple times more combinations of attacks, and defenses, and datasets compared to recent papers such as the Plug and Play paper you suggested.

We also thank you for your comments that our sparse-coding approach is novel, as well as your positive comments about our public model inversion defense PyTorch codebase.

Your 2nd main concern seems to be:

2. Clarifying what is meant by the statement that Sparse-Guard works by “removing unnecessary private information", and clarifying how this statement is compatible with our empirical result that Sparse-Guard unclusters image features:

   We mean to say that the representations that are produced via sparse coding only contain information that is relevant for reconstructing a denoised (a.k.a. less detailed) version of the input, and these representations are much sparser and less detailed than those in standard models. As a result, we were attempting to point out that the attacker has much less information to use when performing model inversion attacks against Sparse-Guard compared to standard models. Put differently, a 'perfect attacker' could only ever hope to reconstruct a sparse representation of an input image under our defense in the worse case, meaning that even a perfect attacker could not hope to reconstruct many input image details that are dropped by sparse-coding. By 'unnecessary private information', we mean input image details that we may desire to keep private but that are not necessary for the task of training a high-accuracy classifier. Specifically, sparse-coding networks are known to sparsify inputs and network layers without losing significant classification accuracy.

   The unclustering effect we observe in our empirical analysis is the direct result of this approach. Specifically, a standard (non-sparse) network trained on dense/noisy/detailed input images results in many partly-redundant network features that are `noisy versions of each other'. Networks with many noisy/partly redundant features are easier to attack, because an adversary can 'home in on' clusters of such features in order to 'home in on' an input image it wants to reconstruct. Our sparse-coding approach's de-noising effect tends to prevent the network from learning many partly-redundant features, resulting in the de-clustering effect we observe in Fig. 4. The resulting de-clustered features are intuitively less susceptible to gradient-based inversion attacks.

We thank you for these helpful suggestions and we ask you to also consider our vastly expanded experiments section including new datasets as well as new attacks & defenses [Appendix tables 8-14] when updating your review score.

We thank you for your positive review. We appreciate your suggestions re: how to improve clarity, and we have updated the descriptions in our paper accordingly. We believe this addresses your reservations, and we would appreciate it if you would improve your score in light of our response.

We thank you for:

• Your "well-written and organized" comment;
• Your comment re: the novelty of our contribution;
• Your positive comments re: our empirical UMAP explorations and our public PyTorch codebase.

Your 2 main concerns seem to be:

1. Clarifying the description of our Sparse-Coding implementation:

   Thank you for this constructive feedback---We have greatly revised Section 3: Sparse-Guard Architecture where we introduce sparse coding and the LCA algorithm to provide a much more clearer picture of our methods. In particular, the update rule provided in equation 3 describes the Rozell update for the convolutional case, which was used in previous work [1,2] and validated against other non-Rozell-based convolutional sparse coding solvers (https://sporco.readthedocs.io/en/latest/examples/csc/index.html). Although the original Rozell sparse coding algorithm was not introduced in the convolutional setting, it can readily be adapted to convolutional networks because it is based on the general principle of feature-similarity-based competition between neurons, which is agnostic to different neural network architectures. As a result, the original Rozell formulation can be readily adapted to the convolutional setting by modifying just two terms in the update rule. The first term, which is represented as $\Psi(t)$ in Equation 2 of our revised paper, is computed via a matrix multiplication between the transposed dictionary matrix and an input vector in the original Rozell formulation. In the convolutional case, the matrix multiplication becomes a convolution between the input and the convolutional dictionary. The second term that needs to be adapted for the convolutional setting is where the similarity between each feature and every other feature is computed, given by $\Omega * \Omega$ in our revised version. In the original Rozell formulation, this was done with a matrix multiplication between the transposed dictionary and the non-transposed dictionary. In the convolutional setting, it can be done with a convolution between the dictionary and itself. We also agree with the reviewer that the general 'spatiotemporal' description we referred to was potentially confusing in our setting so we have replaced it with a more precise description. We thank the reviewer for pointing out the opportunity to improve our description of our sparse-coding approach.

   • [1] Teti et al., Lateral Competition Improves Robustness Against Corruption and Attack
   • [2] Kim et al., Modeling Biological Immunity to Adversarial Examples

Thank you for your constructive review. We believe we address your remaining reservations here and would appreciate it if you would improve your score in light of our response.

The main concern and reason for your score of 3 seems to be your statement that "The major drawback is that the experiments are only conducted on simple, low-resolution datasets." We've taken your advice and:

• Re-run all experiments on CIFAR-10 as well as an additional dataset of medical images [Appendix Tables 8-13]. Sparse-Guard outperforms benchmarks by a factor of up to 173.9 on CIFAR-10.
• Re-run all experiments on all 4 datasets using the Plug-and-Play attack you suggested [Appendix Tables 12-13]. Sparse-Guard outperforms benchmarks by a factor of up to 6.6 under this attack.
• We also added 2 more state-of-the-art defense benchmarks including the new differential-privacy guaranteeing DP-SGD defense from Google DeepMind and Meta AI [Appendix Tables 10, 11, & 13].

Across all (quadrupled) experiments, Sparse-Guard now outperforms benchmarks by a factor of up to 704. It also has the best PSNR (the most important metric) across every single experiment.

We absolutely agree with your conclusion that "an effective defense would be significant" on CIFAR-10 and similar hi-res data, and we believe we have addressed this concern.

We also thank you for your comments that the paper is "well motivated and easy to follow" and your positive comments about our PyTorch codebase.

Dear authors,

thanks for the rebuttal, additional experiments, and clarifications. Just a small remark: Plug and Play Attacks are not black-box but white-box since the attack exploits a model's gradients to optimize the GAN's latent vectors.

We have re-uploaded our paper with even more experiments on the CelebA hi-res celebrity faces dataset suggested by Reviewer ETXH (this is our 5th dataset, and our 3rd added since the original submission). Please see Appendix Tables 14 and 15. We note that this CelebA dataset is significantly larger in terms of resolution and image count, so compute times for all benchmarks are significantly greater (and the re-upload deadline is just hours away). Therefore, in the interest of time, we compare Sparse-Guard to the best of the benchmarks (Wang et al.) suggested by the reviewers, rather than all benchmarks. Sparse-Guard outperforms this best defense, and we also show that our performance advantage grows greater under the new Plug and Play attack (Table 15). We are adding remaining benchmarks as the finish computing, but we note that they tend to perform worse than the Wang et al. benchmark that we outperform here.

We strongly feel that a practically effective defense based on a completely novel research direction is of immediate interest both to other researchers and also to companies like Google and Facebook that are actively developing public products for sensitive application domains using available model inversion defenses.