Gradient Inversion Transcript: A Generative Model to Reconstruct Training Data by Gradient Leakage

Answer to Weakness 1. Practical Scenario

To the best of our knowledge, most existing works of training data reconstruction also assume that the attacker is able to intercept gradients transmitted from client nodes to the server but not the model weights. [2], [3], [4] and [5] assume gradient leakage and model querying, which can be seen as a gray box. [1] assumes gradient leakage and part of training data, which can be seen as a black box. These two assumptions are both reasonable and based on different attacker's access to the target model, gray box with multiple querying and black box with several input-gradient pairs, are both able to steal information from the model.

Weights leakage, in contrast, represents a stronger assumption as it reveals the complete model information. This essentially transforms the Federated Learning (FL) model into a white-box scenario, enabling the derivation of a closed-form reconstruction algorithm ([6], for example, assumes both gradient leakage and weights leakage, regarded as a white box, where the model information is totally leaked).

In our work we consider a more relaxed assumption compared to weights leakage, where attacker intercepts gradients transmitted from client nodes to the central server and additionally breaches one client node to gain access to its local dataset. In addition, it is noteworthy that if model querying is allowed, we can use public data to generate input-gradient pairs and there is no need for obtaining the real leaked data.

Answer to Weakness 2. Stronger Assumption

It is challenging to determine which assumption is stronger, as the strength of an assumption depends on the amount of leaked training data. With sufficient training data, a closer estimation of the model's information can be achieved, albeit at a higher cost.

DAGER primarily focuses on large language models (LLMs), which is interesting but falls outside the scope of this paper. On the other hand, SPEAR operates under stronger assumptions, requiring the gradients of biases for all layers and no activations are allowed except in the final layer. However, it addresses a critical issue commonly faced in training data reconstruction from gradient leakage: the information loss caused by large batch sizes. While SPEAR is an instructive study, large batch size reconstruction is not the focus of our work, and the assumption of attacker's access is too strong in SPEAR compared with our settings.

Instead, we concentrate on achieving optimal recovery performance with the same model complexity under weaker assumptions by designing a robust and efficient attack model structure.

Answer to Editorial Comments

We agree that changing "threat model" to "attack model" or directly "GIT" would better reflects its meaning in the given context. Other modifications for editorial comments please refer to revision.

Answer to Question 1. Scenario

In federated learning settings like Federated Averaging (FedAvg), during a communication round, each client trains the model on its local dataset. The transmitted gradients are then intercepted by the attacker, and one client node is breached to the attacker so that part of the training data is leaked.

Answer to Question 2. Architecture

Based on the backpropagation, our methods (both CoarseGIT and FineGIT) can be applied to general neural architectures as long as backpropagation can be applied. In addition, we need to point out that the architecture of the generative model is not a direct inverse of the target model but based on the derivations like Equation (4) and (5). To the best of our knowledge, transformer has not been used in previous works as the target model for gradient inversion under our settings. Therefore, we do not include it for the lack of baselines.

Answer to Question 3. Assumptions Comparison

Refer to Answer to Weakness 2. Stronger Assumption.

Answer to Question 4. Weights Reconstruction

It's a constructive research direction, we also consider using gradient matching to reconstruct weights, which is the inverse of reconstructing inputs by gradient matching.

But model querying is necessary to implement gradient matching approach while we assume the attacker has no access to backpropagation queries in this work.

Answer to Weakness 1. Overfitting

Overfitting is not the major issue contributing to the performance difference, because both GIT and MLP have such issue. There are some general solutions to mitigate overfitting, such as data augmentation, but it is out of the scope of this paper.

Answer to Weakness 2. Experimental Scope

The complementary experiments for additional dataset TinyImageNet-200 is shown in the table below. The reconstructed images are shown in Appendix E.2 in revision, with its best MSE of 0.0317 $\pm$ 0.0003. The results demonstrate that high-frequency information, including object contours and background details, is effectively recovered. Although the MSE and visual quality are lower compared to CIFAR10 under the same configuration, reconstructing data from higher-resolution images poses a significant challenge. Notably, resolutions larger than CIFAR10 have not been explored in baseline methods MLP and DLG.

Answer to Weakness 3. Baseline comparison

Gradient matching approaches have different assumptions compared to ours, so we conduct experiments only on the most popular framework DLG as a reference.

For a more comprehensive comparison and reference of existing training data reconstruction approaches, we conduct complementary experiments on generative approach with UNet as the additional baselines. The table below (also in Table to in the revision) shows quantitative comparison of GIT with prior works on different networks and batch sizes. We use MLP & UNet to represent the generative method using a fixed MLP & UNet architecture, which shares similar number of parameters to our proposed method. The numbers in the table represent the MSE & PSNR between the reconstructed data and the ground truth on the test set. We use $10000$ samples to train generative models.

Answer to Weakness. Theoretical Side

Although Equation 4 is based on the back propagation, it is generic and applicable to general model architectures. To the best of our knowledge, this is the first generative model to conduct gradient inversion where the architecture of the generator adapts to the leaked model. This improves not only the performance but also the interpretability of the method. Moreover, the inverse of activation function and the derived computational graph is not a direct inverse of the original model. For example, Equation (4) use $g_{i}$, $g_{i + 1}$ and $a_i$ to reconstruct $a_{i - 1}$, which is different computational graph from back propagation. Equation (5) is a similar example as well.

Answer to Weakness. Empirical Side 1. Different datasets

The complementary experiments for additional dataset TinyImageNet-200 is shown in the table below. The reconstructed images are shown in Appendix E.2 in revision, with its best MSE of 0.0317 $\pm$ 0.0003. The results demonstrate that high-frequency information, including object contours and background details, is effectively recovered. Although the MSE and visual quality are lower compared to CIFAR10 under the same configuration, reconstructing data from higher-resolution images poses a significant challenge. Notably, resolutions larger than CIFAR10 have not been explored in baseline methods MLP and DLG.

Answer to Weakness. Empirical Side 2. Implementation Details

For detailed implementation, please refer to Appendix B - Experiment Configuration in the paper. The images reconstructed by baselines and our proposed method are both shown in Figure 5 & 6 in Appendix E in revision.

The ablation studies of number of parameters are shown in the table below (also in Table 5 in the paper).

Since GIT learns to invert gradients based on the architecture dependent on the model under attack, its model complexity varies for different FL models and differs from the generative approach that employs a fixed architecture. Generally speaking, during inference, the complexity of running GIT is proportional to running the model under attack, because their architectures are related.

To understand this dependence, we conduct ablation studies by varying the depth and width of an MLP. The results are shown in the table below. All ablation studies are performed with a LeNet FL model and a batch size of 1. In the generative approach with a fixed MLP, a three-layer structure is adopted, each with 1000 hidden units. The first ablation study keeps the model depth constant but increases the depth of each layer to match the number of parameters of GIT, allowing us to evaluate the differences between our method and a standard MLP under the same model complexity. The second experiment maintains the hidden size of each MLP layer but increases the model depth to match that of GIT. However, in these configurations, all gradients are flattened and input from the first layer, rather than fed incrementally through layers as in Coarse GIT. The results show that our proposed GIT model performs better than all configurations of baselines, highlighting its superior performance under fair conditions of equal computational complexity.

Answer to Weakness 1. Access to Training Data

The same assumptions in our work have been investigated in existing literature as well (see [1]), where the attacker has access to some input-gradient pairs of local dataset. Our assumptions are based on the circumstance where some local dataset and its corresponding gradients on one client are leaked to the attacker, which is indeed different from training data reconstruction approach by gradient matching (see [2], [3], [4] and [5]).

The ablation study over the number of input-gradient pairs to train the generative model is conducted in Table 5 of Section 5.4 in revision.

Answer to Weakness 1. Training Data Labels

In most existing training data reconstruction approach by gradient matching, labels and back propagation process are necessary (see [2], [3], [4] and [5]). While generative method (see [1]) including our work does not rely on the labels of training dataset, although it may be leaked to the attacker along with the training data as well.

Answer to Weakness 1. Access to Gradients

Our proposed method is based on the gradient leakage during federated learning settings like Federated Averaging (FedAvg). In FedAvg, during a communication round, each client trains the model on its local dataset and then transmit the gradients to the central server to update the model. To the best of our knowledge, most existing works of training data reconstruction (see [2], [3], [4], [5] and [6]) also assume that the attacker is able to intercept gradients transmitted from client nodes to the server. Since the central server is not hacked by the attacker, we have no access to the model parameters there.

Answer to Weakness 2

Calculating and transmitting gradients are the essential processes of FedAvg. The attacker is able to intercept these gradients and we utilize them to train generative models for training data reconstruction. By contrast, gradient matching approaches require online access to the back propagation of the model on the central server during attack while generative approaches do not.

Answer to Weakness 3. Multiple Models and Datasets

The complementary experiments for additional dataset TinyImageNet-200 is shown in the table below. The reconstructed images are shown in Appendix E.2 in revision, with its best MSE of 0.0317 $\pm$ 0.0003. The results demonstrate that high-frequency information, including object contours and background details, is effectively recovered. Although the MSE and visual quality are lower compared to CIFAR10 under the same configuration, reconstructing data from higher-resolution images poses a significant challenge. Notably, resolutions larger than CIFAR10 have not been explored in baseline methods MLP and DLG.

Answer to Weakness 3. Different assumptions of baselines

There are two baselines considered in Section 5, i.e. MLP & DLG. MLP represents the generative approach using a fixed MLP architecture, which has the same assumptions as ours. While gradient matching approaches have different assumptions, we conduct experiments on DLG, the most popular framework of gradient matching for a more comprehensive comparison and reference of existing training data reconstruction approaches.

Answer to Weakness 3. Batch Size & Image Resolution

For larger image resolution, refer to Answer to Weakness 3. Multiple Models and Datasets. The information loss caused by large batch sizes is a critical issue commonly faced in training data reconstruction, which requires specific methodology to solve (see [5], [6], [7] and [8]). In addition, additional information is needed in these approaches focusing on solving large batch restoration. In this paper, we concentrate on achieving optimal recovery performance with the same model complexity under weaker assumptions by designing a robust and efficient attack model structure. Therefore, large batch size reconstruction is out of the scope of our work.

I want to thank the authors for the detailed response. After carefully reading the rebuttal, my concern remains, and I'd like to comment on the following points.

• Access to training data and labels

As discussed in my initial comment, I agree that the scenario might exist. My concern, beyond availability, is the experiment design. The effect and practical implications of leaked images are unclear compared to existing works that do not require any available images. Specifically, (1) When does an attack have access to up to 20% (10000 out of 50000 for CIFAR-10) of training images in practice? (2) Why 20%, not other numbers? (3) What if only fewer images, say 10% and 5%, are available? (4) The author use small batch size in most of the experiments. How do the authors report the performance? Do they measure the performance on images that are available to attackers and hold-out images (namely, the other images that are unseen to the attacker) separately? As these aspects are missing in the paper, I am not fully convinced by the setting.

• Parameters are unkown I found the authors' point unclear. In federated learning, all clients typically start with the same initialization. This means that if an attacker knows the gradients, they can inherently deduce the model parameters.

• Claims of offline training. My understanding of offline training is that the generative model can be pre-trained before initiating the federated learning process and querying the clients. However, the proposed method appears to require collecting gradients during the FL training process, which, in my view, does not qualify as offline training.

• Multiple Models and Datasets While it is great to see that the proposed method can be applied to higher-resolution images, the poor visual quality (on both CIFAR-10 and TinyImageNet) does not appear to leak information.

• Distance between Weights Thanks for the insight. Given that GIT is an approximation approach, it is surprising that both models will be almost identical in the end. Could the authors briefly clarify the scenarios in which fineGIT is most suitable and those where coarseGIT would be more appropriate, particularly regarding the assumption?

Overall, while the authors' response addressed some of my concerns, my biggest concern remains in the setting. Given the current presentation, I'd lean toward keeping my current score but stay open to discussion.