InfoScissors: Defense against Data Leakage in Collaborative Inference through the Lens of Mutual Information

The paper, in its current form, has some significant gaps:

1. In a realistic scenario, the server may have access to lots of similar data (which is also growing over time) it can correlate with the inputs and outputs of the model. Thus, it is not a "single inference" problem. It is unclear how the amount of data available to the server over time affects the privacy guarantees. For example, in the evaluation, the sever is assumed to have 40 and 400 labeled samples to conduct KA and MC attacks for CIFAR10 and CIFAR100 respectively. What is the server has 50% of the data (e.g., 25,000) or 90%?

2. The threat model is not convincing. Why would a strictly untrusted server would strictly follow the collaborative inference protocol?

3. The evaluation is not sufficient. Why a simple image classification problem over a network with 11M parameters would require a hybrid deployment? To make the case convincing a much more significant evaluation should be carried out with larger architectures and diverse tasks. Moreover, even for the CIFAR10 and CIFAR100 over ResNet18, the reported accuracies are significantly lower than an unsecure baseline model (approximately, -15 for CIFAR10 and -25 for CIFAR100). Why such a significant degradation would be acceptable? given that CNNs are fairly robust to noise and secondary objectives, what degradation one should expect in harder tasks (e.g., LLMs)?

Major weaknesses:

1. Limited Novelty: The proposed method is based on Mutual information (MI) regularization, which has been proposed in MID (Zou et al., 2023) . The major difference is that the paper used a different method proposed in (Cheng et al, 2020) for MI estimation, yet is also limited to inference attacks, not training attacks compared to MID. Therefore it appears that the novelty and contribution are limited. It is suggested that the paper highlights its contributions by explaining in-depth the differences between this work and other MI-based works such as MID and [1], and provide more insights on key factors to achieve superior defense results.

[1] Tianhao Wang, Yuheng Zhang, and Ruoxi Jia. Improving robustness to model inversion attacks via mutual information regularization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11666–11673, 2021.

2. Insufficient Experimental Evaluations: The paper evaluated only two feature reconstruction methods and one label leakage attacks (MC). It is also unclear how and why the experiments are performed separately for feature and label attacks when the model is trained end-to-end in Eq.10. For example, is label MI regularization turned off when feature attack is evaluated? It is unclear how the hyperparameters in Eq. 10 are chosen for each of the evaluations. It is suggested that the importance of each component in Eq.10 should be studied to understand their importance. It is also unclear exactly what hyperparameters are used to carry out experiments in Figure 7 and how their values impact the performance, which would provide valuable insights on how to reproduce and use the proposed method in practice. It is also suggested to compare the results of integrated approach with other methods.

3. Overclaim: The paper states that "this is the first paper to systematically defend against data leakage in collaborative inference, encompassing both input leakage and prediction leakage." However, papers like MID (Zou et al. 2023) also defend both feature and label leakages, in both inference and training stage.

Minor points:

1. Fig.1 lacks proper explanation of notions, e.g. x, x', y, y' and \theta^h..., which makes it hard to understand.

2. The approach proposed and evaluation is limited to collaborative inference only.

1. The novelty of the proposed mutual information framework remains ambiguous. Many relevant works have utilized mutual information to balance privacy and utility. It is unclear what is the novel design in the proposed defense. How is the proposed mutual information framework distinguished from these works? How does it address the challenges in the existing works? For example, the paper claims that (Mireshghallah et al., 2020; Wang et al., 2021; Zou et al., 2023) “only achieve decent results when the head model on the edge device is deep.” How does InfoScissors address this limitation? In addition, the paper claims that “(Makhdoumi et al., 2014; Rassouli & Gündüz, 2019; Wu et al., 2022) apply mutual information on input space to protect input data, but their methods are only feasible with limited input dimension.” How does InfoScissors increase the input dimension?
2. The need for protecting data and prediction in edge computing needs further motivated. It would be great to discuss some potential edge applications that require such data protection.
3. The experiment is insufficient. First, five defense baselines are introduced in the evaluation, but the result of PPDL is not reported. Second, recent defenses, such as (Singh et al., 2021) and [1], are not compared in the paper.
4. I assume that the highest accuracy shown in Figure 3 indicates the case where no defense is applied. (Correct me if I am wrong) Then this clean accuracy on the CIFAR10 and CIFAR100 datasets is pretty low (less than 80% and 50%). This indicates the models might not be well-trained. [1] Yang, Mengda, Ziang Li, Juan Wang, Hongxin Hu, Ao Ren, Xiaoyang Xu, and Wenzhe Yi. "Measuring Data Reconstruction Defenses in Collaborative Inference Systems." Advances in Neural Information Processing Systems 35 (2022): 12855-12867.

• The originality of this work is my primary concern. The proposed solution heavily relies on the mutual information estimator from (Cheng et al. 2020), and the idea of DL inference partition is not new.

• While the paper describes deploying the network's first and last few layers on the edge device, it must be clarified if the number of layers is fixed across devices. Further clarification on this aspect is needed.

• In DL networks, specific layers require a broader context of information from the entire input. Partitioning could disrupt this information flow, potentially leading to suboptimal performance. Knowing how the authors have addressed this challenge in their proposed solution would be beneficial.

• The paper only considers a scenario with one edge device and suggests that the proposed defense can be extended to multiple edge devices. However, it would be helpful to provide experimental results supporting this assertion to lend more credibility to this claim.

The authors claim in their paper, "InfoScissor is the first to systematically defend...", but there have been several published papers that have already explored this area [1]. The authors neither cite their own work in the submission nor compare it in the experiments. Can you clarify the difference and original contribution of InfoScissor?

When hosting models in a cloud-edge hybrid format, it is important to know the distribution of computations within the model. For example, which layers are placed on the cloud and which are on the edge. It is known that the latter features are obstacles and hard to attack, but the authors do not clearly discuss their setting nor compare the defense effectiveness in different scenarios.

Using CIFAR10 and 100 as evaluation tasks alone may not provide a holistic view of the ability of InfoScissor. Including more evaluations such as Stanford Cars, PETs, and CUB-200 would make the experiments more solid.

[1] Measuring Data Reconstruction Defenses in Collaborative Inference Systems; NeurIPS'22