Two Birds with One Stone: Protecting DNN Models Against Unauthorized Inference and Domain Transfer

Thanks for your comments.

[For Q1]

We appreciate your feedback regarding the practicality of our threat model. To address your concern effectively, we would like to understand which specific aspect you find impractical. It's crucial for us to receive more technical details/comments to provide a targeted clarification and justification for our chosen threat model. It's worth noting that Reviewer rjhp considered our threat model realistic and listed it as a strength. We are eager to ensure a clear understanding and address any potential concerns regarding the practicality of our threat model.

[For Q2]

Indeed, we have provided detailed justifications for our choices in Sections 4.1 and 4.2, aligning with state-of-the-art works to ensure fair and meaningful comparisons.

Given your positive evaluation of the robustness, presentation, and contribution of our work, and recognizing the impressive results we achieved, we respectfully ask for a reconsideration of the rating in light of the information we provided (including responses to other reviewers if helpful). We firmly believe that our research makes a substantial contribution to enhancing on-device ML protection by effectively tackling dual vulnerabilities, with the results showcasing the effectiveness of our proposed approach.

Thanks for your valuable comments.

[For Q1]

Our work aims to ensure the secure deployment of pre-trained models by generating a protected version. The protected model is deployed on devices, and crucial data (the original values of the perturbed weights) is safeguarded by TEE. During inference, users can access this data but cannot extract it, thanks to the properties of TEE. On the other hand, attackers cannot even gain access to the secure memory of TEE, although they can extract the unprotected model deployed in unsecured memory, which exhibits low performance for both domains. Therefore, our assumption is that the protected model can be white-box for attackers, which is a stronger threat model with a stronger attacker. We will provide further clarification on the threat model and delve into the details of TEE properties in the final version to eliminate any confusion.

[For Q2]

Our method takes pre-trained models as input and generates their protected versions. To avoid introducing any initialization bias, we utilized open-sourced pre-trained models available online. For the selection of the target domain, we adhered to the experiment settings outlined in SOTA works [1] and [2]. Notably, these references did not include the learning curve in their work, thereby we overlooked its significance. However, we appreciate your suggestion and are willing to incorporate it into the final version.

[For Q3]

The primary advantage of our method lies in leveraging TEE to ensure model quality on the source domain for users. Specifically, we perturbed one filter per layer to generate the protected model through bi-level optimization. The original values of these perturbed weights are stored inside the TEE to preserve the source performance for users. Consequently, bi-level optimization does not compromise the model quality in the source domain.

We acknowledge that the way we leverage TEE may cause confusion to you. We hope the information provided above addresses your concerns. We appreciate your acknowledgment that our paper is intriguing and hope we have addressed all your concerns.

Reference:

[1] Non-transferable learning: A new approach for model ownership verification and applicability authorization. In International Conference on Learning Representations (ICLR), 2022.

[2] Model barrier: A compact un-transferable isolation domain for model intellectual property protection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20475–20484, 2023.

Thank you for your feedback, and we appreciate your acknowledgment of the soundness, presentation, and contribution of our work. We understand your interest in broader evaluations and comparisons.

As mentioned in Section 4.2, our work is pioneering in simultaneously addressing both unauthorized inference and cross-domain transfer (i.e., dual protection). Given the novelty of our approach, there are no previous works that are directly comparable or as the baseline. However, we tried our best to find comparison methods, including state-of-the-art approaches like [1-3]. Moreover, for fair evaluation, we intentionally aligned with their datasets and model selections to ensure meaningful comparisons and to underscore the notable performance achieved by our method.

The choice of datasets in our evaluation aligns with the common settings in state-of-the-art works focusing on cross-domain transfer, where the source and target domains share the same label space. To the best of our knowledge, large datasets like CIFAR-100, Tiny-ImageNet, and ImageNet are not commonly used in these works and ours, likely due to the challenge of constructing appropriate target domains. We are open to incorporating your suggestions if you can guide us to cross-domain transfer works that use these larger datasets, allowing us to follow similar settings.

Regarding a wider range of prior arts, we have made efforts to include relevant and state-of-the-art methods for comparison. However, if you have specific suggestions for additional prior arts, we would appreciate your insights.

Considering your positive assessment of our work's soundness, presentation, and contribution, and acknowledging that the evaluation is your primary concern, we kindly request you to reconsider the rating in light of the information provided above. We believe our work significantly advances on-device ML protection by addressing dual vulnerabilities, and the results demonstrate the efficacy of our proposed method.

Reference:

[1] NNSplitter: An active defense solution for DNN model via automated weight obfuscation. In Proceedings of the 40th International Conference on Machine Learning, 2023.

[2] Non-transferable learning: A new approach for model ownership verification and applicability authorization. In International Conference on Learning Representations (ICLR), 2022.

[3] Model barrier: A compact un-transferable isolation domain for model intellectual property protection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20475–20484, 2023.

[Threat Model]

We utilized Sun et al. (2021) [3] as an example to illustrate the practical vulnerability of model extraction, highlighting the white-box nature of the threat. Although various papers, including Sun et al. (2023) [4], have proposed solutions leveraging TEE to mitigate model extraction, the critical challenge of universally employing TEE for DNN model protection still exists. Notably, Sun et al. (2023) [4] has its own limitations, as already discussed in [5]. Importantly, compared to Sun et al. (2023) [4], our work addresses a more challenging security problem, encompassing both model extraction and unauthorized domain transfer in on-device ML scenarios, and we employ TEE in a novel manner to tackle this dual threat.

[Clarification for Q3]

The assumption that attackers are able to extract the model weights completely doesn't violate the requirement that unauthorized inferences on the source domain must be prevented. The reason is that we deployed the protected model on the device instead of the pre-trained/victim model, as clearly stated in the introduction:

Therefore, when deploying the protected model on the device, attackers cannot achieve high performance on both domains, thus achieving double protection. Meanwhile, the original value of the perturbed weights is small enough to be stored in the stringent secure memory of TEE, which can be used to restore high performance on the source domain for users.

Besides, your premise is problematic since the un-obfuscated model weights cannot become available to attackers. We explicitly mentioned that we use TEE to secure these data in the Introduction, Sections 2.1, and 2.3. This reliance on TEE security is fundamental to our approach, which we assume is the basic background for a reviewer like you (given that your review confidence is quite high).

[Clarification for Q4]

The challenge of an unknown target domain is a common concern addressed by various works, and we did not claim it to be a unique challenge.

Specifically, we do not agree with the reviewer that addressing a research challenge loses significance just because there have been prior attempts, even if they are not applicable or offering high-performance. Even for Wang et al. 2022 [1] as you mentioned, it is not the initial work to tackle the challenge of an unknown target domain, as demonstrated by prior work in single-domain generalization, such as [7]. Still, it makes great and unique contributions to solve a different problem. Also, Wang et al. 2022 [1] has limitations as discussed in [6], which tackled the exact same challenge using a different method. We did not thoroughly discuss such limitations since how to tackle an unknown target domain is not our focus or the main contribution we want to make.

Our focus is on providing a comprehensive defense solution, extending to pre-trained/well-trained models for on-device ML. The gap between our work and prior approaches is discussed to highlight the specific contributions and advancements of this work.

Reference:

[3] Mind your weight (s): A large-scale study on insufficient machine learning model protection in mobile apps. In 30th USENIX Security Symposium (USENIX Security 21), pp. 1955–1972, 2021.

[4] Shadownet: A secure and efficient on-device model inference system for convolutional neural networks. In 2023 IEEE Symposium on Security and Privacy (SP).

[5] Teeslice: slicing dnn models for secure and efficient deployment. In Proceedings of the 2nd ACM International Workshop on AI and Software Testing/Analysis, pages 1–8, 2022.

[6] Model barrier: A compact un-transferable isolation domain for model intellectual property protection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20475–20484, 2023.

[7] Learning to learn single domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12556–12565, 2020.

Thank you for your feedback, and we appreciate the opportunity to address your concerns and clarify certain aspects of our work.

[Terminology Clarification]

First, we would like to clarify the misunderstanding between the terms "well-trained models" and "pre-trained models" that were used interchangeably in this paper, to convey the same meaning. We did mention "well-trained" models several times in the introduction section (and in the threat model), to emphasize that our research objective in this work is to securely deploy well-trained/pre-trained models.

Following your suggestion, we will consistently use "pre-trained models" to avoid ambiguity. Additionally, we will explicitly include the keyword "pre-trained models" in the abstract and contribution sections for better clarity.

[Motivation]

We have to emphasize that our work uniquely identifies a security problem related to on-device deep neural networks (DNNs) — specifically, the dual vulnerabilities associated with preventing unauthorized use and restricting cross-domain transfer. The solution we proposed cannot be simply replaced by the combination of current works, for the following reasons.

First, in fact, Wang et al. 2022 [1] did not anticipate combining the two approaches, since their threat model is based on AIaaS, where the model providers have no motivation to degrade the performance on the source domain. However, our threat model is totally different, and based on on-device ML, i.e., we are targeting different problems in this work.

Besides, the suggestion to combine Chakrabory et al. 2020 [2] and Wang et al. 2022 [1] is infeasible, primarily because they employ different training strategies and start from scratch, i.e., model owners cannot first use one method to train the model then use another one to train again and expect the resulting model can address two vulnerabilities at the same time.

The motivation for our work lies in providing a comprehensive defense solution that extends to pre-trained models, granting model owners the flexibility to preserve valuable model properties, such as transferability. This flexibility is a key advantage. To elaborate, model owners can retain the pre-trained model and efficiently transfer it to other domains, if needed. Simultaneously, they can employ our method to securely deploy the model on devices. In contrast, directly adopting the approach proposed in Wang et al. 2022 [1] would confine the model to the domains it was initially trained on, limiting both its flexibility and transferability, even for the model owners themselves.

We will incorporate the above discussion into the final version to provide a more detailed elaboration of our motivation.

Reference:

[1] Non-transferable learning: A new approach for model ownership verification and applicability authorization. In International Conference on Learning Representations (ICLR), 2022

[2] Hardware-assisted intellectual property protection of deep learning models. In 2020 57th ACM/IEEE Design Automation Conference (DAC), pp. 1–6. IEEE, 2020.

I would like to see the revised pitch (Abstract and Intro at least) that restates and clarifies the contributions.

I am not convinced why Wang et al 2022 cannot be combined with a TEE-based approach. Doesn't it provide a valid defense? I get your argument that Wang et al. 2022 may require retraining if the original owners want to transfer the model -- but that doesn't make the combo invalid -- only less efficient. Can you elaborate in more detail on why they can't be combined? There are multiple TEE-based approaches, including the more recent NNSplitter work. None of them can be combined with each other?

It seems to me that if they can be combined, even if that requires additional training, one is largely left with an efficiency argument. (Improved efficiency can still be a valid contribution -- but the paper's pitch has to be consistent with that, acknowledging that there is a different solution strategy.) If you are claiming that a combination is simply not possible for the two threads of research in the past, then I think that needs a much stronger technical argument than currently made in the paper to convince a reader. Either way, it seems to me that the paper's thrust needs to be revised significantly, with a much clearer articulation of the claimed contribution over past work.

Thanks for updating it. I think the paper is improved, but I don't think the paper is quite there yet.

I moved my score up from my original score accordingly to a reject from strong reject, but I think the paper still has issues in clarity of contributions over prior art. You may also want to fix the incomplete sentence just before Section 3 -- I think you were trying to make some argument, but it was left midway.

A few more thoughts as I read the updated paper for you to consider:

1. My understanding is that the cross-domain transfer you are worried about is by authorized users. For unauthorized users, it seems the model performance is anyway degraded by techniques such as (Zhou 2023). If so, it would have been clearer to state that in the Intro clearly.

2. My understanding is that you try to prevent the transfer attacks by authorized users to domains that are "close" to the source domain. Closeness is defined by a distance threshold. Could an adversary first transfer the model to a domain that is just outside that threshold? And then perhaps at a much lower cost, retransfer it to a close function? I am thinking of an adaptive adversary here.

3. On-device versus MLaaS argument: How critical is that to your paper to distinguish from prior work? If it is critical, I think you need to also argue why prior work could not simply be run on-device rather than in MLaaS mode. Is there something fundamental that prevents Wang et al. (2022, 2023) from being used in on-device settings? Regardless, it appears NNLsplitter work (Zhou 2023) assumes on-device as well.

4. The approach in section 3.2 seems to be inspired by Zhou 2023 (NNSplitter). They also modified a small set of weights, but to degrade unauthorized use. I wonder if the paper would have read better if throughout the paper, including the Intro, Zhou 2023 was used as the closest work to your work (acknowledging that). For instance, Zhou et al. addressed the on-device model extraction issue by unauthorized users, but not addressing domain transfer by authorized users. You could also point out why earlier work prior to Zhou et al. were more limited in scope -- e..g, required a much larger-memory TEE or could not be used on-device (and why).

I think scoping the problem correctly with proper context would have gone a long way in improving the paper.

5. Now coming to the question of preventing model transfer by authorized users to a nearby domain in the on-device scenario, it is not clear if you considered the following attack strategy: An authorized user simply steals the source model by using a data-free method such as MAZE that does not require source data. It seems to me that most of the queries with systems such as MAZE would be far away from the source domain and thus not disabled by the proposed defense, since it only addresses queries from a nearby distribution. In the on-device scenario, it seems to me that data-free model stealing attacks should be easier since the attacker has local, fast access to the on-device model.

Note that in Zhou 2023, they assumed that the attacker (an unauthorized user in their case) can completely steal the model, including exact weights. Of course, in their scheme, the attacker would only get a model with many weights incorrect. They further assumed that the attacker does not have source data (or very limited source data) -- and thus cannot retrain a model. Thus, there were fewer concerns under their threat model, since authorized users were considered trusted. In your threat model, authorized users are not so trusted -- they may attempt to transfer the model to nearby domains. So you do need to consider stronger attacks.

Thus, in your case, under the same assumptions, it appears the authorized user can completely extract the source model. A possible next step to bypass your defense would be to then transfer the model (by some retraining). This is model extraction followed by transfer, rather than transfer followed by model extraction that is assumed in your paper.

Overall, I think the paper writing has to improve with more thought given to the pitch, why it really differs from prior work, and why adaptive attacks on your defense may not work (or you may find that they will work, in which case you can write an attack paper showing that no defense that allows model stealing on the source domain can prevent model transfer.) A more comprehensive analysis of potential attacks on your defenses may be needed.

In summary, consider improving the paper significantly by sharpening the pitch. It could be that it is a better fit for a security-oriented conference. I recommend improving the work, thinking a bit more deeply on the defense as well as potential adaptive attacks, and resubmitting to an appropriate venue. Best wishes.

Thanks for pointing out the unfinished sentence and we fixed it in the newly uploaded version (which also clarifies the threat model and introduces more details about TEE for better understanding) .

[For 1] Your understanding is incorrect. In our work, the authorized users are attackers who would extract the model from unsecured memory and use it for direct use or cross-domain transfer.

[For 2] If you are considering a more adaptive adversary, at least you agree that our threat model is reasonable and that our work has addressed the vulnerability we identified. Even if the adaptive attack you briefly mentioned can be a real attack (although we do not how this attack exactly happened), you should recognize that our method has already increased attackers' efforts in obtaining a useful model. This is sufficient to demonstrate the significance of our defense work.

[For 3] Given your high confidence, it should be clear that on-device ML and MLaSS face different vulnerabilities --- We have explained there is no motivation for MLaSS works to degrade performance on the source domain, but we are motivated to do it as a defense method (while leveraging TEE to ensure performance for authorized users). Also, While we acknowledge that NNSplitter is designed for on-device ML, as discussed in the Introduction and related work (Sec. 2.4), it only considers one vulnerability.

[For 4] As mentioned above, we clearly mentioned NNSplitter at the very beginning (the first paragraph in Intro), also discussed in Sec 2.4.

[For 5] In our threat model, authorized users are benign users (e.g., those who bought our ML application) and will not conduct model-stealing attacks, whereas attackers are unauthorized users who did not pay for the application but want to extract it and use it for free (either direct use or do cross-domain transfer). Also, our threat model is quite similar to NNSplitter, with the difference being that we consider attackers could also conduct cross-domain transfer.

Additionally, even for authorized users, they cannot extract the data inside TEE, which remains a black box to them (according to the property of TEE). We added this background in the revision.

We observe that our main discussions revolve around different concepts, unproposed combinations of prior works, or briefly mentioned attacks that may be unpractical. However, we regret that there is no opinion on our methodology/experiment/performance. We hope our response can give you a better understanding of our work so that you may point out any technical problems for us to improve the work. With only conceptual concerns (which do not bother any other reviewers), we can only assume that it is just our storytelling that does not fit your taste, but there's no indication of anything wrong with other parts. We respect your decision if you are reluctant to increase the score, but we would appreciate more constructive suggestions to ensure that we are convinced your rating is fair.

For 1, the threat model you mentioned is different from ours. We are pleased to note that our research has inspired others, such as yourself, to delve into more adaptive attacks or consider different threat models.

For 2, yes, we did consider.

We find it difficult to align with your ratings and justification, especially considering the poor (score 1) ratings for Soundness, Presentation, and Contribution, while all other reviewers have given good (score 3) ratings. Furthermore, considering your self-assessed highest confidence, we are perplexed as to why certain basic aspects of on-device ML and TEE seem unclear to you (while our work is leveraging TEE to mitigate vulnerabilities in on-device ML).

Lastly, we want to emphasize that it is challenging for a research paper to address all problems in a single endeavor, particularly in the realm of security papers, which often spawn follow-up papers proposing adaptive attacks or enhanced defenses. The initial exploration of a novel attack or defense in a paper sets the stage for subsequent researchers to propose more advanced solutions. Such contributions should be acknowledged, or at the very least, not downplayed.

Your raised open questions (e.g., potential adaptive attacks or different threat models), some of which might be worth exploring for other researchers, should not diminish our contribution. On the contrary, they demonstrate that our work could lay the foundation for future endeavors.

As the rebuttal period comes to a close, we genuinely look forward to receiving your response. Your time and consideration are highly valued. Thank you.