CoreGuard: Safeguarding Foundational Capabilities of LLMs Against Model Stealing in Edge Deployment

Dear Reviewer,

Thank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.

On TEE Oracle Attacks --We fully acknowledge your concern. However, we would like to clarify that attackers cannot perform such a TEE oracle attack on CoreGuard. Specifically, the TEE participates in the forward pass but does not support gradient propagation during backpropagation, ensuring that attackers cannot optimize the model. Second, even if attackers attempt to bypass the TEE by training only part of the model—such as freezing layers before or after the TEE and reinitializing the remaining parameters—this remains infeasible. This is because CoreGuard places the authorization point in the middle of the network, forcing attackers to retrain at least half of the model’s parameters, which is unrealistic for an attacker.

Rigorous Proof of the Matrix-LWE Hardness Connection -- We appreciate the reviewer’s feedback and agree that the connection to the Learning With Errors (LWE) problem can benefit from a more rigorous exposition. Below, we provide a refined justification that formalizes the reduction to a Matrix-LWE instance.

Assume the attacker is given: An approximate estimate $\hat{W}_b$ of the true matrix $W_b$ , satisfying $\| \hat{W}_b - W_b \|_F \leq \varepsilon'$; A known output matrix $W = \pi W_b$; The recovery target: $\pi$. What the attacker can actually construct is: $W = \pi W_b = \pi \hat{W}_b + \pi(W_b - \hat{W}_b) = \pi \hat{W}_b + \tilde{\epsilon}$, where: $\tilde{\epsilon} := \pi(W_b - \hat{W}_b), \quad \text{and} \quad \| \tilde{\epsilon} \|_F \leq \| \pi \|_2 \cdot \varepsilon'$. In other words, the attacker can only observe: $W = \pi \hat{W}_b + \tilde{\epsilon}$, which exactly corresponds to the form of a Matrix Learning With Errors (Matrix-LWE) problem.

A formal proof of this reduction will be provided in a subsequent section of the paper.

Applicability of CoreGuard to Non-Transformer Models -- While our implementation focuses on Transformer models, the design of CoreGuard itself is inherently architecture-agnostic, as it builds upon the three core design principles that each independently exhibit compatibility across diverse architectures. First, from the protection perspective, CoreGuard introduces structure-aware defenses by embedding permutation-based transformations into linear mapping layers, which serve to disrupt potential low-loss merging paths between models. This mechanism targets the mathematical structure of linear layers rather than relying on Transformer-specific components. Since linear transformation layers are ubiquitous in neural networks—including convolution layers in CNNs and fully connected layers in MLPs—our protection approach applies readily to those architectures as well. Second, from the authorization perspective, the generation and injection of both the permutation matrices and OTP noise are handled entirely by the TEE. These operations depend solely on the TEE’s capability to manage secure randomness and authorize parameter access. The architecture of the underlying model does not affect the authorization process, regardless of whether the model is a Transformer, CNN, or any other architecture. Third, from the perspective of authorization propagation, CoreGuard ensures that downstream layers can only be unlocked if upstream layers have been correctly authorized. This function operates on the local behavior of each linear transformation layer and does not require any specific structural assumption about the model. Thus, the propagation rule is universally applicable to any architecture that contains linear transformation layers.

We take a standard CNN like ResNet as an example. CoreGuard protects the convolutional kernels by permuting the input channels of the kernel, effectively locking it. To ensure proper operation, the input feature map must be similarly permuted to match the kernel's new channel order (i.e., it must be authorized). Additionally, the output channels of the convolutional kernel are permuted to ensure that the output channels correspond to the input channels of the next kernel. Once the first layer is authorized with the correct input permutation, the authorization propagates automatically to subsequent layers (i.e., authorization propagation). This approach works seamlessly with activation functions like ReLU, as they only process the values of the elements and are independent of their positions. In this way, CoreGuard provides model parameter protection and ensures the model can only operate under authorization.

Once again, thank you for your constructive feedback. We look forward to any further suggestions you may have.

Dear Reviewer,

Thank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.

Real-World TEE Settings -- We appreciate the reviewer’s insightful comment. We clarify that CoreGuard is designed for broader and more prevalent edge hardware, such as smartphones and personal computers, where TEEs are typically implemented on CPUs (e.g., ARM TrustZone, Intel SGX). These TEEs are widely deployed in commodity devices and represent the dominant form of trusted execution available in edge settings. In contrast, GPU-integrated TEEs—such as those in NVIDIA H100—are designed for cloud or datacenter usage, and are generally inaccessible for most edge deployments due to their high cost, hardware footprint, and specialized infrastructure requirements. Therefore, CoreGuard does not conflict with these server-side TEE solutions but rather serves as a complementary defense tailored for practical, real-world edge scenarios.

Why Reducing TEE-Induced Overhead Still Matters? -- In edge deployment scenarios, where efficient computation and minimal latency are crucial, reducing TEE-induced overhead is essential. Specifically, TEEs inherently impose significant limitations on practical model deployment, causing computation and transfer bottlenecks that have been consistently observed in prior work. For instance, Table 7 in NPLO[51] shows that even when shielding only small adapters, 97.06% of total inference time is spent on TEE-related operations—35.61% on TEE-GPU transfer and 61.39% on TEE execution. Methods that attempt to protect model backbones face even worse bottlenecks. For example, ShadowNet incurs 1.3GB/token transfer for LLaMA3-8B due to frequent TEE-GPU interactions (Lines 64–72), making inference impractical. Given that mainstream mobile platform TEEs, like TrustZone, have a transfer rate of about 1GB/s between TEE and GPU [3], generating a single token takes about 1.3 seconds. Consequently, producing a complete output would require several hundred seconds (assuming it consists of 100 tokens) solely for data TEE-GPU transfer. This transfer overhead is a significant bottleneck, making inference impractical.

Key Distinction from ShadowNet -- Your concern is understandable. It is true that our method protects parameters through permutation, while ShadowNet uses shuffling, and they share a certain degree of similarity. However, the challenges they face are entirely different. Parameter protection is only the most basic goal of our method. Our main contribution lies in addressing the efficiency problem, which is not a simple “extension,” but the key to whether the scheme is practical. Specifically, ShadowNet still relies on TEE to directly execute model computations; even though it avoids computing linear layers, it still requires the TEE to execute nonlinear layers. Experimental results show that ShadowNet needs to keep about 30% of FLOPs inside the TEE. In contrast, CoreGuard uses the TEE only for authorization, without performing any model computation, reducing the FLOPs inside the TEE to 1e-5 of the original. Moreover, ShadowNet requires the feature to be transferred into and out of the TEE before every linear layer, resulting in enormous communication overhead (hundreds of transfers per inference). CoreGuard avoids this problem through one-time authorization and a propagation protocol, reducing the number of TEE transfers to just 5.

Once again, thank you for your constructive feedback. We look forward to any further suggestions you may have.

Dear Reviewer,

Thank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.

Scalability and Bottleneck Analysis of CoreGuard -- We appreciate the reviewer’s question regarding the scalability of CoreGuard. While our current evaluation focuses on 0.5B–8B models to align with real-world edge deployment scenarios (e.g., Apple Intelligence, Gemma2, Qwen2), CoreGuard is fundamentally designed to scale to much larger models such as 13B or 70B without incurring bottlenecks in memory or latency. Specifically, first, regarding memory usage, it is important to clarify that the permutation matrix never leaves the TEE and does not occupy GPU memory. Additionally, with CoreGuard, only one permutation matrix is needed for the entire model, which is a negligible storage requirement for the TEE. Regarding propagation overhead, no costs are incurred during inference, regardless of the model's size. Specifically, the propagation protocol involves only the column permutation of the weight matrices, which is applied before deployment and is not part of the inference process. Therefore, no additional propagation overhead is introduced compared to the original model.

Compatibility with Side-Channel Defenses such as HybCache -- CoreGuard and HybCache can be seamlessly integrated. HybCache is a side-channel defense mechanism designed to prevent cache-based attacks in SGX by periodically and randomly changing how memory pages are mapped within the enclave, making it difficult for attackers to infer sensitive information from access patterns. These measures, both functionally and in terms of efficiency, are fully compatible with CoreGuard. Specifically, from a functional perspective, CoreGuard’s use of the TEE involves a few simple matrix operations, such as applying row and column permutations and basic matrix addition. These operations follow predictable memory access patterns and avoid data-dependent branching, ensuring they do not interfere with HybCache’s memory randomization, which periodically changes memory page mappings within the enclave to prevent cache-based attacks. From an efficiency perspective, CoreGuard’s lightweight matrix operations are computationally efficient, as they involve simple arithmetic. This means CoreGuard’s operations do not introduce additional memory accesses or data transfers, allowing HybCache to perform its memory page randomization without extra overhead.

Moreover, CoreGuard can be integrated with a wide range of side-channel defense mechanisms because it relies only on basic TEE functionality. It does not require specialized memory layouts, privileged instructions, or microarchitectural modifications. Specifically, from a software perspective, CoreGuard uses the TEE only for basic matrix operations, making it easy to integrate with existing side-channel defense algorithms such as constant-time programming and memory access randomization. From a hardware perspective, CoreGuard requires no changes to the TEE’s microarchitecture, enabling compatibility with physical protection techniques such as EM shielding or power noise injection. These properties allow CoreGuard to serve as a complementary layer of defense in diverse threat models.

Security Under Joint Recovery Attacks Against CoreGuard -- We would like to clarify that joint attacks attempting to recover both permutations and OTP noise represent a very difficult problem. Specifically, in Appendix D, we present our Security against Permutation Matrix Simulation Attack, where we specifically attempt to recover the permutation matrix in isolation. Our results demonstrate that such attacks are unsuccessful, emphasizing the robustness of our approach to individual attacks targeting permutations. Additionally, recovering OTP noise is nearly impossible due to the nature of OTP encryption. Each OTP noise element is unique and used only once, making it infeasible for attackers to deduce the noise matrix through training or other methods. Moreover, attempting to recover both jointly is an even larger and more difficult problem. Specifically, we further analyze this infeasibility from a theoretical perspective in Appendix C. We show that any attempt to jointly fit the permutation and the noise leads to a system of equations that is mathematically equivalent to the Matrix Learning With Errors (Matrix-LWE) problem—a well-studied extension of the standard LWE problem. Matrix-LWE is widely believed to be computationally intractable, with hardness reductions to NP-hard lattice problems such as the Shortest Independent Vector Problem (SIVP) and the Gap Shortest Vector Problem (GapSVP). Therefore, even with unlimited access to input-output pairs, solving for both $\pi$ and $p^\pi$ remains provably hard. If the reviewer has other concrete attack strategies or threat assumptions in mind—such as specific attacker capabilities or attack methods—we would be happy to examine and evaluate them within our framework.

Clarifying the Contribution of CoreGuard -- We respectfully clarify that our approach is not merely a systems engineering effort. Rather, it introduces a novel method that deeply intervenes in the inference pathway of LLMs, by modeling and controlling the transformation of internal features through structured authorization and propagation mechanisms. Specifically, CoreGuard is designed to address foundational capability stealing, a threat unique to LLMs due to their strong generalization abilities. To counter this, we propose a semantic-preserving but structurally disruptive transformation on LLM layers (e.g., linear projections, FFN blocks, Add&Norm), which enforces runtime authorization and blocks unauthorized reassembly of the model’s knowledge pipeline. This design goes beyond conventional deployment strategies by reconstructing the LLM’s computation graph in a way that binds its functionality to a trusted enclave, thus preventing misuse of the model’s generalization ability. We believe this contribution is not only technically novel but also methodologically grounded in LLM-specific reasoning patterns and architecture semantics, making it relevant to the NeurIPS audience concerned with LLM behavior modeling, robustness, and secure deployment.

Clarifying the Formal Problem Formulation -- We recognize that introducing the formal definitions earlier would have better clarified the problem setup and supported the threat model more clearly. In the revised version, we have reorganized Section 2 (now titled Problem Statement and Threat Model) to include a formal definition of the model structure, the protected components, and the adversarial objective. Specifically, we define the LLM as a function $f_w: \mathbb{R}^{l \times d} \rightarrow \mathbb{R}^{l \times d}$, and identify the input-processing and output-processing layers involved. We explicitly describe how the model is “locked” via permutation of parameter matrices, and how this locking constrains unauthorized usage. The revised section also clarifies what parts of the model are exposed to the attacker (e.g., weights outside the TEE) and what transformations are applied to enforce protection. These definitions align with the attacker’s capabilities and the defender’s goals, and lay the foundation for the mechanism presented in Section 3.

Once again, thank you for your constructive feedback. We look forward to any further suggestions you may have.

Dear Reviewer,

Thank you very much for your valuable feedback and thoughtful comments. We appreciate the time and effort you have dedicated to reviewing our paper. In the following, we will address each of your concerns point by point.

Regarding Black-Box Distillation Attacks -- We acknowledge that our work does not address black-box distillation attacks, as this constitutes a fundamentally different research topic. CoreGuard focuses on enforcing access control—ensuring the model cannot run without authorization—while assuming that authorized users can access the model normally. In such cases, attackers could indeed collect input-output pairs and perform distillation. Defending against distillation attacks is a new problem, typically categorized into passive defenses (e.g., output watermarking, randomness injection) and active defenses (e.g., training-time obfuscation, anti-distillation regularization). These techniques aim to reduce the effectiveness or fidelity of the distilled model but are orthogonal to CoreGuard’s objective.

Does More Training Data Break CoreGuard? -- Thank you for raising this important point. To evaluate the robustness of CoreGuard under more powerful attackers, we conducted additional experiments targeting both the finetuning attack and the Authorization Simulation Attack. To evaluate the robustness of CoreGuard under more powerful attackers, First, for the finetuning attack, we conducted an additional experiment where we expanded the original training set by querying the protected model to generate $5\times$ more data. We then used this enlarged dataset to perform model stealing attacks following the same procedure as in our main experiments. The results show no meaningful improvement in attack accuracy—accuracy remains close to the black-box baseline, indicating that even significantly increased query data does not compromise CoreGuard’s protection. Next, for the Authorization Simulation Attack, which attempts to simulate the entire TEE structure, including the FFN block and the permutation matrix inside the TEE, we trained with the enlarged $5\times$ dataset and ensured that the loss had already converged. The results show that accuracy remains close to the black-box baseline. This is because CoreGuard requires high precision in the authorization process, and even slight simulation errors can compromise the model’s performance. Since the results cannot be displayed on the webpage, all detailed results will be updated in the Appendix.

On the Existence of a Practical Upper Bound for Defense -- While we agree that attackers can, in theory, issue an unlimited number of queries, each query incurs real-world computational cost and, when scaled up, can become equivalent to training a new model from scratch—ultimately making the attack economically meaningless. From a defense perspective, CoreGuard is explicitly designed to raise the attack cost close to this upper bound: the cost of training a model from scratch given only the architecture (i.e., the black-box setting, where the attacker knows the model structure but must re-learn all parameters). Achieving such black-box-level protection is both a meaningful and practical target for securing edge-deployed LLMs, as it renders model stealing attacks economically and technically infeasible.

Applicability of CoreGuard to Non-Transformer Models -- While our implementation focuses on Transformer models, the design of CoreGuard itself is inherently architecture-agnostic, as it builds upon the three core design principles that each independently exhibit compatibility across diverse architectures. First, from the protection perspective, CoreGuard introduces structure-aware defenses by embedding permutation-based transformations into linear mapping layers, which serve to disrupt potential low-loss merging paths between models. This mechanism targets the mathematical structure of linear layers rather than relying on Transformer-specific components. Since linear transformation layers are ubiquitous in neural networks—including convolution layers in CNNs and fully connected layers in MLPs—our protection approach applies readily to those architectures as well. Second, from the authorization perspective, the generation and injection of both the permutation matrices and OTP noise are handled entirely by the TEE. These operations depend solely on the TEE’s capability to manage secure randomness and authorize parameter access. The architecture of the underlying model does not affect the authorization process, regardless of whether the model is a Transformer, CNN, or any other architecture. Third, from the perspective of authorization propagation, CoreGuard ensures that downstream layers can only be unlocked if upstream layers have been correctly authorized. This function operates on the local behavior of each linear transformation layer and does not require any specific structural assumption about the model. Thus, the propagation rule is universally applicable to any architecture that contains linear transformation layers.

We take a standard CNN like ResNet as an example. CoreGuard protects the convolutional kernels by permuting the input channels of the kernel, effectively locking it. To ensure proper operation, the input feature map must be similarly permuted to match the kernel's new channel order (i.e., it must be authorized). Additionally, the output channels of the convolutional kernel are permuted to ensure that the output channels correspond to the input channels of the next kernel. Once the first layer is authorized with the correct input permutation, the authorization propagates automatically to subsequent layers (i.e., authorization propagation). This approach works seamlessly with nonlinear layers like ReLU, batch normalization (BN), and LeakyReLU, as they only process the values of the elements and are independent of their positions. In this way, CoreGuard provides model parameter protection and ensures the model can only operate under authorization.

Once again, thank you for your constructive feedback. We look forward to any further suggestions you may have.