Permissioned LLMs: Enforcing Access Control in Large Language Models

We thank the reviewer for their detailed feedback and for recognizing the merits of our access control formalism and evaluation framework and their relevance to the enterprise settings. Below we would like to respond to the questions and concerns raised in the review:

Guidance on usability and scalability

Regarding the comment "...insufficient analysis of when scalability becomes prohibitive or guidance on the maximum practical number of domains.": The usability is subjective and depends on the dataset sizes, size of the models and the compute resources available. We expect the mechanisms to scale well to few dozen domains for the models of sizes similar to Llama-8B and Mistral-7B models. We have experimented with 3 to 10 security domains in our experiments across different datasets. In practice, the combinations are few and sparse in enterprise settings and thus we don't need all possible permutations for Union. For instance, in enterprises there would be hierarchy in organization and not all employees would get all possible access combinations: employees typically only have access to lower levels than their designation. In such settings, the combinations could be in the order of nlog(n) for a tree-type hierarchical organization structure. All these interesting scalability issues are a part of our future work.

Handling domain segregation

In response to the question "How does the system handle noisy domain boundaries or cross-domain data leakage (e.g., data records shared across silos)?": We assume the separation of data across security domains has happened before our access control mechanisms come into play. Most enterprise settings already have strict separation/segregation of data across different silos or organizations. Any individual that is part of any two organizations would naturally get access to data from both domains. The actual handling of separation of noisy cross-domain boundaries is beyond the scope of this work.

Hyperparameter sensitivity

In response to the question "Given that only default LoRA rank (64) was tested with limited ablation studies, how sensitive are the results to key hyperparameters like adapter rank, learning rate schedules, and the choice of layers for adapter placement?": We tried adapters with different LoRA ranks ranging from 2 to 64, and found it to have limited impact on the metrics, and hence set the default value to 64. We also tried different learning rates between 1e-5 and 5e-4 using grid search in our initial experiments and set the best values for all our experiments. We have not experimented with choosing specific layers for adapter placement and went with the standard LoRA implementation of using adapters for all layers. We will add this discussion in the Detailed Experiment Setup section in the appendix.

We thank the reviewer for their detailed feedback and for acknowledging the merits of our access control formalism and evaluation framework. We would like to take the opportunity to respond to the questions and concerns raised in the review:

Regarding the auditing process

In response to the question "Given that the construction is private by construction as each user can only use an adapter from security domains it has access to, its not clear how much value all the empirical metrics are adding to understanding the privacy of the mechanisms? If the question is about auditing, would it be easier and more effective to audit that the adapters themselves are trained on the correct data instead?":

Our access control evaluation is with respect to an auditor having access to all the domain adapters and domain datasets who wants to verify if the access control mechanism is implemented correctly. Hence the auditor chooses a query from a target domain and evaluates the access advantage when querying the target domain adapter (i.e., "in" adapter) instead of any other adapter (i.e., "out" adapter). This also answers how much information a user can gain for a target domain query when they have access to the target domain versus when they do not, thereby bounding the leakage about an individual query/record.

The above problem can also be reformulated by fixing an adapter and querying it with "in" and "out" domain data, and our experiments already capture this. The UGI metrics reported in Figure 2 and 3, and Table 3, show the average gaps across all domain adapters and domain datasets, which implicitly captures this scenario. For example, given two adapters, A1 and A2, and two security domains, D1 and D2, the UGI metric finds the average utility gap between "in" and "out" adapters for both domains: ((U(A1, D1) - U(A2, D1)) + (U(A2, D2) - U(A1, D2))) / 2. The alternate problem formulation is also equivalent to this: ((U(A1, D1) - U(A1, D2)) + (U(A2, D2) - U(A2, D1))) / 2.

Finally, both DDI and UGI are agnostic to the underlying implementation of access control. Whether the system is built using PEFT or other future techniques, our metrics remain applicable. This makes them broadly useful for auditing the effectiveness of any access control enforcement mechanism, not just the specific construction used in this work.

Regarding the UGI metric

In response to the question "Can you elaborate on how UGI makes sense as a metric? From what I can tell, it doesn't tell us much about utility and can only tell us about privacy leakage in a very roundabout way.": We note that the UGI metric quantifies the difference in utility between the cases when a user has access to a security domain and when they do not have access to that domain. This pertains to how much information a user can gain for a target domain query when they have access to the target domain versus when they do not. UGI does not quantify the model utility itself.

Regarding the scalability and simplicity of the approach

Regarding the comment "The provided methods for PermLLMs seem fairly simple and combine already existing methods.": While the access control mechanisms are simple and straightforward, we would like to highlight the effectiveness of these approaches in enforcing access control as shown in our experiments. Additionally, we also implemented a naive prompt-based access control baseline which fails to achieve any meaningful access control, while also being susceptible to potential jailbreaking attacks. We will include these new results and a thorough discussion in the paper's revision.

Regarding the comment "The methods provided do not scale well with increasing number of domains...": Since the problem of access control in LLMs has not been formalized before and ours is the first work in this effort, the major goals of our work are to formalize access control enforcement in LLMs and introduce empirical metrics to evaluate the effectiveness. As we stated in our limitations section, the scalability of our approach to large number of domains (order of thousands) is not feasible at the moment. For large number of domains or permission groups, we can cluster similar permission groups into hierarchical clusters and train the PEFT modules on the clusters. We leave this extension for future work.

We thank the reviewer for their detailed feedback and for acknowledging the practical significance of the problem and our evaluation framework. Below we would like to clarify the concerns and questions raised:

On the reliance on MIA success

The reviewer raised a valid concern that the DDI evaluation is only as good as the best MIA in their comment "One particular weakness in the framework is the design of DDI, which utilizes membership inference attacks...".

We agree with the reviewer that the empirical success of DDI is dependent on the strongest membership inference attacks, which could get trickier for LLMs as the training set sizes increase in the general LLM training setting. However, in our scenario the security domains are distinct from each other in the data distribution sense, and as such even with larger domains the MIAs would be successful in distinguishing the domains.

Moreover, DDI is designed to be modular with respect to the choice of MIA. In our experiments, we already evaluate DDI using five different MIA techniques, demonstrating its flexibility and robustness across a range of attack strategies. As the field progresses and stronger and more refined MIAs are developed, they can be directly incorporated into the DDI calculation without modification to the framework. This design ensures that DDI remains adaptable and relevant in assessing access control effectiveness.

On the clarification of UGI metric

Regarding the comment "The notion of Access Advantage (Def. 2.2) and UGI (Def. 4.2) needs more clarification. From a user perspective, one would certainly expect that an LLM should not suffer utility loss on any query. Suppose the model can generalize on a particular domain based on other pre-trained data alone; should the model be penalized for its output even though it has not observed this domain?": We note that there is no penalization on model output/utility with Permissioned LLMs. If the model generalizes well with just its pre-training data, then it will perform well on queries related to domains the user does not have access to. However, this does not imply the access control does not work as intended. The access advantage definition only bounds the gap in model utility between the case where the user has access to a domain and the case where the user has no access to that domain, which includes any model utility obtained from learnings without the model gleaning the data from the domain the user doesn't have access to. We will clarify this in the revision.

Regarding the comment "Can you clarify why the relevance score function is the utility value itself (Sec. 4.2)?...": For UGI, we are using the gap in utility, which is why the relevance score defined in Def 2.2 is the utility for the target domain for the UGI instantiation. Other access advantage instantiations could use different relevance score metrics, for instance DDI uses AUC and TPR@(low)FPR as relevance score metrics.

We thank the reviewer for their detailed feedback and for acknowledging the merits of our access control formalism and evaluation framework. We would like to clarify the concerns and questions raised in the review:

Scalability and applicability to complex settings

The reviewer raised some valid concerns about the scalability of the approach in the comments "It would be beneficial to discuss how PermLLMs scale..." and " How do the proposed PermLLM mechanisms scale to very large numbers of users...".

A big part of the goals of our work was to formalize access control enforcement in LLMs and introduce empirical metrics to evaluate its effectiveness. In our paper, we focus on domain-level segregation as a foundation step. Our access control algorithms (Activate, Merge, and Union), though simple, are concrete mechanisms that achieve the domain-level segregation of LLM responses based on user access privileges. More complex scenarios such as scalability, fine-grained access and hierarchical domains would require additional research and is left for future work. That said, we believe that finer-grained or attribute-based access control policies can, in principle, be reduced to controlled filtering or masking of training inputs within the PermLLM framework. For example, filtering specific attribute values before adapter training enables extension toward such policies without fundamental changes to the framework.

As we stated in our limitations section, we believe the scalability of our approach to large number of domains (order of thousands) is not feasible at the moment. For large number of users or permission groups, we can cluster similar permission groups into hierarchical clusters and train the PEFT modules on the clusters. We leave this extension for future work.

Performance overhead of Permissioned LLMs

Regarding the comment "...the paper does not explicitly detail the performance overhead...": We can add the additional memory footprint data for our permissioned LLMs in terms of additional PEFT model parameters in the revision. We do not observe any increase in training cost for Active, for Merge the cost is nominal and independent of training data size or training time. For Union, we require training the models on the union of domains which scales the training time with the number of domain sets and their sizes. Although this scaling can inflate to 2^n sets of domains in principle, in practice, we expect the sets of domains to be much lower in many important application settings.

Clarification on utility–access control trade-offs

Regarding the comment "...discussion on the inherent trade-offs between strict access control enforcement and the LLM's overall utility...": The UGI metric measures the gap in model utility between a domain a user has access to and a domain the user doesn't have access to. It thus impacts the utility for unauthorized users by design. It does not, however, trade-off the utility for users with legitimate access to security domains even for complex queries as long as they have access to the domains required to answer the query. The UGI could however be upper-bounded by the model generalizability across different security domains in cases where the domains have correlations and/or when learnings from one security domain help answer queries for another security domain. In such scenarios, a user with no access to a target domain could still get relevant answers from a model that has generalized the learnings from other domains the user has access to, and the access controls are not violated as long as the model does not glean over the data from target domain to answer the user's query. We do not consider any cross-domain correlations in our experiment setup to keep the analysis straightforward.