Independence Tests for Language Models

We thank the reviewer for their time and address some of their concerns below. We will add the references suggested.

$**Claims and Evidence Comment 1**$

For the constrained test, we in fact guarantee that the test leads to exact p-values under the null hypothesis when two models are independent, i.e. two independent models will not lead to a low p-value.

But with regards to a Type II error, we agree that our tests do not inherently guarantee that two non-independent models will always lead to a low p-value. However, empirically we find this is true in all our experiments. Specifically, Figures 5 and 6 in Appendix E show that on all (69) dependent Llama 7B pairs, the tests $\phi_{U}$ and $\phi_H$ yield p-values less than 2.2e-308 (which is our Type-II error rate, i.e. the maximum p-value we observe in the case that the null hypothesis (independence) does not hold). The same holds for $\phi_\text{MATCH}$ (Figure 7). We believe that this is sufficiently addressed through evidence from the Llama 7B and 70B results. We note that with regards to a Type I error, that our theorem does guarantee a uniform distribution for two independent models under the equivariance assumption, which we discuss more below.

$**Claims and Evidence Comment 2**$

Standard machine learning algorithms such as SGD, which are the most common algorithms for training language models follow the equivariance condition, as the gradients are permutation-equivariant, explained briefly in Example 2. We also emphasize that only one of the models needs to satisfy these assumptions, so a trusted model’s developer (who used SGD for example) can run this test without assumptions on the training strategy for another model.

$**Methods and Evaluations Criteria**$

The independence is statistical independence of two random variables, $\theta_1$ and $\theta_2$ (equivalently, zero mutual information), i.e. $X \perp Y$ if and only if $P(X|Y) = P(X)$. We will clarify this in a footnote in updated writing.

In this case, if $\theta_1^0$ and $\theta_2^0$ are independent random variables, then by the post-processing inequality, since $A$ is a (deterministic) function, then $A(\theta_1^0)$ and $A(\theta_2^0)$ are also independent. We will add these details and this explanation.

The Type-I error rate is the $\alpha$ threshold determined by the test user, since our tests give p-values. If the user chooses a threshold of $\alpha = 0.0001$ for example, then the Type-I error rate is 0.0001. This result is a consequence of our Theorem 1 with the equivariance condition. Further, we also plotted the null distribution for the 141 independent model pairs and found that the values are uniformly distributed, with exact values shown in Figure 6, for example.

In our experiments (Figures 5, 6, 7 in the Appendix) (using ground truth from Hugging Face), choosing $\alpha =$ 1e-307 would yield a Type-I error rate of 1e-307 and a Type-II error rate of 0.

$**Experimental Designs Or Analyses**$

Our experiments are already conducted on neural networks; $\phi_U$, $\phi_H$, and $\phi_\text{MATCH}$ are all defined over the individual MLP component of the Transformer models, which are in principle simple neural networks. For example, $\phi_U$ uses the weights of the up-projection matrix, i.e. one layer of a neural network.

By varying model size and the dimensions of the weight / activation matrices from 1B to 7B to 70B models, we test the effectiveness and robustness of the test; we also discuss Type-I and Type-II errors above.

Our tests $\phi_U$, $\phi_H$, and $\phi_\text{MATCH}$ do not use a $T$ value, as they follow the general form provided in Equation (2). In principle, $T$ is actually the total number of permutations, which is why we get a very low p-value like e-308.

$**Theoretical Claims comment and Weakness 1**$

We agree that our proof holds for a broad class of machine learning models and neural networks. We intentionally chose this to highlight the generality of our method, but we focused our empirical experiments on language models due to their widespread re-use and the subsequent intellectual property concerns. Specifically, language model capabilities and pretraining costs are growing, which makes models more at risk of being stolen. Furthermore, many parties will fine tune an open-source model for a downstream task rather than pretraining their own model. We briefly discussed this in the second paragraph of the introduction but can further expand on the motivation. But, we note that the unconstrained test is geared towards language models (GLU MLPs).

We acknowledge your concern about model-specific considerations and would appreciate it if you could point out any aspects that would significantly benefit from incorporating more language model specific characteristics, and if there was a specific issue you found with the proof.

We will also work on the clarity of writing and appreciate the in-depth feedback.

Thank you!

We thank the reviewer for their time and positive feedback!

We find the tests also work on smaller-scale models such as the Phi-3 family. Both $\phi_\text{CSH}$ and $\phi_\text{MATCH}$ return a statistic approximately 1e-308 (aggregated with Fisher’s method) on the fine-tuned model pair microsoft/Phi-3.5-mini-instruct and numind/NuExtract-v1.5 (3.8B parameters). We are also happy to add more experiments on other model families and will update our experiment section in our paper

Thank you!

We thank the reviewer for their time and positive feedback!

$**Weakness 1**$

Thank you for bringing up this concern. Standard machine learning algorithms such as SGD which are the most common algorithms for training language models follow the equivariance condition, as the gradients are permutation-equivariant, explained briefly in Example 2. We also emphasize that only one of the models needs to satisfy these assumptions, so a trusted model’s developer (who used SGD for example) can run this test without assumptions on the training strategy for another model.

$**Weakness 2**$

Yes, we agree, and we do not claim our test is in fact robust to all adversarial attacks. We have at least empirically validated that our test is robust to a superset of attacks over prior work (Zeng et al. 2024).

$**Weakness 3**$

We agree that the final model weights are heavily influenced by the training data. However, in our definition of a learning algorithm (Section 2.1) we write “$A$ includes the choice of training data…” Given the training data, minibatch ordering, and other parameters, then in fact $A$ is deterministic and it is possible to fully reproduce the learning algorithm given the initial weights.

This is an abstraction to simplify our framework and may be an unconventional way to describe learning algorithms, so we will add more clarification. We also acknowledge there are other sources of randomness which deterministic functions with a fixed seed do not capture such as dropout; thus, in Appendix A, we state and prove a more general version of Theorem 1 for randomized learning algorithms, for which our conclusions still hold.

Thank you, and we are happy to answer more questions!

We thank the reviewer for their time and feedback! We will add the references mentioned.

$**Experimental Designs Or Analyses Comment 1**$

We report experiments on Llama 70B, the hybrid StripedHyena and Mistral model, and (distilled) GPT 2 models in Table 5, 9, and 15 and find our statistics work for these different architectures as well. In Reviewer RDaz's comment, we also experiment with smaller Phi models and find the test holds high power for those models too. We believe these models likely encompass a broad range of training regimes.

In Table 7 of the Appendix, we also run our tests on adversarially-transformed models and show how our unconstrained test is robust to the transformation (whereas prior work is not). If there are other model architectures or families that would be beneficial, we would be happy to add those experiments!

$**Experimental Designs Or Analyses Comment 2**$

We include more discussion and experiments in Appendix F.1 (HuREF Invariants) but are happy to move discussion to the main text. Specifically in Table 7, we demonstrate how our adversarial transformation can be used to break the HuREF invariants — each of the transformed Llama-2-7b-hf, vicuna-7b-v1.5, and Nous-Hermes-llama-2-7b models have low $M_a$, $M_b$, and $M_c$ values when compared with Llama-2-7b-hf, whereas our unconstrained statistic $\phi_\text{MATCH}$ gives a value of 2.2e-308.

We also mention in the paragraph at Line 96 of the introduction that our tests yield p-values whereas Zeng et al.’s do not. Please let us know if there is more we can provide about comparison with Zeng et al. (2024).

$**Weakness 1**$

We agree. But in those cases, then causal relationships can be determined by first using our test, then using metadata, such as the dates of model releases.

$**Weakness 2**$

We report our times on a Nvidia RTX A6000. For $\phi_H$, the bottleneck is computing the forward pass to obtain the intermediate activations — and on two 7B models, the test on all 32 Transformer blocks combined takes on average less than 2 minutes. $\phi_\text{MATCH}$ requires the forward pass and aligning the activations; on dependent models, this will also take around 2 minutes total, whereas it may take 5-10 minutes per Transformer block for independent models. We believe this is reasonable computationally. We will include these details.

$**Weakness 3**$

The hidden dimension $h$ is determined by the model weights, i.e. for Llama 2-7B the hidden dimension is 11008. This dimension varies for the models we test (28672 for Llama 2-70B, 8192 for Llama 3.2-3B) but do not affect the strength of our tests.

In our paper, we only report results for using WikiText as the input distribution, but in fact we are able to achieve similar performance using random tokens from the vocabulary as the input distribution. We will include further experiments and ablations on this distribution.

$**Question 1**$

We choose the gate and up projections of an MLP because they are two matrices that are combined via a direct product and activation function — which makes significant transformations to the weights difficult. However, the results in Figure 2 from the GLU MLP remain a conjecture. We reason that the gate and up projections are trained to be very aligned during training with a very large loss landscape for two independent models.

$**Question 2**$

Empirically, we find our results hold with a threshold of even 2.2e-308, and e-61 for the generalized unconstrained test where we distill the MLP. In practice, a third party or model provider may choose $\alpha =$ 1e-5 for example, or their own confidence level.

$**Question 3**$

We have tested non-bijective cases such as between Llama 3.1-8B and Llama 3.2-3B, where the dimension is reduced from 11008 to 8192 (more than 30%). We test $\phi_\text{MATCH}$ on many pruned models including the Nvidia Minitron models and Sheared Llama models in Appendix I.2, where the bijective assumption does not hold and find that the test is still strong.

Also, if the reviewer is asking about many-to-many sub-models, we also run experiments, such as with the hybrid StripedHyena model, where only some layers are taken from the Mistral 7B model (i.e. embedding) but not others (MLP projections). We are happy to run more experiments or provide more clarification.

We will also add more explanation with regards to equivariance and invariance. We appreciate the in-depth review and are happy to answer more questions!

We thank the reviewer for their time and positive feedback! We will add the Hemerik and Goeman reference mentioned, thank you.

About some of the weaknesses discussed:

From our empirical results, the power of our constrained and unconstrained test is as low as 2.2e-308 — in the cases of rejecting the null hypothesis, the p-value derived is less than 2.2e-308. (See Figure 5, 6, 7 in the Appendix.) In particular, in the best case where $\theta_1 = \theta_2$, then the power will be exp(- # hidden units).

The test could have trivial power in the case where the learning algorithms are constant functions, then the p-values will be non-significant for any initializations even if they are non-independent. However, this would never occur in practice for any non-trivially trained language model, and our tests have strong power for the wide array of language models we evaluate. We agree that the "in general" does a lot of work, so we only make our claims here via experimental results. We are happy to amend our writing to better calibrate to your feedback

"For the retraining and distilling test": The unconstrained test is robust to permutations --- permuting the hidden units before and after the MLP (or if we permute the entire model) and distilling the permuted model would not change the efficacy of the test, as the gate and up projection matrices would need to share that original permutation.

We will also apply the edits from "Other Comments or Suggestions." We are happy to answer more questions and appreciate the in-depth response!