Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimension

We thank the reviewer for their constructive suggestions and supportive feedback. We are glad that they found our paper well-written and provided novel perspectives on W2S. Since we could not include figures in OpenReview rebuttal, we will synopsize our new experiments in text and present the formal results in an anonymous URL: https://anonymous.4open.science/r/W2S_IntrinsicDim_demo-FEAF/demo.pdf.

Key clarifications

1. Additional experiments on image regression and LLMs: We appreciate the suggestion on strengthening the experimental validation. In the revision, we will include additional real experiments on image regression (UTKFace) based on CLIP and ResNet embeddings, as well as NLP task (SST-2) based on finetuning LLMs (see the URL for concrete plots). In particular, our new experiments provide strong empirical evidence for our theoretical results:

   1. Discrepancies lead to better W2S: As Figures 4 and 5 in the submission, we plot the scaling of PGR/OPR against sample sizes $n,N$ for weak-strong pairs with various correlation dimensions on real vision and NLP tasks. The results show that the lower $d_{s \wedge w}/d_w$ (larger discrepancy) consistently brings better W2S.
   2. Variance reduction is a key advantage of W2S: While the variance-dominated regime that we focus on is overlooked by most prior theoretical works on W2S, our new experiments confirm that this is a particularly important regime where strong W2S performance (in terms of PGR/OPR) occurs in practice. By injecting artificial label noise that leads to larger variance, we observe significant improvements in PGR and OPR, implying that variance reduction is a key advantage of W2S.

2. Our analysis extends to weak-strong pairs with different feature dimensions: Please see R#2 (WLsf), Key#3 for detailed explanations.

3. Empirical estimation of intrinsic and correlation dimensions: Thanks for the question. In practice, the feature dimensions of the weak and strong models can be different. Please see R#1 (5dnE), Key#2.1 and 2.2 for how we estimate $d_{s \wedge w}$ in practice.

4. Our main technical contributions are:

   1. the theoretical framework based on low intrinsic dimensions of finetuning that provides a theoretical underpinning for W2S in the variance-dominated regime, along with
   2. the insight on how weak-strong discrepancy brings better W2S generalization.

   As pointed out in R#2 (WLsf), Key#4, W2S has shown appealing empirical performance when variance dominates (e.g., see Key#1.2). However, the variance-dominated regime is often overlooked in prior analyses on W2S. (We kindly point out the confusion that Ildiz et al. (2024) explain W2S from the bias instead of variance perspective. We will further clarify this in the revision.) To the best of our knowledge, this is the first theoretical explanation for W2S in the important variance-dominated regime. While our analysis is built upon various fundamental tools in random matrix theory, high-dimensional statistics, and linear algebra, the unique structure of the learning problem posed by W2S makes the combination of these tools highly non-trivial. Moreover, our framework is flexible enough to be extended to finetuning problems with similar structures like knowledge distillation and model collapse. We will clarify this in the revision.

Detailed responses

We appreciate the detailed suggestions on notations and presentation. Here, we address questions in "Other Comments Or Suggestions" (O):

• O#1: We assume $f_*$ being bounded for conciseness of the analysis. This can be trivially relaxed by adding a constant $C = \max_x |f(x)|$ to the generalization error in our analysis.
• O#2, 3: In Case I, $(\rho_s + \rho_w)/\sigma^2 \to 0$ essentially says that the FT approximation errors are small enough compared to the label noise so that we can treat them as zeros. In contrast, for Case II, we define a small constant $\varrho = (\rho_s + \rho_w)/\sigma^2 > 0$ to quantify the variance domination. This $\varrho$ affects the PGR and OPR lower bounds in Coro. 3.6. While the quantitative conclusions are different, it's correct that the qualitative takeaways for both cases are similar.
• O#4: We agree. This is exactly a simple explanation for how $f_s$ eventually outperforms $f_{w2s}$ as $n$ increases. We will rephrase the sentence to avoid ambiguity.
• O#5: The reason why we did not report the spectra is that they do not contain much relevant information. See the spectra of ResNet embeddings in our new UTKFace experiments for examples: https://anonymous.4open.science/r/W2S_IntrinsicDim_demo-FEAF/fig/utkface_svd_cutoff0.99.pdf.

We are happy to answer any further questions you may have. If our responses above help address your concerns, we would truly appreciate a re-evaluation accordingly.

We thank the reviewer for their time and suggestions. We are glad that they found our paper well-written and provides novel understanding for W2S. Since we could not include figures in OpenReview rebuttal, we will synopsize our new experiments in text and present the formal results in an anonymous URL: https://anonymous.4open.science/r/W2S_IntrinsicDim_demo-FEAF/demo.pdf.

Key clarifications

1. The key intuitions on how discrepancies lead to better W2S are unrolled in two steps in the introduction:

   1. In Lines 68-79, we first break down our main theoretical result $Var(f_{w2s}) \asymp \frac{d_{s \wedge w}}{n} + \frac{d_s}{N}\frac{d_w - d_{s \wedge w}}{n}$ to explain why a larger discrepancy (smaller $d_{s \wedge w}$) leads to better W2S. That is, the strong student mimics variance of the weak teacher in the overlapped subspace of weak and strong features (with dimension $d_{s \wedge w}$). In contrast, variance in the discrepancy subspace between weak and strong features (with dimension $d_w - d_{s \wedge w}$) is reduced by a factor of $\frac{d_s}{N}$ in W2S. Recall that $d_s$ is the low intrinsic dimension, and the unlabeled W2S sample size $N$ is generally large in practice. Therefore, a multiplicative factor of $\frac{d_s}{N}$ significantly reduces the variance in the discrepancy subspace, leading to better W2S performance.

   2. Then, in Lines 91-107, we use an example to provide high-level intuition for how variance reduction in the discrepancy subspace happens. In particular, we consider a downstream task on classifying the brands of vehicles based on their images.

      • The weak features, spanning a low-dimensional subspace $\mathcal{V}_w$, capture the designs of vehicles that tend to be more complicated (with a higher intrinsic dimension $d_w$) but contain irrelevant/spurious information that makes the model weak for the downstream task.
      • The strong features, spanning a low-dimensional subspace $\mathcal{V}_s$, capture the logos of vehicles that are often simpler (with a lower intrinsic dimension $d_s$) and more relevant for the downstream task.

      Since the design and logo of a vehicle are typically irrelevant, $\mathcal{V}\_w$ and $\mathcal{V}\_s$ are likely to be almost orthogonal in a high-dimensional feature space, leading to a small $d_{s \wedge w}$. Then, the weak teacher $f_w$ brought by noisy SFT labels will make mistakes that only correlate to the design features in $\mathcal{V}_w$, independent of the logo features in $\mathcal{V}_s$. Such errors in weak supervision can be viewed as independent label noise with respect to the strong features in $\mathcal{V}_s$. With an intrinsic dimension $d_s$, the generalization error of strong student induced by such independent label noise vanishes at a rate of $\frac{d_s}{N}$ (following the intuition of classical regression analysis).

   We will further clarify these key intuitions in the revision.

2. Empirical estimation of intrinsic and correlation dimensions: Please see R#1 (5dnE), Key#2.

3. Additional experiments on image regression and LLMs: In the URL, we included additional real experiments on image regression (UTKFace) based on CLIP and ResNet embeddings, as well as NLP task (SST-2) based on finetuning LLMs. In particular, our new experiments provide strong empirical evidence for our theoretical results by showing that (i) model discrepancies lead to better W2S; and (ii) variance reduction is a key advantage of W2S. Please see R#4 (W6Uk), Key#1 for details.

Detailed responses

• Discussion on concurrent works: We are keeping track of relevant concurrent works on W2S and will include discussions on them in the revision.
• We appreciate all the detailed suggestions in "Other Comments Or Suggestions" and will revise accordingly in the next version.

We are happy to answer any further questions you may have. If our responses above help address your concerns, we would truly appreciate a re-evaluation accordingly.

We appreciate the constructive suggestions from the reviewer. We are glad that they found our paper well-presented and our perspective novel. Since we could not include figures in OpenReview rebuttal, we will synopsize our new experiments in text and present the formal results in an anonymous URL: https://anonymous.4open.science/r/W2S_IntrinsicDim_demo-FEAF/demo.pdf.

Key clarifications

1. Additional experiments on image regression and LLMs: In the URL, we included additional real experiments on image regression (UTKFace) based on CLIP and ResNet embeddings, as well as NLP task (SST-2) based on finetuning LLMs. Please see R#4 (W6Uk), Key#1 for details.

2. Empirical estimation of intrinsic and correlation dimensions: Please see R#1 (5dnE), Key#2.

3. Our analysis extends to weak-strong pairs with different feature dimensions: Notice that since the intrinsic dimensions are far lower than the feature dimensions, the larger feature dimension (e.g., $D_s$) can always be reduced to the smaller one ($D_w < D_s$) via random projection (Johnson-Lindenstrauss transforms) with a negligible information loss $\epsilon \asymp d_s / D_w$. This is the main idea of the empirical estimation in Key#2. Since the high feature dimensions are not essential in our setting, we consider $D_s = D_w = d$ for a clean analysis. We will clarify this in the revision.

4. Variance-dominated regime is crucial for understanding W2S, focusing on which does not compromise our contributions: We respectfully disagree with the reviewer and emphasize that our focus on variance-dominated regime is a strength rather than a weakness. This setting is empirically motivated and fills an important gap in existing theoretical understanding. As our motivation in Lines 23-26 right said, empirical evidence in (Burns et al. 2023) suggests that larger W2S gain tends to occur on easier tasks (i.e., variance-dominated tasks). This can also be observed in our synthetic experiments (cf. Fig. 2&3) and our new vision and NLP experiments (see R#4 (W6Uk), Key#1.2). Despite the appealing empirical performance of W2S in variance-dominated regimes, to the best of our knowledge, this regime has never been rigorously studied in prior theories on W2S. Our work fills this gap by providing a descriptive yet clean theoretical underpinning for W2S in the variance-dominated regime under finite samples.

Detailed responses

• Concern in "Theoretical Claims": Notice that taking expectation over the training dataset is a standard way to connect the excess risks (ER) of random and fixed designs: $\text{random design ER}=\mathbb{E}\_{S_x \sim \mathcal{D}^N}[\text{fixed design ER over }S_x]$ Thanks to the reviewer's question, we realized that the inner expectation $\mathbb{E}\_{\theta_{w2s}}$ with shared randomness between $S_x$ and $\theta_{w2s}$ could be misleading (although it is technically correct as $\theta_{w2s}$ inside is conditioned on $S_x$). A better expression for the right-hand side may be $\mathbb{E}\_{S_x,\widetilde{S}}[||\Phi_s\theta - f_*||^2/N]$. We will revise these notations in the proofs.

• "Questions For Authors" QFA#3,5,6 are addressed in the Key clarifications.

• QFA#1: The variance-bias decomposition in Lines 595-604 is equivalent to directly deriving $Var(f)$ and $Bias(f)$ in Lines 121-123, both following the standard variance-bias decomposition for linear regression (see e.g. Liang, (2016), Statistical learning theory, Sec 2.9). The key observation here is that when opening the square in Line 600, the cross term vanishes because $\widetilde{z}$ is an independent random vector with zero mean.

• QFA#2: First, as the analysis in Sec 3.2 shows, even if the weak teacher lacks capacity (FT approximation error $\rho_w$ is large), as long as the label noise $\sigma^2$ dominates $\rho_w$ (intuitively, harder tasks are more likely to have noisy labels), the problem still falls in the variance-dominated regime. Second, beyond the variance-dominated regime, our theory suggests a degeneration of W2S performance due to the vanishing advantage in variance reduction. This is empirically confirmed in both synthetic and real experiments (see discussions in Key#4).

• QFA#4: Our analysis is not directly related to early stopping, but there are shared insights. Early stopping has a known intuitive connection with weight decay, which translates to ridge regression in our setting. As footnote 3 explained, ridge regression effectively brings low intrinsic dimensions by filtering out small eigenvalues. In the revision, we formally extend our analysis to ridge regression (see R#1 (5dnE), Key#4 or the URL). We show that a suitable choice of ridge parameter (or early stopping intuitively) can bring better W2S performance by revealing the underlying low intrinsic dimensions.

We are happy to answer any further questions you may have. If our responses above help address your concerns, we would truly appreciate a re-evaluation accordingly.

We thank the reviewer for their time and suggestions. We are glad that they found this work well-presented and provided a good understanding of W2S. Since we could not include figures in the OpenReview rebuttal, we will synopsize our new experiments in text and present the formal results in an anonymous URL: https://anonymous.4open.science/r/W2S_IntrinsicDim_demo-FEAF/demo.pdf.

Key clarifications

1. Additional experiments on image regression and LLMs: We appreciate the suggestion on strengthening the experimental validation. In the URL, we included additional real experiments on image regression (UTKFace) based on CLIP and ResNet embeddings, as well as NLP task (SST-2) based on finetuning LLMs. Please see R#4 (W6Uk), Key#1 for detailed discussions.

2. Empirical estimation of intrinsic and correlation dimensions: We first highlight that with small finetunable parameter counts (e.g., linear probing or finetuning last few layers), we estimate intrinsic dimensions based on traces of data covariances (see Line 413-419, i.e., the minimum rank for the best low-rank approximation of $\Sigma_w, \Sigma_s$ with a relative error in trace less than $\tau=0.01$).

   1. For correlation dimension, a practical challenge is that the feature dimensions of the weak and strong model can be different--$V_s, V_w$ of sizes $D_s \times d_s$ and $D_w \times d_w$ can have $D_s \ne D_w$. In this case, we gauge the correlation dimension by matching $V_s, V_w$ through a random unitary matrix $G \in \mathbb{R}^{D_s \times D_w}$ s.t. $d_{s \wedge w} = ||V_s^\top G V_w||^2_F$. This provides a good estimation for $d_{s \wedge w}$ because with low intrinsic dimensions $\max(d_s, d_w) \ll D_s, D_w$ in practice, mild dimension reduction through $G$ well preserves the essential information in $V_s, V_w$.
   2. We appreciate the question on empirical estimations of intrinsic and correlation dimensions for finetuning large models. When finetuning LLMs, $D_s, D_w$ will be on the order of millions, making the covariance-based estimation infeasible. In this case, we use the SAID method proposed by Aghajanyan et al. (2020) to estimate intrinsic dimensions. Following Remark 2.5, for full FT, we take $\Phi_s, \Phi_w$ as the gradients of the strong and weak models at pretrained initializations. We use randomized rangefinder based on sparse JLTs and the random unitary projection trick above to estimate $d_{s \wedge w}$ efficiently (see Appendix C in the above URL for detailed procedures).

3. Our assumptions are reasonable and generalizable. We kindly highlight that footnote 4 and Theorem 3.1 provide explicit pointers to Theorem A.1, the formal version of Theorem 3.1 that rigorously extends the results to sub-Gaussian features. As explained in footnote 4, both theorems convey the same message. Due to page limit, we present the Gaussian results in the main text for clarity. Meanwhile, in the introduction, we provide strong motivations for studying FT in the kernel regime. The choice of subgaussian features is also well-justified by literature (Wei et al., 2022) and related works (Wu & Sahai, 2024; Ildiz et al., 2024) on W2S. Most importantly, the empirical evidence in our new experiments demonstrates the generalizability of our assumptions and results to real-world scenarios.

4. Ridge regression analysis: Our analysis can be extended to the ridge regression setting. As mentioned in footnote 1, when $\Sigma_s, \Sigma_w$ admit full ranks with fast decaying eigenvalues, ridge regression effectively brings low intrinsic dimensions by filtering out the small eigenvalues. The result for ridge regression again conveys the same message as Theorem 3.1, with the correlation dimension replaced by $tr(\Sigma_s \Sigma_w)$. Intuitively, a large discrepancy corresponds to a $\Sigma_s, \Sigma_w$ pair with approximately orthogonal leading eigenvectors associated with the large eigenvalues, bringing a small $tr(\Sigma_s \Sigma_w)$. (See Appendix A in the above URL for detailed statements and proofs.)

Detailed responses

• The practical implications of our theoretical insights on the choice of weak v.s. strong models and sample sizes in W2S are self-evident and discussed in detail in Sections 3.2 and 5. We will try to further emphasize them in the revision.

• Assuming both weak and strong models have sufficient capacities to achieve low FT approximation errors on the downstream task, our theory and experiments show that the larger discrepancy between weak and strong models brings better W2S, with no potential downsides.

We are happy to answer any further questions you may have. If our responses above have addressed your concerns, we would truly appreciate a re-evaluation accordingly.