Divergence at the Interpolation Threshold: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle

We appreciate your feedback on our paper. Concrete, actionable feedback on how to improve is extremely valuable.

Below is an incomplete response that we will complete tomorrow.

I think the statement on page 1 that "our goal is to [...] mathematical tool" might be a very slight misrepresentation without mentioning that this is restricted to linear models.

Agreed. We will explicitly state the paper considers only linear models.

I thought Fig 1 to be a strange choice and would prefer to see something that illustrates the message of the paper.

Agreed. We moved the (previous) Fig 1 to Section 5.

Why not provide an illustrative plot of double descent in test loss with the decomposition of the three factors plotted in parallel?

We like this suggestion and tried reorganizing the figures to more clearly communicate this (see Page 3). Tomorrow, we’ll try creating a new figure that plots the 3 factors separately from their ablations.

The other figures could also be improved in terms of clarity. The text size is far too small. The legends are sometimes incomplete (e.g. Fig 3 + 4 - what are the blue lines?). The titles are unclear (what does "condition" mean?)

Text size has been increased, legends made more clear, and titles edited. “Condition” meant “Ablation”; this should now be more clear.

I thought Fig 7 in particular could do with some work. It is currently very difficult to understand exactly what is happening in each panel.

The overall exposition has been tightened up. We have expanded the explanation of the previous Fig 7 (now Fig 6) in the figure caption.

It would also be nice to explicitly link each point to its corresponding plot.

Thank you - fixed!

There seems to be some contradiction in the ablation discussion on p7+8. [...] however, the actual ablation implemented is to "sweep different singular value cutoffs" and remove corresponding values. These are different things

Your description is correct. We have fixed the incorrect text.

It would be nice to include the code used to produce the results in this paper such that readers can easily interact with the experiments.

Code will be made publicly available. In the interim, we’ll share code with you tomorrow.

page 5, point 2 references the wrong figure.

Thank you - fixed!

I thank the authors for their partial response and I look forward to reading the complete response (and updated manuscript) once posted.

The result is very specific to least squares regression.

This is true, but our goal was not to explain everything, nor was this our claimed contribution. Rather, our claimed contribution is to study the simplest model possible that exhibits this phenomenon of interest, and explain what causes it and why in quantitative terms with as much generality and intuition as possible.

In other words, our contribution is not new conclusions, but an identification of the three general factors that are necessary to produce the divergence, along with a quantitative explanation of (a) why they are the correct factors to consider and (b) how they together interact to produce the divergence.

If you know of prior work identifying these three factors and explaining why they generally matter, please do let us know!

We appreciate your feedback on our paper. In response:

The main results are contained in previous literature [...] I appreciate the authors intention summarise and provide their own point of view to these results. However, I also feel that their justification that these previous works are "complex", "difficult" and "muddies the water" is subjective and comes across borderline disrespectful.

We agree that the main results are contained in previous literature through one lens or another, and we have deep appreciation for the prior work.

However, we believe that “complex” and “difficult” are objectively true. Our paper requires only two mathematical primitives: SVD and matrix calculus. We know of no other paper this simple.

In contrast, consider the techniques used by some previous papers:

• Wu & Xu 2020: Bounded 12th absolute central moment, asymptotic prediction risk, Stieltjes transform of limiting distribution described by Marchenko–Pastur distribution, self-consistent equations
• Mei & Montanari 2022: random features models, proportional asymptotics, Gaussian processes, weak differentiability, convergence in probability, random matrix theory including Marchenko-Pastur distribution, Stieltjes transform
• Hastie et al. 2022: Proportional asymptotics, uniformly bounded moments, sequences of deterministic PD matrices, random matrix theory, concentration of measure

These techniques are harder than SVD + vector derivatives. We mean no disrespect by "complex" or "difficult". Our goal in writing this manuscript is to explain this phenomenon as simply as possible, to complement the precise-but-narrow results of previous papers with an imprecise-but-more-general result.

Regarding “muddies the water,” we would be happy to use alternative phrasing. Our point - which we stand by - is that the numerous, specific and differing assumptions used by prior papers makes it difficult to determine which properties are necessary and/or sufficient for the phenomenon of interest. For example, consider the sentence:

The fact that double descent can occur with zero label noise due to model misspecification appeared in [Mei & Montanari 2022

While Mei & Monanari 2022 did study model misspecification, they make multiple assumptions: covariates are uniform on the hypersphere, y has i.i.d. additive noise with finite 4th moment, the model is ridge regression, the learning algorithm is gradient flow, the solution requires large $N, n, d$, the nonlinear model component is a centered isotropic GP, the nonlinearity must be weakly differentiable, etc. Theorem 1’s result is that the prediction risk converges in probability to $(F_1^2 + \tau^2) \mathscr{R}(\rho, \zeta, \phi_1, \phi_2, \lambda/\mu_*^2)$. It is not obvious what qualitative behavior to generally expect, nor is it obvious how results change if assumptions change. Moreover, it is not obvious how the results depend on the particular model misspecification considered versus general model misspecification.

We adopted a methodology analogous to the model-experimental systems approach, prevalent in the natural sciences, in which simple and controllable toy models are studied. The goal is to develop an intuitive yet quantitative understanding of deep learning phenomena. Multiple published papers have built and non-rigorously analyze models of deep learning phenomena, with a prominent recent example of grokking:

[1] Power et al. Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets. [2] Liu et al. Towards Understanding Grokking: An Effective Theory of Representation Learning. [3] Thilak et al. The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon. [4] Nanda et al. Progress measures for grokking via mechanistic interpretability.

this terminology is misleading, since in least-squares increasing the dimension $D$ both increases the number of parameters and decreases the sample complexity "N/D".

We conceptually think of fixed $D$: “Consequently, rather than thinking about changing the number of parameters P , we’ll instead think about changing the number of data points N.” We considered using the terminology “under/overconstrained” but decided this terminology was less common.

The figures are small and hard to read. Different from Figs. 2-9, Fig. 1 is not in a vector format, so it gets pixelised when zoomed.

We converted figures from PNGs to PDFs to improve readability. Figure 1 is not ours, but we moved it to a less prominent position. We are limited by space in making the figures larger.

To my best knowledge, the first works discussing the interpolation peak and how to mitigate them were [Opper et al. 1990; Krogh & Hertz (1991, 1992); Geman et al. 1992].

The earliest reference we found was Vallet 1989’s “The Hebb rule for learning linearly separable boolean functions: learning and generalization,” which we of course cited. We appreciate and have added these additional citations.

Based on our reviews, this paper will likely be rejected and it doesn't make sense to invest additional time (or ask more of yours). We think it makes more sense to submit to the ICLR 2024 blog track https://iclr-blogposts.github.io/2024/about/.

Thank you for your feedback and time.

Thank you for your valuable feedback and insights on our manuscript. We appreciate the time and effort you've dedicated to reviewing our work. We would like to address the concerns raised and provide additional context to clarify our contributions to the field.

it's unclear how this advances understanding beyond existing literature. Notably, there is an extensive body of work on double descent in the context of Kernel Ridge Regression (you can think of linear regression as an special case), such as the detailed discussion in Montanari et al. (https://arxiv.org/pdf/2308.13431.pdf), and references there.

Our work was motivated by a conversation with Montanari himself. We had a different research problem, related to whether one should expect a particular overparameterized model to generalize in a particular setting, and met with Montanari who kindly walked us through different results for multiple analytically solvable models. But there was a clear gap: there was no indication of what we should expect in our particular setting because the rigorous precision of the models hamstrung their generality. We initially thought that our failure to understand was our own, but when the Anthropic paper was released, it became clear that the failure was the field’s.

To drive the point home, we can use the recommended Arxiv link. Page 4-5 states that we can consider a linearization so long as Equation 1.3.8. holds:

$L_n \leq \frac{1}{||\Phi^+ (y - f_n(\theta_0) ||_2 \phantom{\cdot } ||\Phi^+ || }$

When will this hold? When should one expect this to be applicable to their setting? The first major result (Theorem 1 on Page 11), while extremely precise, reveals zero intuition. Moreover, the models considered assume additive noise, which erroneously leads the reader to believe that noise is necessary, when in fact, noise is not necessary.

It's unclear how this advances understanding beyond existing literature.

Thank you for giving us a chance to clarify. These are our contributions:

• To the best of our knowledge, our paper is the first to enumerate the three factors that cause the divergence, or explain their significance and how they interact. Many have discussed the small non-zero singular values, but this is only one of three factors and is insufficient by itself. Perhaps you know of a reference explicitly and prominently identifying the three factors?
• Our paper identifies these factors in the simplest model possible, without making hyperspecific and limiting assumptions, while still describing the phenomenon of interest in its entirety
• Our paper clarifies several misconceptions, including: (a) the divergence does not require deep networks or nonlinear models; (b) the divergence does not require noise and can occur in a fully deterministic setting, a point that was previously obscured because multiple papers require assuming noise; (c) The divergence does not have a single cause and in fact requires all three factors to be present in order to occur.
• Our paper also provides (approximate) geometric intuition for the Marchenko–Pastur distribution.

If you would like, we would be happy to provide an extended Related Work section detailing what models and assumptions key previous papers considered. Perhaps this would be a satisfactory way to explain how our paper differs from theirs?

I think this analysis is a nice way of thinking about double descent, like for a blog post or a lecture note, but I do not see any significant contribution.

We adopted a methodology analogous to the model-experimental systems approach, prevalent in the natural sciences, in which simple and controllable toy models are studied. The goal is to develop an intuitive yet quantitative understanding of deep learning phenomena. Multiple published papers have built and non-rigorously analyze models of deep learning phenomena, with a prominent recent example of grokking [1, 2, 3, 4].

[1] Power et al. Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets.

[2] Liu et al. Towards Understanding Grokking: An Effective Theory of Representation Learning.

[3] Thilak et al. The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon.

[4] Nanda et al. Progress measures for grokking via mechanistic interpretability.

We hope this response clarifies the novelty and significance of our research. We are committed to further refining our paper to address these concerns and look forward to the opportunity to improve our manuscript based on your valuable feedback.

Based on our reviews, this paper will likely be rejected and it doesn't make sense to invest additional time (or ask more of yours). We think it makes more sense to submit to the ICLR 2024 blog track https://iclr-blogposts.github.io/2024/about/. Thank you for your feedback and time.

We sincerely appreciate the time you have invested in evaluating our manuscript. We are eager to address your concerns to enhance the quality of our work.

Line-by-Line Response

The models considered are regularized (singular value thresholding) [...]

We would like to clarify a factual inaccuracy regarding the use of singular value thresholding in our paper. Neither the mathematical model (OLS) nor Figures 2, 4, 5, 6, 7 have singular value thresholding. Only Figure 3 has singular value thresholding, specifically to make the point of singular value thresholding.

As a result, many of the models considered in the overparameterized regime do not interpolate.

We would also like to clarify this factual inaccuracy. The mathematical model (OLS) does interpolate in the overparameterized regime, and Figures 2, 3, 4, 5 likewise show interpolation in the overparameterized regime.

The arguments are not statistically precise, which makes them hard to trust.

We appreciate your point on the utility for statistical precision in arguments. However, our perspective is that many papers have already presented such precise arguments, but require overly specific and complex assumptions e.g. (8+m)-moments, that obscure core concepts and lead to incorrect beliefs such as: (1) small non-zero singular values are the sole cause of the divergence, or (2) noise is necessary for the divergence.

Our paper seeks to clarify such misunderstandings.

the analysis does not provide a crisp enough answer to warrant looking at the simplified model of OLS.

As we state in the paper, OLS is the simplest possible model we know of that exhibits the phenomenon of interest and that has known closed-form solutions which can be analyzed and understood; additionally, by using the closed-form solutions, we can exclude the choice of learning algorithm and its (possible) learning dynamics. We could analyze a more complicated model, but doing so would require significantly more effort and we doubt it would contribute much more.

Can you please clarify what you find inadequate about this rationale?

The discussion around prior work is very limited and does not fully and clearly explain what was known previously and what gap is this filling in the existing literature.

Our paper states what gap it fills in the literature:

• Abstract: “analytically solvable models in this area employ a range of assumptions and use complex techniques from random matrix theory, statistical mechanics, and kernel methods, making it difficult to assess when and why test error might diverge”
• Introduction: “a comprehensive understanding of why test loss behaves erratically at this threshold remains elusive. Many analytical models aiming to explain this behavior rely on a plethora of assumptions (e.g., i.i.d additive Gaussian noise, sub-Gaussian covariates, (8 + m)- moments) and use advanced proof techniques from random matrix theory, statistical mechanics, and kernel methods. This complexity muddies the waters, making it challenging to pinpoint the precise conditions leading to test error divergence”

To the best of our knowledge, we know of no other paper that identifies the three interpretable factors that we do. Many have discussed small non-zero singular values, but not the other two interpretable factors.

Would the reviewer recommend an extended “Related Work” section to be added, explicitly stating what previous work showed and how our work differs?

The points where the discussion is high level, e.g. the geometric intuition, the writing is hard to follow and the authors do not get their point across satisfactorily.

Regarding the clarity of our high-level discussions, such as the geometric intuition, we seek your specific feedback to improve our exposition. If there are particular sections or paragraphs, like “Divergence at the Interpolation Threshold”, that require more clarity, please let us know. We are committed to revising these sections to ensure they are accessible and clear to our readers.

Overall Response

We would like to again express our gratitude for your time and effort in reviewing our manuscript. Your insights are invaluable to the refinement of our work. We have carefully considered your comments and would like to address two areas where we believe further clarification could enhance our mutual understanding.

1. Regarding Factual Accuracies: We have noted a few instances in your review where the interpretation of our work might not align with what was presented in the manuscript. We would be grateful if you could revisit these sections, as we are keen to ensure that our work is evaluated based on its actual content.

2. On Specificity in the Review: We also observed that some aspects of the review discuss our paper in general terms, raising concerns that we address in the main text. It would greatly assist us if you could respond to the text in a specific, detailed manner.

Based on our reviews, this paper will likely be rejected and it doesn't make sense to invest additional time (or ask more of yours). We think it makes more sense to submit to the ICLR 2024 blog track https://iclr-blogposts.github.io/2024/about/.