Curvature Explains Loss of Plasticity

Thank you for your review, below we address your specific concerns:

Definition of curvature, scale of eigenvalues and rank as a normalized quantity.

Please see the shared reply, [Curvature Definitions], for more details

There are several definitions of curvature, but the Hessian describes the local curvature of the function. The srank of the Hessian gives the number of curvature directions and we are arguing this property of the curvature explains loss of plasticity.

The fact that the rank is a normalized quantity (normalized meaning it ignores the scale of the singular values) means that it provides information about the relative distribution of the singular values. Our findings suggest that this normalized quantity is useful for explaining plasticity.

include a figure whose x-axis is the srank and the y axis is the batch error

Please see shared reply [Correlation Plots] and updated manuscript (Appendix A.1, D.3 and D.4) with correlation plots.

Discussing “Symmetry Leads to Structured Constraint of Learning”

First, the paper that you link to was uploaded to arxiv after the ICLR submission deadline. We do appreciate the reference, as the paper looks quite interesting and relevant. Discussion of the linked paper, however, should not be relevant to the reviewing process as per the reviewer FAQ. That being said, we are not sure what conclusions to draw regarding linear networks in that paper. Furthermore, L2 regularization has been shown to mitigate plasticity loss in several other works, which we reference in Section 2.1, but it requires careful hyperparameter tuning.

Why is the stable rank based on the singular values but not the eigenvalues?

The stable rank is always defined with respect to the singular values because the magnitude of the singular values is the magnitude of curvature in the corresponding. The stable rank measures the number of singular values that account for a large fraction of the sum of singular values.

Thank you for your review, below we address your specific concerns:

It's unclear why early layers are seemingly disregarded.

The goal of this paper is to compute, exactly, the Hessian rank since the main claim of our paper is that the curvature, as measured by the Hessian rank, explains loss of plasticity. This computation has a O(d^3) cost where d is the number of parameters. It turns out that we do not need the exact hessian, because layer-wise approximations provide a good approximation of the Hessian for stationary tasks [1].

provide a layer-wise or partial block-wise Hessian rank

While we have argued that the layer-wise Hessian is a good approximation for stationary tasks, it is not a good approximation for non-stationary tasks like in our continual learning experiments. This is because the layer-wise Hessian does not change when the target function changes if we are using a piecewise-linear activation function like ReLU. Taking the last 2 layers is the simplest way to avoid this issue with the layerwise approximation, and our results demonstrate that it is sufficient to explain plasticity loss.

The choice of the Wasserstein distance measure as a regularizer adds complexity, especially when it comes to tuning for smaller sample sizes Why not employ the simpler L2 regularization from the initial parameter?

The Wasserstein distance is not any more complicated than the L2 or regen (which is L2 regularization from the initial parameter). Like both L2 and regen, it uses a single hyperparameter and does not depend on the sample size. In Appendix C.5, we show that the Wasserstein regularizer is actually more robust to the hyperparameter than both L2 and regen.

It remains ambiguous how the Wasserstein distance regularizer directly contributes to increasing the rank of the Hessian matrix.

See shared reply [Connecting Curvature and Regularization]

The omission of experiments using permuted CIFAR-10 seems to be an oversight

Thank you for bringing this up as it was not made explicit in the manuscript. Observation permuted CIFAR 10 is not a common benchmark for studying plasticity, because the CIFAR experiment uses a convolutional neural network. Observation permutations would destroy spatial regularity, and the translation invariance of convolutions would be an improper inductive bias.

Comparison with dropout, CReLU, layer norm, last layer reset

Thank you for your suggestions, please see the shared reply [Additional Experiments] and updated manuscript (Appendix D.2, D.3 and D.4)

[1] Sankar et al. A deeper look at the hessian eigenspectrum of deep neural networks and its applications to regularization, AAAI 2021

Thank you for your review, below we address your specific concerns:

a linear network is able to attain near-0 error on MNIST(?)

Our experiments use a subset of the MNIST dataset (similar to [1,2]), and a linear network can achieve a low error on this subset. The non-linear networks achieve a lower error than the linear network on the first task (see Appendix D.5), but due to loss of plasticity this error increases to be much higher than the error of the linear network. We emphasize that it is striking that loss of plasticity still occurs on problems for which linear features can be predictive.

correlations between feature, gradient, and Hessian srank, and can the authors provide evidence that the Hessian srank is providing a meaningful notion of curvature or trainability not captured by the other two quantities?

Please see shared reply and updated manuscript (Appendix A.1, D.3 and D.4). These results validate the fact that the number of curvature directions is important in explaining plasticity.

The paper doesn't evaluate across different architectures, and only considers extremely small MLPs. [...] The paper would benefit from evaluations in other architectures … Does it generalize to convolutional networks / ResNets / sequence models?

Please see the shared reply [Neural Network Architecture] and updated manuscript (Appendix D.2, D.4) We speculate that a resnet would mitigate some plasticity loss because its residual connections provide a deep linear subnetwork, but we leave this for future work.

reduction in srank might go against existing notions of curvature in the literature on deep neural network training, which often characterize the curvature by the maximal Hessian eigenvalue.

See shared reply, under [Curvature Definitions]

I think the paper would benefit from a closer examination of the difference between the Hessian rank and the feature rank in these networks.

The Hessian is a second-order property of the objective function, describing the local curvature and its rank counts the number of directions of curvature. In contrast, the feature representation is a zeroth-order property of the neural network. For example, if the labels are shuffled between tasks then the feature rank at the end of one task and at the beginning of the subsequent task would be the same. Thus, the feature rank cannot distinguish between some changes in the optimization landscape. There is no a-priori reason to believe that the feature rank is related to curvature or the Hessian rank, and indeed previous papers have also found that feature rank is not a consistent explanation for plasticity loss [2].

For example, weight decay and regenerative regularization both exhibit similar Hessian srank, but weight decay exhibits greater plasticity loss in Figure 4.

Although the Hessian srank decreases in the regularized experiment, the rank is still substantially larger than the unregularized setting, even compared to the linear baseline. We do not claim that weight decay exhibits plasticity loss in Figure 4, as the confidence interval is large we conclude that weight decay is unstable. See the new correlation plots for further evidence.

The choice to study the rank of the Hessian, as opposed to the rank of the empirical NTK or the features, is not given sufficient motivation.

The motivation is twofold. First, previously proposed explanations of plasticity have focused on zeroth (feature rank) and first (gradient/update norm) order properties. Second, there is a growing literature finding that curvature (which the Hessian describes locally) strongly influences learning dynamics. It is thus a natural hypothesis that the Hessian may help explain loss of plasticity.

While the proposed regularizer is reasonable, it feels very much orthogonal to the curvature of the network.

Please see shared reply, [Connecting Curvature and Regularization]

for example, the worse online learning results for the wasserstein regularization coupled with the better end-of-training results suggest that not only does the loss start out much higher, but that it stays higher than the other methods for a long time -- in other words, it is resulting in much slower learning.

We found that our proposed Wasserstein regularizer does not result in slower learning than the regen regularizer (See Appendix D.5 for inter-task learning curves). In general, the Wasserstein regularizer is smaller than the L2 and regen regularizer. Thus, there is a larger set of parameters within a set distance with respect to the Wasserstein distance.

[1] Kumar et al. Maintaining Plasticity via Regenerative Regularization, CoRR, abs/2308.11958v1, 2023 [2] Lyle et al. Understanding Plasticity in Neural Networks. ICML 2023

Thanks to the authors for their response, which has clarified a number of my questions. However, several concerns remain.

1. Success of identity activation on MNIST: can the authors specify the exact number of samples used and include this in the experiment description? I am skeptical of the significance of results on a task which can be solved by a linear model.
2. Network architecture: thanks to the authors for clarifying that a convolutional network is used in the CIFAR-10 experiments. However, in light of the above response I do need to express my concern over the significance of including different architectures given that the task can still be solved perfectly by a linear model.
3. Hessian vs feature rank: I am not convinced by the authors’ response that “there is no a-priori reason to believe that the feature rank is related to curvature or the Hessian rank, and indeed previous papers have also found that feature rank is not a consistent explanation for plasticity loss.” Given that the Hessian is the ‘derivative’ of the gradient, and that gradient geometry is closely connected with feature geometry, it is intuitive that the rank of the features would give a rough sense of the degeneracy of the Hessian, particularly if considered over the course of training on a few structurally similar tasks. Further, the similar trends exhibited by Hessian rank and feature rank suggest a connection between the two. I would be more convinced of the independence of these two quantities if they made different predictions about which networks are likely to lose plasticity or could be shown to have differing trends in different experimental settings.
4. Justification of title: I remain unconvinced that the paper’s titular claim, that “curvature explains plasticity” is supported by the experiments provided. The experimental results in the paper show that the Hessian rank does often correlate with plasticity, but so do other quantities which were shown by Lyle et al. to not fully explain plasticity. Further, there are examples in the paper itself which “falsify” Hessian rank as an explanation, for example Figure 3, where batch error and Hessian rank both increase after the 20th task.

Thank you for your review, below we address your specific concerns:

the authors should provide additional experiments … Moreover, it would be great if the authors could measure the consistency quantitatively.

Please see the shared reply [Additional Experiments] and updated manuscript (Appendix A.1, D.3 and D.4).

Forgetting

Our work focuses on the problem of plasticity, and is explicitly isolated from the problem of forgetting. Our experiments, for example, measure error with respect to the task being trained and not on any previous tasks. This is by design; we want to cultivate a better understanding of continual learning by first explaining the phenomenon of plasticity.

As tanh is experiencing small decrease on the rank of representation, the batch error also showed relatively small increase.

The feature rank is an inconsistent explanation because, for tanh, the increase in the batch error occurs and continues when the feature rank is constant. We have added correlation plots to show this more clearly, please see the shared reply [Correlation Plots] and the updated manuscript (Appendix A.1)

why is Wasserstein regularization better than regenerative regularization?

Please see shared reply, [Connecting Curvature and Regularization]