Why do Features of Multi-Layer Perceptrons Condense in Training?

I am not sure I buy the contribution of this paper. In particular---

1. Could methods in this paper be extended to more common architectures, such as CNN, Transformer, etc in any way?
2. Does the problem of feature condensation still exist in other architectures like Transformers or even MLP with a residual connection?

I would be happy to change my score if the authors convince me either (1) that there are practical applications for deep MLP, or (2) that the theoretical framework developed in this paper could be extended to modern architectures. Otherwise, I am afraid not enough people in the community will be interested in this paper.

The novelty of the question explored in this work is not obvious. Figure 1c provides a typical example illustrating what is referred to as 'feature condensation.' This figure showcases the training profile of deep networks, a problem that has been extensively studied under the umbrella of what is commonly known as the 'vanishing gradient problem.' In this scenario, it often takes several epochs before the synaptic weights begin to change significantly. In fact, some of the techniques listed as potential ways to mitigate feature condensation are already well-documented as methods for addressing the vanishing gradient issue or expediting the update process. However, this paper does not offer a comprehensive explanation of the differences or similarities between feature condensation and the vanishing gradient problem.

The concept of feature condensation may not be immediately clear, as the paper lacks a precise definition of the term on its initial pages. Furthermore, this paper mentions a resemblance between its findings and the work by Williams et al. (2019), which studied the 'neural collapse' phenomenon. It remains unclear whether there are significant distinctions between the terms 'neural collapse' and 'feature condensation

Major Issues:

1. Section 2 begins with "It has been widely observed that the loss decrease of DNNs is likely to have two phases". However, in training SOTA models, the loss starts decreasing almost immediately after the first step, and the authors themselves also noted in appendix that training standard ResNets and ViTs do not suffer from the issue that the loss gets stuck in the beginning, and the feature condensation phenomenon does not appear either. In my understanding, the setting being studied in this paper deviates significantly from the standard practice (no normalization, Xavier init instead of He init, etc.), which could be the main reason that feature condensation can be reliably observed in the paper. I would recommend the authors add more discussion and introduce more background at the beginning of Section 2, instead of just claiming that loss getting stuck is "widely observed".
2. The theory part is not rigorous enough. I believe if the authors make more efforts to make the analyses more formal, they will find many missing gaps in theory. For example, why does weight condensation imply feature condensation? (different input + same weight can lead to different features!) The self-enhanced system of weight condensation makes sense (existence of a dominating common direction -> the common direction becomes even more dominating), but how does the dominating common direction appear from random initialization? How do the analyses ensure that the noise term at each iteration won't be accumulated to be something bigger than the weights in the common direction? I can sort of understand the intuitions provided in this paper for sure, but since the paper claims that it is providing a theoretical analysis of feature condensation, it is better to do it more formally and provide an end-to-end theorem that rigorously proves feature condensation based on assumptions. It would also be better to instantiate the analysis rigorously on some simple and solvable examples.
3. Again, the theoretical analyses of the common tricks have the same issue: many insights are stated without any quantitative analyses. These analyses are thus not strong enough to be called theoretical analyses.

Minor Issues:

1. In the introduction, the summary of Williams et al. (2019); Lyu et al. (2021); Boursier et al. (2022); Wang & Ma (2023) is not very fair: some of these papers are only assuming the data is linearly separable. "the feature direction of each arbitrary training sample could act as a perfect linear classifier" seems to be an even stronger assumption. Does it refer to orthogonally separable data?
2. It is discussed in Section 3.3 how common tricks affect feature condensation, but what is missing in this part of the paper is a sanity check to see whether less feature condensation is indeed beneficial to generalization. At least for the effect of the initialization scale, it is not very clear whether reducing feature condensation by increasing the initialization scale can lead to better generalization --- larger initialization means the neural net will stick to random features, which may not be of good quality.