Dynamic Compression Strategies for Uniform Low-Dimensional Representations in Human Brain and Neural Network

Weak Points

1. The authors should justify this juxtaposition of results on EEG signals and CLIP embeddings. It makes more sense to use neural pixel recording data, for example, from the perspective of system identification.
2. The theory contribution of this paper is weak for the following reasons: 1) the result on interpolation probability is from past work and does not explain why “dynamic” compression is needed. 2) For theorem 6.1, what is actually proved is that if the data is concentrated enough, i.e. (Dx + Dy)^2 is small enough, then there is high probability that the generalization gap is small, which is obvious given the Lipschitz property of the loss function. The handwavy argument that Dx and Dy scale with the dimension should be justified.
3. While the authors claim that dynamic compression is beneficial for generalization, no empirical result on either artificial or biological networks is shown regarding the generalization performance. The authors should think of a unified framework to evaluate generalization performance on difference datasets.
4. The concept of dynamic compression is rather qualitative in the way that it is introduced in the paper. The result in the figure should be more thoroughly explored. For example, while MNIST and FashionMNIST has the same raw data distribution, but FashionMNIST’s embedding intrinsic dimension is higher (than the raw data even).

In its current form, the paper lacks in a few ways. The biggest problem, in my view, is the disconnect between the theoretical result and the empirical results. Indeed, the authors first measure the intrinsic dimension of representations in pre-trained networks and in the brain (based on EEG recordings). Using these measurements, they argue that neural networks employ a dynamic compression scheme which adapts the level of dimensionality reduction in latent representations based on the complexity of the data itself. They then present a theorem that prdicts generalization error bounds based on dimensionality of represenations. There are no empirical validation of this bound, and no clear link with the mesurements made, other than measuring dimensionality itself. It is then puzzling to see the link between the two contributions in this paper. The relation "neural networks compress data" + "compressed data helps generalization when doing interpolation" is circumstantial in this case. The authors do not directly test generalization and actually only measure dimensionality in one, pre-trained model. Furthermore, there are some questions about the dimensionality measurements and the comparisons presented. See questions below.

• To my eye, the empirical results of Figure 1 don't strongly (or at least, clearly) support the "uniform low dimensional representation" hypothesis. Firstly, for particularly simple datasets the ID of embeddings can actually be larger than that of the input data. Secondly, the ID does not seem to be uniform across different input datasets and indeed seems to grow at a similar rate to input ID for many of the considered datasets (i.e. in the rightmost four columns). Is the argument in fact that ID is reduced at a uniform rate across different input datasets?

• Only one network (CLIP) is considered, and it is not necessarily obvious that similar observations would be seen in i.e. different architectures or for networks trained using different objectives.

• Section 4.2 is lacking sufficient detail to evaluate the claims:

  • Are the neural signal rows in Table 1 related to the measurements analyzed in Figure 2? Do these represent dimensionalities after aggregating across conditions?
  • Are the natural and synthetic audio signals related to the task conditions referenced in Figure 2? Furthermore, what task are participants performing?

• Is the takeaway from Table 1 supposed to be that EEG measurements are lower dimensional than natural or synthetic signals? Even if this is the case, this does not constitute sufficient evidence of the claim that the brain prefers "uniform low dimensional representations." For one the EEG signal is at best an incomplete measurement of the brain's representation of these signals. For example, how would the dimensionality change if the number of EEG channels was increased, what if we could perfectly record all of the neural activity of each unit in auditory cortex?

1. The experimental setup and corresponding conclusions regarding artificial NNs are questionable.

   Firstly, there are other equally feasible explanations of the phenomenon observed in Figure 1, which should be explored (and negated):

   • As the CLIP model is pretrained on a separate dataset, there may be a fixed set of features which this NN is trained to extract. Thus, when the NN is fed with simple datasets, the latent dimensions corresponding to the missing features become degenerate or even hallucinatory (which might explain the unexpected increase in the dimensionality when embedding "simple" datasets).

     One can even imagine a situation, in which the embedder network is pretrained on a "simple" dataset, thus yielding degenerate embeddings for complex datasets and reversing the trend from Figure 1 (i.e., the embeddings corresponding to complex datasets would be of lower intrinsic dimensionality compared to the "simple" counterparts).

     To negate this hypothesis, one should consider using different models pretrained on different datasets with varying complexity (or even on the combined dataset).

   • Networks pretrained on one dataset might compress other datasets suboptimally.

     To negate this hypothesis, one should consider training CLIP (or other self-supervised, unsupervised or even supervised models) on each dataset separately.

   • The intrinsic dimensionality estimator might become unreliable when the input data reaches certain level of complexity, yielding higher estimates for unstructured data given the same ground truth intrinsic dimensionality.

     This can be negated through providing elaborate statistical analysis and confidence intervals.

   Secondly, as the reliability of the intrinsic dimensionality estimator can be questioned, the work would benefit greatly from synthetic tests with tractable intrinsic dimensionality (can be achieved via using synthetic manifolds).

   Finally, additional details regarding the CLIP model (e.g., the dataset it was pretrained on) should be provided.

2. Using single slices from EEG may lead to loosing temporal structure of the underlying data with the corresponding additional level of complexity. Other methods of sampling (using several slices, embedding the whole sequence via a NN, etc.) should be considered.

3. Downscaling the images in Section 4.1 can also severely affect the intrinsic dimensionality. Additional results for uncompressed images would certainly improve this section.

4. The work would benefit greatly from improving the structure. For example, the current division into subsections is too verbose, with Section 3.3 being dedicated to a single definition. Please, consider merging sections and/or using the \paragraph{Paragraph name} command. The definition of a convex hull is provided twice: Definition 1 and line 177. The connection between Theorems 5.1-5.2 and 6.1 remains somewhat vague and requires an additional discussion. For example, a lemma can be provided that formally connects the results in question.

5. The choice of $C = C_L \sqrt{d}$ and $D_x = C_x \sqrt{d}$ in Theorem 6 seems arbitrary.