Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression

We sincerely thank Reviewer Lyy3 for the work! We are more than glad to receive helpful critique of our article. We further provide answers to the problems and questions raised in the review. The references in our response correspond to the reference list in our manuscript.

Weaknesses:

1. Indeed, Table 1 from Geiger (2020) shows that compression phase was observed in some (not all) experiments. For clarification, it is therefore better to replace “has only been verified” with “was only partially verified”. This will be done in the next revision.
2. We agree with the reviewer: the mentioned paper contains the analysis of quantized and stochastic NN's. We will add the citation in the next revision.
3. We kindly ask for clarification of the question, as in this particular part of our manuscript, we refer to the compression of data via an autoencoder. This should not be confused with the so-called “compression stage” in the “compression-fitting hypothesis”. We will change this part to “an arbitrary amount of information can be lost through compression of the data” in the next revision to avoid any potential confusion.

We again thank Reviewer Lyy3 for the constructive review! We hope that our answers helped to better understand the manuscript. Please feel free to let us know if there is anything specific we can provide or clarify. We are more than willing to answer any further questions and resolve any additional concerns. Looking forward to your response!

We thank Reviewer d5Zt for the work! We are glad to receive helpful critique of our article. We further provide answers to the main points raised in the review. The references in our response correspond to the reference list in our manuscript.

Weaknesses:

1. Although the entropy estimation is an example of a classical problem, it is still problematic to acquire the estimates for high-dimensional data, as the estimation requires an exponentially (in dimension) large number of samples (see Goldfeld et al. (2020); McAllester & Stratos (2020); $\textnormal{estimation error} \in O(\textnormal{n.samples}^{-1/\textnormal{dim}})$). The proposed approach makes the existing MI estimators less vulnerable to the curse of dimensionality by incorporating an explicit compression step. While other complex parametric MI estimators (like MINE (Belghazi et al., 2018)) may be assumed to compress data implicitly (as it is usually possible to extract some information about the PDF $\rho_{X,Y}$ from the critic network), we argue that explicit compression allows for useful theoretical results and better estimation accuracy.

   We also note that, unlike MI, entropy can be alternated via compression. That is why a theoretical analysis had to be conducted to justify the proposed approach.

2. Information bottleneck analysis of bigger or other non-classification problems (regression, generative or language modeling) is one of the possible directions for the future work on this topic. The current paper is devoted to the validation of the proposed method via a theoretical analysis, tests on synthetic data and application to some non-synthetic high-dimensional examples. Although the used problem is toy, it still poses difficulties in the task of MI estimation. That is why we believe our results to be a step forward.

3. The exact choice of $f_i$ and $g_i$ is not very important. In the main text, we only require $f_i$ and $g_i$ to satisfy the conditions of Statement 1 (see Algorithm 1; we will also add this requirement to Figure 2 in the next revision) to properly define the experimental setup. Defining $f_i$ and $g_i$ used in the tests requires a lot of space, as these functions smoothly map low-dimensional vectors to high-dimensional images and, thus, are very complex. A Python implementation of the functions in question is available in the supplementary material. We will consider adding the corresponding listings in the Appendix in the next revision.

Questions:

1. If $n' \neq n$, it is impossible to define $f$ as a smooth bijective mapping between $\mathbb{R}^{n'}$ and $\mathbb{R}^n$. However, as $f$ is injective, it is a bijection between $\mathbb{R}^{n'}$ and $f(\mathbb{R}^{n'}) \subseteq \mathbb{R}^n$. So $f$ can be considered as a smooth bijection between $\mathbb{R}^{n'}$ and a corresponding manifold in $\mathbb{R}^n$.

Finally, we thank Reviewer d5Zt for the valuable feedback. We are more than willing to address any further questions and will make every effort to resolve any additional concerns. Please feel free to let us know if there is anything specific we can provide or clarify. We hope that our answers helped to better understand the idea and contribution of our work. Looking forward to your response!

Dear Reviewer d5Zt,

Once again thank you very much for your detailed review and the time you spent. As we are nearing the end of the discussion period, we would like to ask if the questions you raised have been addressed. We hope you find our responses useful and would love to engage with you further if there are any remaining points.

We understand that the discussion period is short, and we sincerely appreciate your time and help!

Questions:

• In Figure 3 we attribute such behavior to the floating point error, as $I(\xi;\eta) = 10$ in case of $n' = m' = 2$ leads to correlation coefficient $\sqrt{1 - e^{-10}} \approx 1 - 2 \cdot 10^{-5}$, yielding an ill-posed covariance matrix, which is then inverted on the first step of dataset generation.

  In case of $n' = m' = 4$ (Figure 4) the correlation coefficient is $\sqrt{1 - e^{-10/2}} \approx 1 - 3 \cdot 10^{-3}$, and WKL yields better results. However, due to a doubled dimension, KDE and KL estimators exhibit an increased bias, as these methods are more sensitive to high dimensionality.

• MINE is fed with uncompressed data. The choice of colors for this particular case is indeed misleading. Please refer to the text (“We also conduct experiments with MINE (without compression)”, Section 5, paragraph 3). We will change the color in the next revision. As MINE is fed with uncompressed data, the critic network should be at least as complex as the autoencoder to allow for fair comparison.

• This hypothesis has not been tested. However, such good agreement may indeed indicate (almost) bijectivity of $E_X \circ f_2$ and $E_Y \circ g_2$. We will consider investigating this hypothesis in our future work.

• KL estimator is proved to be biased for dimensions $\ge 4$, see Berrett et al. (2019).

• Noise is present both during the training and the MI estimation. We kindly ask for clarification regarding potential geometric affects. Is this question related to the manifold hypothesis?

• Confidence intervals are computed via a standard deviation derived from the entropy estimator. Around $35$ networks were trained in total. For the final plot, only one network has been used.

• We kindly ask for clarification of the question: all the networks used to acquire Figure 6 are autoencoders, which were trained before the MI evaluation on synthetic examples. Perhaps, the question is about Figure 5. In such case, the variation actually stayed almost the same throughout the training; we believe an increased density of points near the end of training might give the impression of an increased variance.

• This happens because of the NN's loss reaching a plateau at the end of the training.

• Yes, we refer to it as “loss function delta per epoch” at the end of page 9. We will define it more clearly in the next revision to avoid confusion.

• As we plot estimates of MI, violation of DPI ranging within CIs is expected. If this violation is systematic, it may be attributed to more refined latent representations at the final layers of the network, which probably allow for a better MI estimation.

Finally, we thank Reviewer MJhB again for the detailed constructive review. We are more than willing to address any further questions and will make every effort to resolve any additional concerns. Please feel free to let us know if there is anything specific we can provide or clarify. Looking for your response!

Dear Reviewer MJhB, thank you sincerely for your careful and constructive review. We are glad to receive helpful critique. In the following text, we provide responses to your questions. We hope that all the concerns are properly addressed. The references in our response correspond to the reference list in our manuscript.

Weaknesses:

• It is, of course, true that a MI estimate obtained from the original data generally is not equal to an estimate obtained from the compressed representations; otherwise there is no point in compressing the data (except, probably, for computational complexity). However, conventional estimators completely fail to correctly estimate MI in high-dimensional cases due to the exponential sample complexity (see Goldfeld et al. (2020); McAllester & Stratos (2020); $\textnormal{estimation error} \in O(\textnormal{n.samples}^{-1/\textnormal{dim}})$). It might be possible that equation (4) has been misunderstood: we do not claim that estimates for the original and compressed data are equal. Instead, we define MI estimate for high-dimensional compressible data through a conventional (e.g., via KDE) MI estimate for compressed representation. We will definitely improve this part of our work in the next revision to avoid such critical misunderstanding.

  Our main theoretical and practical contribution is showing that the proposed MI estimator (defined by eq. (4)) is valid: we (a) show that MI is not lost via lossless compression, (b) bound MI loss in case of lossy compression, (c) provide experimental results with synthetic data showing good performance of the proposed method. One can be assured that the proposed estimation method represents the “true” behavior of MI via combining our results with a convergence analysis of the third-party estimator, which the compressed representations are fed into. In most of the cases, it can be done via the triangle inequality: $|I(X;Y) - \hat I(E(X);Y)| \leq |I(X;Y) - I(E(X);Y)| + |I(E(X);Y) - \hat I(E(X);Y)|$, where the first term is analyzed in our article, and the second term should be acquired from a convergence analysis of a particular MI estimator.

• It is a valid point that the contribution of our work would be more clear if we provide estimates computed directly from the data. However, as conventional entropy estimators exhibit bias and variance of such a high magnitude in case of high-dimensional data, it is impossible to plot results for the compressed and uncompressed data side by side without a complete loss of readability.

  In particular, weighted Kozachenko-Leonenko (WKL) yields estimates ranging from about $-3000$ to $3000$ nats with a variance around $2000$ nats in the test with $8 \times 8$ images of rectangles and the true value of MI varying from $0$ to $10$ nats. Other estimators yield the same nonsensical results for this $64$-dimensional example. Thus, the error is of several orders of magnitude. In contrary, our method works fine even for the $32 \times 32 = 1024$-dimensional data of the same kind.

  One can reproduce these results via running the provided source code. We will add experimental proofs of the inability of the conventional methods to estimate MI of high-dimensional random vectors in the next revision.

We hope that provided clarifications will help to better understand the manuscript and reevaluate the contribution of our work. We will add all the required evidence in the next revision to stress out that obtaining MI estimates via conventional estimators without compression is practically impossible. We will also correct all the mentioned minor weaknesses in the next revision.

Dear Reviewer MJhB, thank you a lot for the reply.

Main Claim

It is our fault that we plot the CI obtained from entropy estimators only. That is why they represent only the error of a third-party estimator ran on compressed data, but not the information lost due to the data compression. We should change our experimental protocol to account for that. We must also change our claim from “the proposed estimation method represents the “true” behavior of MI...” to “the proposed estimation method represents the “true” behavior of MI up to the information lost due to data compression, which can be bounded via Statement 3 and Corollary 6...”. This claim should better represent the fact that our method is not ideal, and the estimation error can still be large in case of hard-to-compress data. This seems to happen in the case of $L_2$, as outputs of $L_2$ have dimension $400$ and are linearly compressed to dimension $4$; the reconstruction error is also relatively high compared to the consequent layers. We will investigate it further in the next revision.

However, we still would like to stress the following:

1. Previous theoretical results (see Goldfeld et al. (2020); McAllester & Stratos (2020)) and our experimental results (Section E.3 in the new revision) show that estimation without compression is a much harder task, which is expected to result in estimates of a much lower quality.
2. If the compression is performed well, the estimate is close to the real value. It is demonstrated both via the experiments with synthetic data and the acquired bounds on the information lost due to the compression. Thanks to Statements 7,8,9 and Corollary 4, the information-theoretic quality of the compression can (under certain assumptions) be measured via the reconstruction error. We additionally note that the proposed bounds are tight, as they are derived from the data processing inequality.
3. In the work of Pool et al. (2019) it has been shown that other complex parametric NN-based estimators (NJW, JS, InfoNCE, etc.) exhibit poor performance during the estimation of MI between a pair of $20$-dimensional incompressible (i.e., not lying along a manifold) synthetic vectors. These vectors, however, are of much simpler structure than the synthetic datasets used in our work (Gaussian vectors and $x \mapsto x^3$ mapping applied to Gaussian vectors in (Pool et al., 2019) versus high-dimensional images of geometric shapes and functions in our work). We interpret these results as follows: if it is impossible to compress high-dimensional data to a lower dimension, not only our method fails, but the other modern parametric NN-based methods fail too.

We hope that this information will (a) help to better understand the benefits of the explicit compression, (b) show that other modern and acknowledged methods exhibit the same limitations in case of incompressible data as our method do. Overall, judging by the tests performed in our work (Sections 5, C, E.3) and by the comprehensive overviews presented in (Poole et al., 2019; McAllester & Stratos, 2020), we think that our method is not flawless, but performs better than the existing MI estimators.

Geometric effects

Yes, these geometric effects are also mentioned in the work of Goldfeld et al. (2019). We do not ignore them, as we remember that the compression phase in the “fitting-compression hypothesis” can be caused by such clustering of the latent representations. However, keeping it in mind, no additional modifications to the overall experimental setup are required, as autoencoders should account for any peculiarities of the data while learning to reconstruct it. Moreover, in any case, (a) noise is widely used in modern networks for regularization purposes, (b) stochasticity still has to be introduced into non-quantized networks, otherwise it will be impossible to conduct information-theoretic analysis.

We are sincerely thankful for the valuable feedback. We would be glad to continue the conversation if there are any questions left.

We are very grateful for the raised score! We again want to say thanks for the very careful and attentive reviewing of our paper, especially the critique of our graphics!

We thank Reviewer Fd1K for reading the article and providing us with a profound review! We further provide answers to the main points raised by the Reviewer. The references in our response correspond to the reference list in our manuscript.

Weaknesses:

• To compress synthetic data (Figures 3-4) and input data in experiments with DNN classifier, we use CNN autoencoders (AEs) trained to minimize MAE (yields less blurry results compared to MSE). We also use PCA — linear AE with MSE loss (allows for analytic expression of optimal compression mapping) to compress outputs of layers in DNN classifier experiment. We will add the requested details and more information on used networks to the Appendix in the next revision.

  Statement 3 shows (under corresponding assumptions) the connection between loss of MI under compression and AE manifold learning capabilities (this connection is further investigated in Corollary 7 via deriving explicit bounds on loss of MI for PCA). Thus, the performance of the method is as sensitive to the AEs architecture as the reconstruction error is. Because reconstruction error is easy to track, AE architecture can be relatively easily validated or adjusted to match the needs of the task. We believe this makes our method more robust compared to other NN-based MI estimation approaches, where compression is assumed to be performed implicitly and thus cannot be validated (like MINE).

  We also provide several results regarding much simpler architecture of AE — linear (PCA), see Appendix, Figure 6. As this figure shows, even PCA is capable of achieving almost the same quality as nonlinear AEs in the tests used in Section 5. One can consider it to be a somewhat practical evidence of low sensitivity to AE architecture. However, it is also possible to construct an adversarial example when the AE with insufficiently complex architecture fails (see Figure 6 (c)); such cases correspond to large reconstruction errors, which is a reliable signal to consider other AE architectures.

• We explicitly mention stochasticity in Section 6: "To avoid the problem of a deterministic relationship between input and output, a small additive Gaussian noise is injected at each layer (we use a constant signal-to-noise ratio)". This setup is similar to Adilova et al. (2023), except we do not pass gradients through the noise variance. These details will also be emphasized in the Appendix in the next revision.

• This is a valid point, and we are planning to perform more experiments with other neural networks and real datasets. However, we consider the proposed compression step and corresponding theoretical (bounds on MI under compression) and practical (tests on synthetic datasets with known MI) results to be the main contribution of our work. We use the experiment with the DNN classifier as a sanity check, trying to reproduce the compression-fitting pattern on the information plane.

We again thank the Reviewer for pointing out weaknesses of our work. We will properly address them in the next revision.

Questions:

• In the introduction (page 2) we mention the work of Fefferman et al. (2013) [1]. This work provides a more formal and profound analysis of the manifold hypothesis (MH). We will add the citation to Section 3.1 as well.

  One could think of photogrammetry (3D object reconstruction based on series of photos) as a good example of precise (up to imperfections in the setup) satisfaction of MH, as all the photos can be parametrized by a relative position and orientation of the camera. In particular, one can consider ENRICH dataset [2]. We also note NNs trained on such datasets, see [3].

• Yes, stochasticity affects training of NN. Injecting random noise is widely considered as a way to increase model performance and generalization capabilities. For example, dropout layer [4] is widely used to regularize NNs. We also mention Gaussian dropout, which is known to benefit generalization even more than binary dropout [5]. Previous works on information bottleneck analysis also argue that using stochastic NNs does not have negative impact on training (see Goldfeld et al. (2019), Adilova et al. (2023)). In previous and in our work, stochastic NNs are used as a proxy to analyze deterministic NNs.

The reasoning above will be incorporated into the article in the next revision to make it more accessible. We hope that we were able to provide sufficient answers to the mentioned concerns. We again thank Reviewer Fd1K for the helpful critique. We will be more than glad to continue the conversation if there are any questions left.

Additional references list:

[1] Fefferman et al. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29, 10 2013. doi: 10.1090/jams/852. [2] Marelli et al. ENRICH: Multi-purposE dataset for beNchmaRking In Computer vision and pHotogrammetry. ISPRS Journal of Photogrammetry and Remote Sensing, v. 198, pp 84-98, 2023, doi: 10.1016/j.isprsjprs.2023.03.002 [3] Mildenhall et al. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science, vol 12346. Springer, Cham. doi: 10.1007/978-3-030-58452-8_24 [4] Geoffrey et al. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580 [5] Srivastava et al. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929–1958, 2014.

Dear Reviewer Fd1K,

Once again thank you very much for your detailed review and the time you spent. As we are nearing the end of the discussion period, we would like to ask if the questions you raised have been addressed. We hope you find our responses useful and would love to engage with you further if there are any remaining points.

We understand that the discussion period is short, and we sincerely appreciate your time and help!