How Does Cross-Layer Correlation in Deep Neural Networks Influence Generalization and Adversarial Robustness?

The statement of the theoretical results:

1. We will remove the word "domain" at the end of the sentence and rephrase it for clarity.

2. In Equation (3), for the notation of $W$, only the vectors are bolded. The matrices and scalars are not. I will make this clearer and add a notation table in a later version.

3. In line 226, as shown in Theorem 4.1, $\text{vec}(W_l) \in \mathbb{R}^{N_l N_{l-1}}$ denotes the result of vectorizing the matrix $W_l$, which is a real-valued vector with dimension $N_l N_{l-1}$. Here, $N_l$ and $N_{l-1}$ represent the width and length of the matrix $W_l$, respectively. We will clarify this further in a later version.

4. The subscript for the covariance is due to the fact that the vectorized weights, i.e., $\mathbf{\omega}_l, \mathbf{\omega}_s$, come from different measures \pi\ and $\rho$. Hence, we use the subscripts to denote the differences.

5. There is a typo in line 239. It should be "variance" instead of "covariance." We will correct it.

6. The repeated words will be corrected in a later version.

---

The reason for studying the lower bound of KL divergence:

1. Equation (11) indeed provides an upper bound for the generalization gap. However, this upper bound is causally correlated with the generalization ability, as indicated in lines 82-83 (Jiang et al., 2019). Hence, an increase in this upper bound also suggests a larger generalization gap. In Equation (13), the lower bound of KL divergence indicates that the upper bound, considering the cross-layer correlation in Equation (11), is at least this much larger than the one without it. Hence, we conclude that cross-layer correlation will increase the generalization gap.

2. I do not think your statement about Equation (23) is true, since there is a negative sign in the natural risk term, and the probability is non-negative. The only case where the adversarial risk increases monotonically with the natural risk is when the probability in the second term of Equation (23) is zero. Please check twice on Equation 23.

---

Responses to Specific Questions:

Q1. I have provided the complete proof of all the theorems in the appendix. The difference lies in the $2\gamma$, $\gamma$ term, and the logarithmic term in Equation (11). The proof requires the existence of $2\gamma$. The original expression is shown in Lemma 1 from the work of Neyshabur et al., where $R_0(\theta)$ and $\widehat{R}_{\gamma}(\theta)$ are considered.

Q2. As indicated by Theorem 4.2, the KL divergence is monotonically increasing with respect to the square of the cross-layer correlation. Hence, the smallest KL divergence corresponds to zero cross-layer correlation, indicating the smallest generalization gap.

Q3. As shown in lines 57-66, 93-94, and 122-124, we consider the correlation across layers and its impact on both generalization and adversarial robustness. Those are not included in the Jin's work.

Thank you so much for your time and your appreciation of our work. Our feedback is as follows:

Q1. In Table 1, the correlation across layers is not the only factor influencing generalization. It is infeasible to exclude all other factors, such as the weight correlation within layers and optimization dynamics. We can only verify the trend predicted by our theorem. As shown, except for the two misalignments, the other results (e.g., the maximal and minimal values for the 8-layer MLP) still support our theorem.

Q2. Yes, the results hold for larger neural networks. As stated in Theorem 4.1, we did not restrict the size of the neural networks. As described in Equation (13), when we increase the width of the neural network, the change in the bound depends on both the first and second terms. If the first term is fixed, the bound still depends on the assumption of $K_{l-1,l}$. However, this is an interesting topic, and we will include a discussion on it in a later version.

Q3. The "phase transition" in Figure 2 denotes the different behaviors of training in different layers. It suggests that only in the later layers is there a high value of cross-layer correlation. I will make this much clearer in a later version.

Q4. Thank you very much for your suggestion. Since we only consider linear correlation between adjacent layers, the correlation of higher-order terms and the correlation for non-adjacent layers are still unclear. We will add a discussion and note the limitations in a later version.

Thank you very much for your time and detailed review. Our feedback is as follows:

Theoretical Analysis of the Triviality of the Work

We do not agree that our work is trivial or merely a special case of the work by Jin et al. Jin’s work focuses on assessing the impact of neuron correlation in neural networks on generalization power. In the case of MLP, it only considers the linear correlation of rows in the weight matrix. In contrast, our work considers the correlation of the weight matrix across different layers. The mathematical analysis for our work is quite different, comparing to his work. Additionally, we also investigate the impact of this correlation on adversarial robustness and present both generalization and adversarial robustness within a unified framework. If you have checked the appendix for both papers and compared them in detail, you will find the differences is large.

Empirical Analysis

In Table 1, the discrepancy between the lowest correlation and the lowest robust gap for the 2- and 4-layer MLPs is due to the fact that correlation across layers is not the only factor influencing generalization. It is infeasible to exclude all other factors, such as weight correlation within layers. We can only verify our theorem by the overall trend predicted. Despite the two misalignments, the other results (e.g., the maximal and minimal values for the 8-layer MLP) still support our theorem.

In Figure 2, we only use the TRADE method because it can control the intensity of adversarial training. The selection of $\beta$ is a heuristic, as suggested by the original paper. Different adversarial training method can be used; however, our work is more focused on the theoretical analysis, which is why we include less empirical work to verify our ideas.

"Phase Transition'' in Figure 2

The "phase transition" in Figure 2 represents the different behaviors of training for layers. It suggests that only the later layers exhibit cross-layer correlation. I will clarify this further in a later version.

Responses to Specific Comments

Q1: I have checked the reference for Fawzi et al., 2018, which is incorrectly grouped with Tsipras et al. and Zhang et al. in line 43. This was an editorial error. The references in lines 99 and 105 present different perspectives of the same paper. I will correct this and clarify the reference in a later version.

Q2: The phrase "fundamental to the nature" is meant to convey that this trade-off is universal for all neural networks under the problem we set. We will rephrase it to clearer wording.

Q3: In line 100, the phrase "robust in tasks involving small distinguishability" corresponds to a sentence in the abstract of the original paper: "In both cases, our upper bound depends on a distinguishability measure that captures the notion of difficulty of the classification task." This refers to how difficult it is for the classifier to distinguish between different classes based on the data distribution. We will clarify this in a later version.

Q4: In line 169, yes, the phenomenon was first noted by Szegedy et al. in 2014. We will correct this.

Q5: In line 214, when the value of $\gamma$ is not too prominent, it can influence generalization through optimization. The proper phrasing should be: ``As the value of $\gamma$ is given.'' We will rephrase this in a later version.

Q6: In Equation (12), yes, $\left| l - s \right| < 1$ implies $l = s$. We will clarify this in a later version.

Q7: In line 448, we will correct the issue in a later version.

Q8: The average of all cross-layer correlations is the sample mean of all correlations between adjacent layers. Our analysis is based on correlations between adjacent layers, as shown in Equation (12). The cross-layer correlation between non-adjacent layers (e.g., layer 1 and layer 3) is not included, and such inclusion would not align with our theorem. We will include a more detailed explanation of the average of the cross-layer correlation in a later version.