Beyond the Lazy versus Rich Dichotomy: Geometry Insights in Feature Learning from Task-Relevant Manifold Untangling

We thank Reviewer D4ag for your time and effort for providing detailed suggestions and questions to improve our manuscript.

To recap, in this paper, we propose using manifold capacity and its geometric measures to investigate the lazy vs. rich dichotomy in feature learning. We appreciate that the reviewers found our approach using manifold capacity to measure feature learning to be "novel application" (reviewer nCob, cJiT), have "better alignment with the degree of feature learning compared to other metrics" (reviewer nCob), with "robust experiments substantiate these findings" (reviewer D4ag). Reviewer v9qp, and D4ag also noted that our findings in learning stages and OOD generalization being insightful.

Before addressing each of your points in detail , we would like to draw your attention to our general responses, which provide context and address several common concerns raised across the reviews.

• Q1: The structure needs refinement, as it currently feels like the authors have packed too much content into the main paper, resulting in diminished clarity.
  • A1: We thank the reviewer for pointing out the structural issue of our manuscript. We completely agree with the suggestion and we’ve spent significant amount of effort in reorganizing the structure of the paper. In particular, we’ve increased the coverage on the definition and background information in Section 2 (on the introduction of manifold capacity and geometry) and Section 3.1 (the theoretical results). We invite the reviewer to skim through the changes we made according to your suggestions in the revised version. We look forward to hearing Reviewer D4ag’s thoughts and further advice on the revised version.
• Q2: The algorithm is not clearly presented, which also appears to be a side effect of the structural issue noted above.
  • A2: We thank the reviewer for the very good suggestion. We’ve added the results of the mean-field approximation for manifold capacity in Equation 1 (line 187), which can be empirically computed. We also added the pseudocode of estimating manifold capacity and its effective geometric measures in page 22. We look forward to the reviewer’s thoughts on the revised version and welcome any further suggestions and comments.
• Q3: Could the authors provide an example of the lazy regime? Specifically, how can a neural network be trained without actually modifying its internal features?
  • A3: We provide an example of lazy regime presented in Fig. 2b where the blue curve in the capacity panel doesn’t change at all across training while the text accuracy increases. We remark that this is the same setting used in [Chizat et al., NeurIPS 2019] and they have found that both the network weights and NTK are not changing during training. Namely, the network mainly learns via adjusting its final readout weights (and the features are hence the so called random features).

We thank the reviewer for the thoughtful suggestions, which have significantly improved the paper’s clarity and presentation. We also appreciate the reviewer’s time and reevaluation of our revised work. Thank you again for your thoughtful feedback and time!

References:

• [Chizat et al., NeurIPS 2019] Chizat, Lenaic, Edouard Oyallon, and Francis Bach. "On lazy training in differentiable programming." Advances in neural information processing systems 32 (2019).

We thank Reviewer K6x2 for your time and effort for providing detailed suggestions and questions to improve our manuscript.

To recap, in this paper, we propose using manifold capacity and its geometric measures to investigate the lazy vs. rich dichotomy in feature learning. We appreciate that the reviewers found our approach using manifold capacity to measure feature learning to be "novel application" (reviewer nCob, cJiT), have "better alignment with the degree of feature learning compared to other metrics" (reviewer nCob), with "robust experiments substantiate these findings" (reviewer D4ag). Reviewer v9qp, and D4ag also noted that our findings in learning stages and OOD generalization being insightful.

Before addressing each of your points in detail , we would like to draw your attention to our general responses, which provide context and address several common concerns raised across the reviews.

• Q1: The difference between previous work and their problem statement is not well delineated. Their goals aren’t succinctly mentioned.
  • A1: The goal of this work is to provide a representation-based method to quantify task-relevant changes in neural representations, offering a novel approach to investigating the lazy vs. rich feature learning question. To the best of our knowledge, no previous work has proposed a representational-based method that can quantify such task-relevant changes in neural representations. For instance, the most common method involves tracking weight changes in a network. However, a network might change its weights while degrading its representations. Another common method, which is to track the change in the norm of the neural tangent kernel (NTK), requires the knowledge of the entire model weights to compute the NTK.
  • About the distinction of our work from Chung et al 2018, while this previous work propose the use of mean-field approximation to compute manifold capacity (cited at Equation 1, L187 in our revised main text), our work introduces theoretical work to characterize manifold capacity after 1-step gradient update in 2-layer neural network, which is to our knowledge, a novel theoretical result. We look forward to the reviewer’s thoughts on the revised version and welcome any further suggestions.
• Q2: The main theorem statement is nearly vacuous in the main paper. "In 2-layer neural networks trained with gradient descent in the rich regime, the changes in capacity track the underlying degree of richness in feature learning." Since richness itself is defined as capacity, this is either circular or vacuous.
  • A2a: We apologize for the confusion in the initial version! In the revised version, we provide a more detailed statement of the theorem in lines 301–317, clarifying how the setting relates to rich learning. Specifically, richness is defined by how well the neural representations align with a hidden signal direction (in a teacher-student setting) [Ba et al., NeurIPS 2022]. In this setting, the alignment to signal is proportional to the learning rate, hence the learning rate can be considered the ground-truth for the degree of richness [Ba et al., NeurIPS 2022] (see lines 291–295).
  • A2b: The main result of Theorem 1 is to establish the connection between capacity and learning rate, which is not circular and requires substantial technical development (lines 315–317). We welcome the reviewer’s further thoughts on the revised version.
  • A2c: We interpret these results as a justification for using capacity to track the degree of richness in feature learning in a well-studied theoretical setting. We remark that our proof requires substantial technical improvements from previous work [Ba et al., NeurIPS 2022] due to the fundamental difference between regression and classification.

We thank Reviewer v9qp for your time and effort for providing detailed suggestions and questions to improve our manuscript.

To recap, in this paper, we propose using manifold capacity and its geometric measures to investigate the lazy vs. rich dichotomy in feature learning. We appreciate that the reviewers found our approach using manifold capacity to measure feature learning to be "novel application" (reviewer nCob, cJiT), have "better alignment with the degree of feature learning compared to other metrics" (reviewer nCob), with "robust experiments substantiate these findings" (reviewer D4ag). Reviewer v9qp, and D4ag also noted that our findings in learning stages and OOD generalization being insightful.

Before addressing each of your points in detail , we would like to draw your attention to our general responses, which provide context and address several common concerns raised across the reviews.

• Q1: There are several critical measures, such as effective radius, dimension, and center alignment, but they are not defined (at least not in the main paper).
  • A1a: We thank the reviewer for pointing out the confusion in the previous version of the manuscript. We have taken these suggestions seriously and significantly improved clarity in the revised version. In particular, we now provide detailed definitions and intuitive explanations of the geometric measures in lines 205–223 of the main paper. Additionally, Fig. 2c has been updated to include an intuitive depiction of how these geometric measures influence manifold capacity. We also kindly invite the reviewer to look at Issue 1 in general response which provides a detailed explanation on the definition of these metrics.
  • A1b: We have also improved the presentation of the definition of manifold capacity in Section 2.1, now including concrete mathematical equations and definitions for clarity. Furthermore, Appendix B on manifold capacity theory has been significantly restructured for better readability. We look forward to the reviewer’s thoughts on the revised version and welcome any further suggestions or comments.
• Q2: The main paper claimed that Theorem 1 is proved for a 2-layer NN. However, the NN used in the proof is different from what people would expect to be a "2-layer NN," since, based on Assumption 1: (1) the second layer is not trained but set to random, and (2) the choice of activation function is very limited, as there is a strong condition on the activation function.
  • A2a: We now highlight this limitation in the main text and discuss its connection to the literature. Specifically, we explain the motivation to only update the first-layer weight W and fix the readout layer weight a in the main text (lines 287-289). Particularly, following the convention from prior work [Ba et al., NeurIPS 2022], the second-layer (read-out weight) is fixed to (1) avoid lazy learning, in which the network minimally adjust the hidden layer representation and focus on learning the readout weight, and (2) enable mathematically analysis focusing on the hidden layer representations.
  • A2b: Regarding the choice of activation, we are using standard assumptions from the theory literature. The following is a quote from [Ba et al., NeurIPS 2022]: “Following [Hu and Lu, ToIT 2022], we assume smooth centered activation to simplify the computation; empirical evidence suggests that similar result holds beyond this condition (e.g. [Loureiro et al., NeurIPS 2021]). We also expect the Gaussian input assumption may be replaced by weaker orthogonality conditions as in [Fan and Wang, NeurIPS 2020].”
  • A2c: We interpret these results as a justification for using capacity to track the degree of richness in feature learning in a well-studied theoretical setting. We remark that our proof requires substantial technical improvements from previous work [Ba et al., NeurIPS 2022] due to the fundamental difference between regression and classification.

We thank Reviewer cJiT for your time and effort for providing detailed suggestions and questions to improve our manuscript.

To recap, in this paper, we propose using manifold capacity and its geometric measures to investigate the lazy vs. rich dichotomy in feature learning. We appreciate that the reviewers found our approach using manifold capacity to measure feature learning to be "novel application" (reviewer nCob, cJiT), have "better alignment with the degree of feature learning compared to other metrics" (reviewer nCob), with "robust experiments substantiate these findings" (reviewer D4ag). Reviewer v9qp, and D4ag also noted that our findings in learning stages and OOD generalization being insightful.

Before addressing each of your points in detail , we would like to draw your attention to our general responses, which provide context and address several common concerns raised across the reviews.

• Q1: There seems to be no explanation for Figure 2c. I don’t know what those operations mean.
  • A1a: In Figure 2c, we presented how the changes of geometric measures would affect the capacity value as shown in [Chou et al., 2024]. For example, smaller manifold radius and smaller manifold dimension would make the capacity value higher as the manifolds become more separable. See the revised version for pictorial intuition and more examples.
  • A1b: In the revised version (lines 225–237), we have added more explanations for Figure 2. We have also included the formula and interpretation for all of the geometric measurements in Section 2.2. In Appendix B.4 and B.5, we provide examples illustrating how these geometric measures interact with capacity as shown in Figure 2c. We look forward to the reviewer’s thoughts on the revised version and welcome any further suggestions and comments.
• Q2: I’m a bit confused about eq (1), the definition of model capacity. Are the manifolds Mi predefined, or do they change as N increases? If they change, how do you choose them? Also, what values should it take?
  • A2: We apologize for the confusion in the previous version. In the revised version first introduces (1) simulated capacity in line 168-177 first, which is more intuitive and can be empirically computed, but cannot be analytically analyzed. Next, we present (2) the capacity formula of the mean-field approximation (line 187-190), which is analytically trackable, more efficient for empirical computation, and link to manifold geometric measures (as delineated later). The definition of mean-field approximation (which is the one we put in the previous version) is now deferred to the Appendix B.3 (line 979-1020). We remark that these are results from [Chung et al., PRX 2018] and [Chou et al., 2024]. We also kindly invite the reviewer to look at Issue 1 in general response which provides a detailed explanation on the definition of manifold capacity.
• Q3: In Appendix C, several notions for computing manifold capacity, such as T and λ, are not defined.
  • A3: Thank you for pointing out this oversight. We have addressed the issue in Appendix B (the previous Appendix C has been reorganized) and double-check that all our notations are consistent.
• Q4: The way of computing manifold capacity should be mentioned in the main text, or at least under what conditions these values are computed. (I believe they cannot be exactly computed unless certain conditions are assumed.)
  • A4a: We appreciate the reviewer’s insightful point. Indeed, manifold capacity cannot be computed exactly; instead, we use a mean-field approximation formula.
  • A4b: We have carefully addressed this concern by adding more explanations about the connection between manifold capacity and its mean-field approximation in lines 180–200. Specifically, we've included in the main text Equation (1) at line 187 the results of the mean-field approximation, which can be empirically computed. For completeness, we also provide pseudocode for estimating manifold capacity via the mean-field formula on page 21. We look forward to the reviewer’s thoughts on the revised version and welcome further suggestions or comments.

I appreciate authors' response to address my concerns. The revised version improves the clarity. I therefore increase my score.

We thank Reviewer nCob for your time and effort for providing detailed suggestions and questions to improve our manuscript.

To recap, in this paper, we propose using manifold capacity and its geometric measures to investigate the lazy vs. rich dichotomy in feature learning. We appreciate that the reviewers found our approach using manifold capacity to measure feature learning to be "novel application" (reviewer nCob, cJiT), have "better alignment with the degree of feature learning compared to other metrics" (reviewer nCob), with "robust experiments substantiate these findings" (reviewer D4ag). Reviewer v9qp, and D4ag also noted that our findings in learning stages and OOD generalization being insightful.

Before addressing each of your points in detail , we would like to draw your attention to our general responses, which provide context and address several common concerns raised across the reviews.

• Q1: The theoretical derivation relies on a one-step gradient argument, but this fact is not mentioned in the manuscript.
  • A1a: We follow the convention of prior work [Ba et al., NeurIPS 2022], which studied the prediction risk of ridge regression in the same 2-layer NN setting in the feature learning regime. Specifically, they focused only on presenting results for a 1-step gradient update. For simplicity, we also presented results for a 1-step update. Extending beyond one gradient step could break the key Gaussian equivalence step (as noted in footnote 2 of [Ba et al., NeurIPS 2022]) and presents significant mathematical challenges in this setting.
  • A1b: We addressed this limitation in footnote 6 on page 6 of the revised version.
  • A1c: We interpret these results as a justification for using capacity to track the degree of richness in feature learning in a well-studied theoretical setting. We remark that our proof requires substantial technical improvements from previous work [Ba et al., NeurIPS 2022] due to the fundamental difference between regression and classification.
• Q2: The link between Theorem 2’s results and the increase in the degree of feature learning is not entirely clear, and additional commentary could enhance clarity.
  • A2a: First, we clarify that we are working with a teacher-student setting where the data labels are correlated with a hidden signal direction (see Setting 1, lines 1164–1168).
  • A2b: In this context, the first-step gradient update can be approximated by a rank-1 matrix containing label information, aligning the updated weights with the hidden signal (Proposition 1, lines 1236–1241; see also Proposition 2 in [Ba et al., NeurIPS 2022]). Thus, in this setting, the learning rate serves as the ground truth to measure the amount of task-relevant information (i.e., richness in learning) within the model representation after gradient updates. We have also added this explanation to the main text (section 3.1, L290-295).
• Q3: It would be helpful to discuss the generalizability of the observations, such as those in Figure 4. Various hyperparameters (e.g., the choice of optimization algorithms, weight initialization methods, batch size, learning rate, and scheduling) could influence implicit biases in the algorithm, affecting neural representations, geometric metrics, and even the stages of learning.
  • A3a: This is an excellent question. In general, we aim for geometric measures that strike a balance between (i) detecting differences in manifold organization and (ii) being robust to minor changes that do not significantly affect performance.
  • A3b: For 2-layer NNs, we found the results to be robust across different hyperparameters (see Appendix D for settings varying network parameters, data distributions, and optimization algorithms). We’ve added remarks about this robustness in the main text.
  • A3c: For DNNs, the results were generally robust within the same architecture but could vary across architectures. For instance, in the OOD experiments, we observed geometric differences in VGG-11 (Fig. 6) and ResNet-18 (Fig. 22).
• Q4: Regarding the proposed effective geometric measures to explain capacity changes, are there standard reference lengths compared to the radius? A simple manifold radius magnitude may not accurately capture the problem's complexity when the differences between manifold means scale identically.
  • A4: This is a very insightful question! The radius is normalized by the norm of the manifold center, allowing it to accurately (and monotonically) capture the packability of manifolds. We've added the formula for all the geometric measures, including the radius, in the main text (line 207-line 220).

Issue 2: Reviewers highlighted the confusion about the concrete setting of our theoretical results and how to interpret them, specifically how the results directly relate to the degree of feature learning.

We built on previous work from [Ba et al. NeurIPS 2022] and [Montanari et al., 2019] to analytically characterize the connection between capacity, prediction error, and the effective degree of richness in a well-studied theoretical model. We interpret our theoretical results as justifications for using capacity to track the degree of richness in feature learning. We remark that our proof requires substantial technical improvements from previous work [Ba et al., NeurIPS 2022] due to the fundamental difference between regression and classification. The following is a list of questions regarding our theoretical results raised by the reviewers and our responses:

• Reviewer nCob, cJiT asked for a clear explanation of the assumptions and precise results in the theoretical framework, particularly the use of a 1-step gradient update.
  • We agree that the precise scope of the theoretical results, particularly about capacity after one-step gradient update, is important to the result interpretation, and have added this assumption in the main text. We provide more context on this assumption below.
  • Here we follow the convention of a previous work [Ba et al., NeurIPS 2022] which studied the prediction risk of ridge regression in the same 2-layer NN setting in the feature learning regime. In particular, they only focused on presenting the result of 1-step gradient update. So for the simplicity of presentation, we also just presented the result of 1-step update. Indeed, going beyond one gradient steps could break the key Gaussian equivalence step (as remarked in footnote 2 in [Ba et al., NeurIPS 2022]) and present a significant mathematical challenge in this setting.
  • We addressed this issue in footnote 6 at the bottom of page 6 in the revised version.
• Reviewer v9qp asked for why the readout weights have been fixed throughout training.
  • The use of a 2-layer network with fixed readout weights is standard in related work (e.g., [Ba et al., NeurIPS 2022]), allowing for mathematical characterization and control over the interpolation between lazy and rich learning by adjusting the learning rate.
  • Our results are technically non-trivial, as previous work focused on regression settings, while we address classification tasks, requiring different mathematical treatments.
• Reviewer nCob, cJiT, v9qp, K6x2 asked for the interpretations of our theoretical results, the relevance of the result to feature learning, as well as the position in the literature.
  • We interpret these results as justifications for using capacity to track the degree of richness in feature learning in a well-studied theoretical setting.
  • About the link between Theorem 1 result (which states capacity after 1-step gradient update is a monotonically increasing function of the learning rate) to the degree of feature learning, in our setting, first-step gradient update can be approximated by a rank-1 matrix that contains label information, resulting in the updated weight to be more aligned with the hidden signal β*. Hence, in this setting, the learning rate η can be used as the ground-truth to measure the amount of task-relevant information (i.e., richness in learning) in the model representation after gradient updates. We have also added this explanation in the main text for better clarification (see line 298-300 and footnote 6 at the bottom of page 6).
  • We remark that our proof requires substantial technical improvements from previous work [Ba et al., NeurIPS 2022] due to the fundamental difference between regression and classification.

Q1-2: Reviewer nCob, cJiT, v9qp, D4ag noted the lack of mathematical definitions for geometric measures in the main text, which makes it difficult to follow and interpret the geometric results.

A1-2: In the revised version, after presenting the mean-field approximation, we introduce the concept of anchor points and define all relevant geometric measures in detail (line 205-220). Examples in the Appendix B.5 further demonstrate the relationship between capacity and these geometric measures, making the connections clear and intuitive. We remark that these are results from [Chung et al., PRX 2018] and [Chou et al., 2024].

The following is a list of formal definition of effective geometric measures (please read them along with the capacity formula (2) above):

• Anchor points: for each $\mathbf{y},T$ and $I\in[P]$, define \{\mathbf{s}\_i(\mathbf{y},T)\} = y_i\cdot\arg\max\_{\{\mathbf{s}\_i\}}\\|\text{proj}_\{\text{cone}(\{y_i\mathbf{s}_i\})}T\\|_2^2 as the anchor points with respect to $\mathbf{y}$ and $T$.
  • Center and axis part of anchor points: For each $i\in[P]$, define \mathbf{s}_i^0:=\mathbb{E}\_\{\mathbf{y},T}[\mathbf{s}_i(\mathbf{y},T)] as the center of the $i$-th manifold and define $\mathbf{s}_i^1(\mathbf{y},T):=\mathbf{s}_i(\mathbf{y},T)-\mathbf{s}_i^0$ to be the axis part of $\mathbf{s}_i(\mathbf{y},T)$ for each pair of $(\mathbf{y},T)$.
• Manifold geometric measures:
  • Manifold dimension captures the degree of freedom of the noises/variations within the manifolds. Formally, it is defined as D_\text{mf}:=\mathbb{E}_\{\mathbf{y},T}[\\|\text{proj}\_\{\text{cone}(\{\mathbf{s}_i^1(\mathbf{y},T)\})}T\\|_2^2].
  • Manifold radius captures the noise-to-signal ratio of the manifolds. Formally, it is defined as R_\text{mf} := \sqrt{\mathbb{E}\_\{\mathbf{y},T}\left[\frac{\\|\text{proj}\_\{\text{cone}(\{\mathbf{s}\_i(\mathbf{y},T)\})}T\\|^2}{\\|\text{proj}\_\{\text{cone}(\{\mathbf{s}^1_i(\mathbf{y},T)\})}T\\|^2-\\|\text{proj}\_\{\text{cone}(\{\mathbf{s}\_\{i}(\mathbf{y},T)\})}T\\|^2}\right]}.
  • Center alignment captures the correlation between the center of different manifolds. Formally, it is defined as $\rho^c_\text{mf}:=\frac{1}{P(P-1)}\sum_{i\neq j}|\langle\mathbf{s}_i^0,\mathbf{s}_j^0\rangle|$.
  • Axis alignment captures the correlation between the axis of different manifolds. Formally, it is defined as $\rho^a_\text{mf}:=\frac{1}{P(P-1)}\sum_{i\neq j}\mathbb{E}\_{\mathbf{y},T}[|\langle\mathbf{s}_i^1(\mathbf{y},T),\mathbf{s}_j^1(\mathbf{y},T)\rangle|]$.
  • Center-axis alignment captures the correlation between the center and axis of different manifolds. Formally, it is defined as $\psi_\text{mf}:=\frac{1}{P(P-1)}\sum_{i\neq j}\mathbb{E}\_{\mathbf{y},T}[|\langle\mathbf{s}_i^0,\mathbf{s}_j^1(\mathbf{y},T)\rangle|]$.

Issue 1: Reviewers asked for formal introduction to manifold capacity, the algorithm to compute manifold capacity, and its connection to effective geometric measures.

We briefly recall that manifold capacity is a recently proposed metric [Chung et al., PRX 2018; Chou et al., 2024] for measuring how separable neural manifolds are in an average-case setting (i.e., the degree of manifold untangling). It has been widely applied in the neuroscience community to quantify the progression of neural computation in the brain. The main contribution of our work is to use manifold capacity as a representation-based method to quantify the degree of richness in feature learning. To the best of our knowledge, no previous work has proposed a representational-based method that can quantify task-relevant changes in neural representations. For instance, the most common method involves tracking weight changes in a network. However, a network might change its weights while degrading its representations. Another common method, which is to track the change in the norm of the neural tangent kernel (NTK), requires the knowledge of the entire model weights to compute the NTK.

The following is a list of questions regarding the clarity of manifold capacity and relevant geometric measures raised by the reviewers and our responses:

Q1-1: Reviewer cJiT, v9qp, K6x2 asked for a clearer definition of manifold capacity. In the initial version, we included the mean-field definition of manifold capacity in the proportional limit but did not sufficiently explain how it is empirically computed. Also, reviewers cJiT, D4ag raised concerns about the practicality of calculating this value.

A1-1: To address these issues, the revised version first introduces simulated capacity in line 168-177 first, which is more intuitive and can be empirically computed, but cannot be analytically analyzed. Next, we present the capacity formula of the mean-field approximation (line 187-190), which is analytically trackable, more efficient for empirical computation, and link to manifold geometric measures (as delineated later). The definition of mean-field approximation (which is the one we put in the previous version) is now deferred to the Appendix B.3 (line 979-1020). We remark that these are results from [Chung et al., PRX 2018] and [Chou et al., 2024].

• (Simulated capacity) The following is the definition of simulation capacity. See also Section 2.1 in the revised version for more explanation.

Definition: (Simulated capacity.) Let $P,N\in\mathbb{N}$ and $\mathcal{M}_i\subseteq\mathbb{R}^N$ be a convex set for each $i\in[P]=\{1,\dots,P\}$. For each $n\in[N]$, define

where $\mathbf{y}$ is a random dichotomy sampled from $\\{\pm1\\}^P$ and $\Pi_n$ is a random projection operator from $\mathbb{R}^N$ to $\mathbb{R}^n$. Suppose $p_N=1$, the simulated capacity of $\\{\mathcal{M}_i\\}\_{i\in[P]}$ is defined as

• (Capacity formula) The following is the mean-field approximation formula to manifold capacity (note that this will be very important for the definition of geometric measures):

where $\mathbf{y}$ is randomly sampled from $\\{\pm1\\}^P$ and $T$ is randomly sampled from $\mathcal{N}(0,I_N)$, the multivariate Gaussian distribution with mean $0$ and covariance $I_N$. $\text{cone}(\cdot)$ is the convex cone spanned by the vectors, i.e., $\text{cone}(\\{y_i\mathbf{s}_i\\})=\\{\sum_i\lambda_iy_i\mathbf{s}_i\\, :\\, \lambda_i\geq0\\}$.

Remark: [Chou et al., 2024] showed that $|\alpha_{\text{sim}}-\alpha_{\text{mf}}|<O(1/N)$.