Is Your Diffusion Model Actually Denoising?

We thank reviewer i2tR for their thoughtful feedback and address their concerns, approximately in order:

1. Self-guidance experiments:

   We acknowledge that we do not directly demonstrate self-guidance in learned models, but instead show the capability of self-guidance to reasonably predict the samples and Schedule Deviation of a learned model.

   It is unclear how to demonstrate self-guidance in itself besides comparing generated samples. Using the error in the flow between the closed-form and the learned model is difficult to interpret as there is no “baseline” flow that we can contrast self-guidance to in order to show that the computed error is reasonably low. We agree that showing that self-guidance is inherently “true” of models would greatly strengthen this paper and are interested in any ideas the reviewer has for how to effectively design such an experiment.

   We naturally do not expect self-guidance to be exactly true for larger models, which necessarily must have more complicated inductive biases than purely linear interpolation of different scores–there is likely significant coupling between x and z which, as we note in Remark 4.1, our simple model cannot capture. However, we believe the “self-guidance” model gives useful intuition for why Schedule Deviation may be fundamental to conditional diffusion.

2. Scaling/computing the proposed schedule deviation metric:

   While we acknowledge that our metric is expensive, we emphasize that it is not prohibitively so: we demonstrate scalability up to ~40 million parameter models without issue.

   We believe that with sufficient compute resources (we would estimate at most several dozen GPUs and a couple weeks’ time) the metric could provide meaningful evaluation of smaller-scale text-conditioned models (~1 billion parameters for e.g. Stable Diffusion 3).

   Unfortunately, this is beyond the limitations of the resources we can acquire, although we believe it is feasible to scale: the metric is highly parallelizable and has approximately the same memory complexity as performing a single forward-backward pass.

   We use t-SNE-conditioned image generation in this work as a small-scale proxy for larger text-conditioned diffusion models and, following the suggestions of Reviewer zm5H, will incorporate a human-faces dataset conditioned on continuous attributes such as age for our next revision as a better proxy for the continuously-conditioned diffusion task.

   We welcome any suggestions from the reviewer for additional conditional generation tasks they believe would be interesting in the ~100 m parameter regime which we could include in the final revision.

3. Unconditional diffusion:

   Unconditional generation should behave analogously to conditional generation in a region with a high density of conditioning data–that is to say, low schedule deviation. While there is still some Schedule Deviation in these regions (presumably due to smoothing/inductive biases with respect to the x and t inputs that our analysis does not capture), this effect is fairly minimal compared to regions with low-density of conditioning support.

   Because the forward process has support everywhere, with a sufficiently large model and enough training time it will always be possible to exactly model the denoiser of the training dataset and overfit to the training data, e.g. achieve low Schedule Deviation for the unconditional setting. However, for conditional models, we show that Schedule Deviation cannot be easily ameliorated by increasing the training data or the model capacity.

   That is not to say there is no deviation from the IDDF in unconditional settings, just that the flow is believed to much more closely approximate a denoising process. The effect of this kind of smoothing has been studied in prior work such as [Aithal 2024], [Scarvelis 2023]. We believe follow-up work explicitly contrasting Schedule Deviation in conditional and unconditional settings to be of interest.

4. More complex conditions:

   Naturally, we do not expect that larger trained models will interpolate via linear combination of the scores–their true behavior is likely a much more complex, architecture-dependent non-parametric. We use Section 4 to highlight how natural and simple inductive biases can be fundamentally at odds with denoising in a manner that is distinct from the smoothing effects studied in unconditional models.

   Although we present no evidence in this paper that self-guidance is directly predictive for larger models, the widespread use and strong empirical performance of classifier-free guidance suggests that self-guidance is an excellent inductive bias. We suggest that better theoretical understanding of classifier-free guidance may be applicable to conditional models as well.

   For higher dimensions, we believe the correct primitive to consider is some complicated nonlinear spline rather than a learned geodesic, as we cannot expect the flow along a path to be purely a combination of its two endpoints. We are considering follow-up directions for larger models motivated by the concept of self-guidance along the lines of what you suggest: namely, can we express a conditional score in terms of a finite basis of unconditional functions, with learned interpolating coefficients? Whether conditional diffusion models interpolate their flows in some low-dimensional manifold is an interesting hypothesis. We hope to study this question in future work.

[Aithal 2024] Aithal, S. K., Maini, P., Lipton, Z., & Kolter, J. Z. (2024). Understanding hallucinations in diffusion models through mode interpolation. Advances in Neural Information Processing Systems, 37, 134614-134644.

[Scarvelis 2023] Scarvelis, C., Borde, H. S. D. O., & Solomon, J. (2023). Closed-form diffusion models. arXiv preprint arXiv:2310.12395.

We thank the reviewer for their review and questions. We address your concerns in order:

1. Motivation:

   We consolidate the different motivations and interpretations of our metric below and will be sure to revise our introduction and preliminaries to make the motivation more explicit:

   We address shortcoming with metrics developed for unconditional settings: It is believed the score-smoothing/non-denoising behavior of models is fundamentally related to generalization in unconditional diffusion [see Scarvelis, Aithal 2024]. However, existing metrics such as the “error from the ideal closed-form denoiser” ($||v^\star_t(x) - v_t(x)||$) [see Scarvelis] require samples from the true data distribution $p^\star(x)$ to compute $v^\star_t(x)$. For conditional diffusion (particularly with continuous conditioning values) we may potentially have either one or zero samples for a given “z” value and therefore cannot estimate the ideal denoiser for $p^\star(x|z)$. This is what principally motivates the construction of a new metric which can be evaluated even for conditional models.

   Existing metrics are ad-hoc designed in other ways. Namely, while the true denoiser/flow $v^\star$ can be estimated from the training data, there are many choices for the distribution over $x$ on which to evaluate $||v^\star_t(x) - v_t(x)||$. It is unclear whether to use (1) reverse process under the learned v_t starting from noise or (2) the forward process using samples from $p^\star(x)$.

   We argue that the principled choice of distribution over x to consider is the one induced by (1) running the full reverse process under the learned $v$ and then (2) subsequently re-noising the generated samples using the forward process. We use $v^{IDDF}$, “the idealized flow” to refer to the flow for the forward process applied to the p(x | z) distribution of the reverse process under the learned v. Crucially, we can compute $v^{IDDF}$ without access to $p^\star(x|z)$ using only the learned $v$, meaning our metric can be computed for conditional models.

2. Intuition:

   Note that while we evaluate the Schedule Deviation using Proposition 3.1 (which resembles earlier metrics), we cleanly derive the form in Proposition 3.1 from the instantaneous rate of deviation of $p^v$ from $p^{IDDF}$ as defined in Definition 3.1. In other words, for some intermediate timestep $t$, if we start at $p^{IDDF}_t(x)$, the Schedule Deviation measures the total rate at which the density under $v^{IDDF}$ diverges from the same density evolved under $v$. We believe this is much easier and more intuitive than the “error from a reference denoiser” as in prior work.

   Proposition 3.1 gives an alternate interpretation of our metric as the error between $v$ and $v^{IDDF}$ along the direction of $\nabla \log p^{IDDF}_t(x)$, plus a divergence-based correction term. This reveals that not all forms of error between $v$ and $v^{IDDF}$ cause deviation from a denoising probability path. At first order, it is only the error along $\nabla \log p^{IDDF}_t(x)$ which causes deviation, a property which our metric makes immediately clear. This makes intuitive sense, as the transport equation (Eq A.1) implies that adding a “curl” component to $p_t(x)v_t(x)$ has no effect on the evolution of the density $p_t(x)$, i.e. $v$ can be arbitrarily far from $v^{IDDF}$ without affecting the density.

   Lastly, as we note just before Theorem 1, our metric resembles the advantage function used in the storied Performance Difference Lemma from reinforcement learning, applied to the evolution of densities. Indeed, an extension of the Performance Difference Lemma is central to the proof of Theorem 1 found in Appendix A.

3. Novelty and Connection to SDE Literature: To our knowledge these contributions above are all novel and we are unaware of a similar metric in either the SDE, optimal transport, or diffusion literature beyond the aforementioned prior works.

   We base our formulation of diffusion on the Stochastic Interpolants and flow-matching frameworks [Albergo, Lipman], introducing diffusion models using its flow-based ODE formulation rather than the (equivalent) SDE formulation as it simplifies the notion of having a non-denoising flow and are generally easier to reason about the SDEs, which require significantly more technical mathematical treatment.

   Note that we additionally provide SDE-based sampling procedures and equivalent formulations in Appendix B for interested readers who wish to understand SDE-based sampling works in the context of our paper. We will signpost to this appendix for readers who are interested in this.

   We are of course happy to reference the connections to the SDE-based literature beyond the original diffusion works by Sohl-Dickenstein, Ho, and Song, which were of course principally SDE-based. We also will create an extended related works in the appendix with the relevant SDE related work.

4. Why not estimate TV directly? Estimating the TV distance directly requires calculating the difference in densities between $p_t$ and $p_t^{IDDF}$ over the entire space x. As we cannot integrate over the space of all images, it is not easy to compute the TV distance directly. Estimating the associated densities $p_t(x)$, $p_t^{IDDF}$ (or their ratios) is also numerically unstable in practice.

   In contrast, our method for computing Schedule Deviation only requires sampling x from $p_t^{IDDF}$, meaning we can restrict our computation to areas with high likelihood. Furthermore, the quantities involved are all log-densities or their gradients,

   Note that $TV(p_t^{IDDF} | p_t)$ depends only on $v_s(x)$ for $s > t$, not $v_t(x)$ itself. Thus, given only the flow $v_t(x)$ at time $t$, it is more intuitive to relate the value of $v_t(x)$ to the different in the change of the density $p_t$ compared to $p_t^{IDDF}$, which is precisely what Schedule Deviation captures.

[Albergo 2023] Albergo, M. S., Boffi, N. M., & Vanden-Eijnden, E. (2023). Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.08797.

[Aithal 2024] Aithal, S. K., Maini, P., Lipton, Z., & Kolter, J. Z. (2024). Understanding hallucinations in diffusion models through mode interpolation. Advances in Neural Information Processing Systems, 37, 134614-134644.

[Lipman 2022] Lipman, Y., Chen, R. T., Ben-Hamu, H., Nickel, M., & Le, M. (2022). Flow matching for generative modeling. arXiv preprint arXiv:2210.02747.

[Scarvelis 2023] Scarvelis, C., Borde, H. S. D. O., & Solomon, J. (2023). Closed-form diffusion models. arXiv preprint arXiv:2310.12395.

The authors have adequately addressed this reviewers concerns about the motivation and intuition behind the authors results.

Furthermore, after consideration of the authors response to the other reviewer's concerns about experiments, the score will be bumped up by 2.

Thank you for the thorough reading and the in-depth review. We will clarify several potential sources of confusion that we realize may not have been sufficiently emphasized in the text.

1. Clarifying the Construction of the IDDF:

   The $p_t(x|z)$ (including $p_0(x|z)$) used in the definition of Schedule Deviation are the marginals consistent with the learned flow $v$, not the true flow $v^\star$. That is to say, $p_0$ arises from the reverse process under the learned model and is not the distribution of the training set (which we define as $p^\star(x|z)$ at the start of Section 2). This makes $p_0(x|z)$ and, by extension, $v^{IDDF}$ well-defined even when far from the support of the training data. We address this point explicitly in Remark 3.1.

   Given the understandable confusion, we will (1) move Remark 3.1 into the preliminaries and (2) make the preliminaries less dense to read, with the appropriate references to Karras and other similar frameworks, (3) we will use a different symbol besides $v$ in the definition of Schedule Deviation to denote the learned flow and (4) we will also rename IDDF from “Ideal Denoising Diffusion Flow” to “Model Consistent Denoising Flow” (MCDF) in order to make the distinction between the ground-truth flow $v^\star$ and $v^{IDDF}$ immediately clear.

   Hopefully this mitigates your concern that the IDDF is based on the empirical score of the training set. In fact, because we do not rely on our training set at all, the schedule deviation can be computed solely given the pretrained model, an advantage we will highlight in our revised introduction.

2. Positioning with Respect to Prior Work:

   We use a flow-based formulation as it is simpler to reason about what it means to diverge from the ideal flow and mathematically simpler to manipulate than the SDE formulation. We based our formulation off of [Albergo 2023] but naturally the same framework exists in other works as well. We include an SDE formulation as well in Appendix B for completeness and will of course cite [Karras] as well.

   Previous work (e.g., Scarvelis, Chen) uses smoothing of the ideal denoiser to study generalization in unconditional models. In contrast, we argue that for conditional diffusion, "approximately denoising" is less informative due to the much higher Schedule Deviation, which we explain by our model of self-guidance. We will include [Wang], [Li], and [Chen] in our prior work and explain how our proposed metric and interpretation of conditional diffusion are fundamentally different from the variants they consider.

3. Schedule Deviation and “Fuzziness” on the Maze Dataset:

   The correlation between Schedule Deviation and DDPM/DDIM for the Maze dataset is still quite strong. The “fuzziness” in Figure 4 is due to when the conditioning is exactly on the walls between the cells in the Maze (see Figure 7), which exhibit extremely high OT distance between DDPM/DDIM exactly on the boundary (Figure 2, upper left) but where the Schedule Deviation (Figure 2, lower left) looks much more clean. Removal of these boundary points makes the association much more clear.

   We believe the noise in the optimal transport distance is due to the OT distance we use (1-Wasserstein distance) being very sensitive to changes in the probabilities for modes that are very far apart, such as when evaluating on the “walls” of the maze. Using “clipped” distances in the cost matrix for the OT distances can remove the associated “fuzziness” you observe, but we did not want to create the appearance of hacking the metric to be more favorable and justify why we used a non-standard metric. In short, we believe this is due to the OT calculation rather than our metric, which does not exhibit such outliers.

4. Relation to Simpler Metrics:

   As you point out, error between the learned model and the ideal flow is sufficient to give an upper bound on the deviation of the probability densities. The key strength of our metric is that it also has a lower bound for the deviation of the probabilities.

   Any simpler metric based on error with the ideal flow cannot yield a lower bound. By the transport equation (Eq A.1), any divergence-free component of $p_t(x|z)v_t(x|z)$ has no effect on the evolution of the marginals of the reverse process. Thus it is possible to add an arbitrary “curl” component to $v^IDDF$ while preserving the same marginals. Thus the “error” between $v$ and $v^IDDF$ can be arbitrarily high despite the fact that both the intermediate marginals and the final distributions are identical. By contrast, our metric directly measures the instantaneous deviation in how the densities are evolved and is agnostic to any such “curl” components in the flow. We will better explain this fact and improve the motivation for our metric in the preliminaries.

   Comparing our metric empirically to the error wrt $v^{IDDF}$ is an excellent suggestion, and we will develop some simple experiments or examples which contrast the two.

5. Regarding the low-dimensional experiments:

   We welcome the reviewer’s suggestion of using a human-faces dataset, and intend to incorporate an experiment on either FFHQ or CelebA using continuous conditioning attributes for a subsequent revision in addition to our existing experiments in Section 3.

   The datasets for the experiments in Section 4 are designed to mirror the settings of Theorem 2/Theorem 3, where we can analytically compute the smoothest possible interpolant. Unfortunately deriving a closed-form analogous to Theorem 2 or 3 for the conditioning space of more complex data such as MNIST or FFHQ seems generally infeasible. The setting of Theorem 2 is solvable due to specific properties of cubic spline interpolations and Theorem 3 is due to the uniform weighting of the conditioning distribution analyzed.

   Although Theorem 3 could potentially be generalized to non-uniform densities, the final expression would involve nested Fourier and inverse Fourier transforms in the final expression which cannot be further simplified in the general case. We believe that deriving the closed form under other “toy” conditioning distributions or smoothness penalties (yielding correspondingly different spline interpolations in discrete conditioning support analogous to Theorem 2) may be more pedagogically instructive for understanding biases in conditional diffusion. We believe such investigations would perhaps be a good fit for a future, explicitly theoretically-focused work.

   In the case of MNIST or FFHQ, any predictive model would almost certainly need to incorporate inductive biases of the NN architecture as well. Modeling these biases is out of scope for this work and we direct the reader to [Kamb 2024] which investigates locality biases in purely convolutional architectures. We will be sure to expand on this in our prior work.

6. Schedule Deviation vs Fidelity:

   Schedule deviation measures whether a model is “consistent” in representing the denoiser for its own final distribution. It is possible to construct models with low schedule deviation that poorly model the desired conditional distribution or, vice-versa, models with high schedule deviation that model the distribution extremely well. For this reason we note in Remark 3.1 that schedule deviation is orthogonal to fidelity.

   We do not claim that schedule deviation needs to be mitigated. In fact, given how effective classifier-free guidance is, schedule deviation arising from “self-guidance”-style smoothing may be an excellent inductive bias.

Addressing other miscellaneous questions/comments:

1. As discussed above, our metric is much stronger than the prior variants used in the literature. We will be sure to expand the prior work to better distinguish the advantages of our approach to justify the additional computational complexity of our approach. We encourage the reviewer to read our response to similar questions raised by reviewer daH1.

2. We did not use classifier-free guidance in these experiments as we were interested in the inductive biases of learning the conditional score by itself. It is known that classifier free guidance results in non-denoising paths [Bradley 2024] (also potentially relevant to the effect of guidance is Skreta 2024), so it would certainly be interesting to investigate empirically the Schedule Deviation of either classifier-based or classifier-free guidance. We believe there are many settings where evaluating the Schedule Deviation may be of interest and unfortunately are not able to address all of them in this paper, but will do so in a follow-up work.

3. We emphasize the term “self guidance” to highlight that better understanding of guidance (which remains poorly understood) may simultaneously yield better understanding of how conditional diffusion models generalize. While it is perhaps intuitive for Theorem 2/3 that smoothness regularization yields a linear combination of the flows at the original datapoints, that this combination is independent of the choice of z is surprising.

[Albergo 2024] Albergo, M. S., Boffi, N. M., & Vanden-Eijnden, E. (2023). Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.08797.

[Kamb 2024] Kamb, M., & Ganguli, S. (2024). An analytic theory of creativity in convolutional diffusion models. arXiv preprint arXiv:2412.20292.

[Bradley 2024] Bradley, A., & Nakkiran, P. (2024). Classifier-free guidance is a predictor-corrector. arXiv preprint arXiv:2408.09000.

[Skreta 2024] Skreta, M., Atanackovic, L., Bose, A. J., Tong, A., & Neklyudov, K. (2024). The Superposition of Diffusion Models Using the It^ o Density Estimator. arXiv preprint arXiv:2412.17762.

We thank the reviewer for their feedback on the presentation of the paper and recommendations. Unfortunately we cannot update the paper until the camera ready deadline, but will detail the changes we intend to make given the reviewer’s constructive feedback.

1. Experiments:

   Although we present only proxies for large-scale models/tasks, we are able to evaluate models up to the ~40 million parameter regime without issue. Thus we believe that with compute resources beyond those in academia this could be applied to smaller-sized text-conditioned models or robot foundation models (given perhaps several weeks compute time on a couple dozen GPUs).

   In addition to the existing image experiments, we will replicate our methodology on either CelebA-Dialog or an annotated FFHQ which contains fine-grained labels for attributes such as age, degree of smile, amount of facial hair, etc. This will serve as a better proxy for text-based conditioning than t-SNE conditional MNIST without incurring the expense of evaluating models in the billion-plus parameter regime.

   We originally ran experiments on the Push-T continuous control environment but found essentially no Schedule Deviation for the state-conditioned models, which we attribute to a lack of multi-modality in the action distributions. We believe this is due to overfitting of these large (50m+ parameter models) to very few (~200) human demonstrations, meaning these models behave more like an adaptive nearest-neighbor approximator than a multi-modal action model. Guidance between with identical, translated distributions simply shifts the mean and does not result in Schedule Deviation (see Appendix B for a discussion of Schedule Deviation for Gaussians), so it is unsurprising that we found no Schedule Deviation for essentially deterministic Diffusion Policies. We conjecture that for robot foundation models this “effective unimodality” is not present with large multi-task training datasets and image-conditioned models, although we acknowledge that additional studies of these models are needed.

   When designing the maze navigation task, we explicitly introduced multi-modality in the action distributions by considering a weighted mixture of all possible paths to the target point when generating the data. Indeed, higher Schedule Deviation can be observed at “bifurcation” points in Figure 3 where the trajectory distribution is more multi-modal.

   We would certainly like to run additional continuous-control experiments in other environments. However, we are unsure what an environment would look like where the diffusion policy is not effectively deterministic. We welcome suggestions from the reviewer on what environment they feel would be most appropriate.

2. Presentation:

   We acknowledge that placing Figure 1 so early is too heavy and confusing for readers. We will move this to coincide with the main experiments to make the introduction more logical and make an easier-to-digest figure (perhaps based on the low-dimensional experiments in Section 4) to familiarize the reader to the problem setting we consider.

   Through different values of “N,” the total amount of training data, we wanted to demonstrate that Schedule Deviation is present even when the model is given access to different quantities of training data at the “low-density” regions–i.e. the effect depends on the relative density rather than the total amount of training data and so we conjecture that Schedule Deviation cannot be ameliorated simply by scaling the amount of training data available. Note that we also provide ablations over the training data for Fashion-MNIST and the Maze Trajectories datasets in our experimental appendix.

3. Prior Work:

   We will definitely expand on the prior work section. In particular, we will include significantly more details as to how our metric relates to work in unconditional models and how previously proposed metrics are either inappropriate for conditional models or otherwise differ from our proposed approach.

   [Bradley 2024] observes that classifier free guidance does not yield a denoiser, although they have no methodology for quantifying the degree of this divergence. In our evaluation setup there is no classifier-free guidance–only a single conditional diffusion model. We use the observation of Bradley et al. that interpolating scores yields non-denoising probability paths to leverage our theoretical results in Section 4 to explain Schedule Deviation. We agree that this part of the prior work was not well-phrased and will better explain the relation to Bradley et. al. in our subsequent revision.

4. Predicting OT Distance:

   Both DDPM/DDIM produce “reasonable” looking samples across the conditioning space. We conjecture that Schedule Deviation is predictive of the OT distance as DDIM/DDPM leverage the assumption that the flow is a denoiser differently in their sampling procedures. This is immaterial when the neural network well-approximates a denoiser (low Schedule Deviation) but can lead to divergence with high Schedule Deviation, which is precisely what we observe.

5. Reducing SD: We take no stance as to whether SD is “good” or “bad.” However, the widespread use of classifier-free guidance to improve generation quality suggests that any Schedule Deviation arising from “self-guidance” may in fact be an excellent inductive bias for models, in which case reducing Schedule Deviation may be detrimental. As we note in Remark 3.1, Schedule Deviation is not directly related to generation fidelity but rather measures the self-consistency of the reverse process.

   Methods such as [Daras 2023] try to induce more denoiser-like behavior by augmenting with samples drawn from the reverse process. Augmenting with an entirely new, much larger, dataset using a pretrained model eliminates the interpolation ambiguity and specifies the correct interpolation. Whether this is beneficial likely depends on the inductive biases of both the model and the sampler used in this data re-generation process. This effectively would “fix” the interpolation to either the distribution generated by DDIM or DDPM.

   Note that our metric of schedule deviation is different from the analogous quantities used in [Daras] or [Scarvelis] in that (1) we run a full reverse process and then re-noise our samples, whereas [Daras] uses only reverse process samples and (2) we more directly measure how the evolved density deviates from the ideal denoiser. It is possible to deviate from the ideal denoiser by adding “curl” components that do not change the probability path. By incorporating the divergence in our metric, we are agnostic to such changes.

   We will expand on the above in the prior work section.

6. Regarding Architectures:

   It would be quite surprising if different architectures interpolated the same way, as we observe empirical differences in the sample quality for different architectures. However both U-Nets and MLPs exhibited schedule deviation. Our paper conjectures this is because “reasonable” smooth interpolations between different denoisers are fundamentally non-denoising. Therefore any architecture which does not exhibit Schedule Deviation would likely need fundamentally very different and seemingly "unnatural" inductive biases compared to the architectures we consider here. We leave an in-depth exploration of the Schedule Deviation of different architectures to future work but believe it is of great interest, both theoretically and empirically.

[Daras 2023] Daras, G., Dagan, Y., Dimakis, A., & Daskalakis, C. (2023). Consistent diffusion models: Mitigating sampling drift by learning to be consistent. Advances in Neural Information Processing Systems, 36, 42038-42063.

[Scarvelis 2023] Scarvelis, C., Borde, H. S. D. O., & Solomon, J. (2023). Closed-form diffusion models. arXiv preprint arXiv:2310.12395.

[Bradley 2024] Bradley, A., & Nakkiran, P. (2024). Classifier-free guidance is a predictor-corrector. arXiv preprint arXiv:2408.09000.