Conditional Trajectories in Diffusion Models - Modeling Galaxy Evolution from Redshift

• Writing: The writing of this paper needs improvement.
  • The paper should include a paragraph that gives a brief overview of DDPM including the relevant notation. Ideally a paper should be self-contained. Only readers who are familiar with the notation in DDPM can follow this paper currently. Define the symbols for Beta Start and Beta End (Line 155) in the overview of DDPM.
  • Line 107: Define the filters, and perhaps include some example galaxy images at different redshift values + these filters as well in the appendix, so that the readers get an idea of what the training data looks like.
  • Define the metrics used to measure quality of generated galaxy images and mention why it is relevant in the Appendix (as opposed to mentioning these metrics in passing in Line 177). Also, indicate if these are ordered metrics - i.e. are higher values better or vice versa for some these. Further, indicate reliability of these metrics and discuss consequences of high variance seen for these metrics in generated trajectory (Line 414, Fig 9).
• The primary contribution this paper is the method to generate trajectory of galaxy evolution at different redshift values. It is centered on the assumption that $KL(p(X|z) || p(X|z + \Delta z)) \rightarrow 0$ as $\Delta z \rightarrow 0$. However, in practice, $\Delta z$ is not small. For instance, a redshift value of $z=1$ is not close to 0. In fact, the step size will increase as trajectory increases. Therefore, I’m unsure about the validity of assumption, especially because the paper also notes that the accumulated error of the generated images increases along the trajectories. Further, the idea of conditional DDPM, where conditioning is on redshift values is not completely novel too, as this was previously explored in Li et al. 2024.
• I would argue that the proof in A.1.2. needs more rigor. It currently bounds the error in moving from $X_{t-1}$ to $X_t$. We also need to bound how the error accumulates along the trajectory (with a uniform bound). Further, given that this is continuous trajectory, the evolution of $X_t$ according to diffusion SDE needs to be accounted in the error calculation. The proposed algorithm runs a complete forward diffusion and backward diffusion at each z, therefore additional error will be accumulated due to the dynamics of diffusion SDE and it needs to be accounted.

• Insufficient Citations and Background Discussion : The paper’s background and related work sections could be strengthened by including foundational references and prior works. For instance, the original paper on diffusion models [1] is missing. [2] addresses similar challenges of continuous conditional generation, (please add in line 43).

• Lack of Novelty in Conditioning and Training Techniques: The contributions, while valuable, are somewhat limited, as they primarily reiterate the standard goals of diffusion models in modeling data distributions, whether conditional or unconditional. The use of Exponential Moving Average (EMA) during training, while effective, is a widely adopted optimization technique that does not add significant novelty to the approach. To enhance originality, the authors could explore alternatives to improve the conditioning signal.

[1] Dickstein et al., Deep unsupervised learning using nonequilibrium thermodynamics

[2] Ding et.a., CCDM: Continuous Conditional Diffusion Models for Image Generation. 2024

• “Continuous trajectory reconstruction” is a little bit misnomer. From what I understand of the proof in A.1.1, A.1.2, I can see that you can construct a smooth trajectory corresponding to changing $z$ values from an initial image, but I don’t think it’s reconstruction of any particular trajectory. The proof doesn’t show the construction have to correspond to any physical trajectory other than it’s smooth. “Reconstruction” sounds like it’s reconstructing a physical trajectory of evolution, which could be misleading to uncareful readers. The authors should consider change "reconstruction" to "construction" throughout, unless they can provide additional justification for using "reconstruction".
  • If the authors do want to claim “reconstruction”, another proof may be necessary: like given what assumption about the distribution of trajectories occurring physically, your construction will correspond to the “true” one. If not, the authors may want to tune down the phrasing slightly.
• Current sampling algorithm for continuous trajectory is smooth in deed, but very much heuristic. But what exactly is it sampling from? More concretely can you say sth. about what distribution is $X^{z+\Delta z}$ sampling from using your sampling algorithm? A more precise characterization will make the paper much stronger.
• In figure 6, 7, right panel, could you explain what exactly is being plotted on the y-axis? specifically how the tensor $\mu$ is converted to a scalar value, and why are negative values possible if it's the L2 norm of the gradient.
• Minor: Figure 6 is not very legible, authors can consider change the alpha value of the lines.
• The conditional mechanism seems a bit ad hoc and not thoroughly discussed. (See questions)
• A.1.1, A.1.2 the proof could be better structured with the proposition / theorem more clearly stated at front as a result.

• Algorithm 1 is not consistent with the results. From my understanding of the provided code (perturb_redshift_iterative function), it seems that authors are starting with $X_T^{s+(n-1)\Delta z}=X^{s+(n-1)\Delta z}$. This initialization if out of distribution for the backward process. If this is indeed what the authors are doing, it would explain structural similarities between images from the same trajectory (but it does not make sense from an ML/generative modelling point of view).
• In scientific applications, providing estimates for sources of errors is crucial. Appart from the theory part, authors do not measure (or calibrate) the impact of the Huber loss in the denoising score matching objective (which is bound to induce some bias, especially at large noise level/large timesteps).
• No comparison with existing baselines (even with other generative model architectures like GANs that authors mentionned, or with discretized redshift and diffusion models)
• Conditionning on a continuous variable lacks novelty. Authors mention that "few studies have explored the behaviour of thses models when the conditionning variable is continuous". They do not provide any reference. For example in audio we have Audio Generation with Multiple Conditional Diffusion Model, Guo et al. AAAI 2024. For text to images generation: text tokens are usually encoded into a conditionnal vector (albeit the conditionning is different here). In scientific applications, diffusion models are often conditionned on physical paremeters (like redshift in this case).