Thank you so much for acknowledging our contribution and the constructive feedback. Below we address each concern raised.

---

Q1. Where marginal distribution scaling is mentioned in CFG paper?

A1. Thank you for the great question. As a quick recap, Section II of our paper explains the theoretical pitfall of guidance. We begin by examining the case of scaling only the terminal marginal distribution (Eq. (5)) and then generalize to scaling all marginal distributions (Eq. (8)). The latter is a stricter goal that subsumes the former. We show that neither is theoretically well-justified.

Strictly speaking, Ho & Salimans (2022) state that classifier guidance (CG) leads to sampling from scaled versions of all marginal distributions—see the second equation under Algorithm 1 on page 4. This aligns with our Eq. (8)–(9), where their reward term $p_\theta(c | z_\lambda)^w$ corresponds to our notation $R_t(x_t, y) = p_\phi(y | x_t)^w$ in Eq. (9). They also express the CFG reward using $p^i(c | z_\lambda)$ in the second-to-last paragraph before Section 4. Thus, the CG and CFG reward forms are indeed explictly stated there.

For presentation clarity, we choose to start with the simpler case of scaling only the terminal marginal (Eq. (5)), but we acknowledge that Eq. (8) is the original formulation in Ho & Salimans (2022). We will revise the text and footnote accordingly to make our claim more accurate.

---

Q2. The qualitative visualizations in Fig. 8-10.

A2. Thanks for the great question. To elaborate: in Figure 8, the knight’s gesture while holding the sword appears unnatural, and the horse’s legs are blurry in the baseline images. In Figure 9, there are black artifacts in the sky (not birds), and the lower part of the tree suffers from low contrast in the baseline images. In Figure 10, the dog’s paws are unnatural, and the sunglasses are deformed in the baseline images.

We agree with the reviewer that these differences might be subtle. For additional context, it might be helpful to consult qualitative results reported in other works to put our qualitative results in context --- see, for example, Figure 6-7 of [1], Figure 4 of [2], Figure 1 of [3], and [4].

To better address the concern, we have followed the suggestions and added multiple samples with the same method for a given prompt. These can be found in File 1 and File 2 at this anonymous link.

Please note that due to OpenReview rebuttal format constraints, we are unable to include figures directly in this response. However, anonymous links are permitted under ICML guidelines, and we will include these visualizations in the revised manuscript.

---

Q3. Apply REG to flow-matching based diffusion models.

A3. Thank you for the insightful question. We agree that evaluating REG in the context of flow-matching models would further enhance the impact of our work.

Due to time and resource constraints—particularly the large model size (SDv3 has over 2 billion parameters) and its lengthy inference time—we were unable to run experiments on SD v3. However, we conduct extra experiments on SD v2.1, a velocity-parameterized diffusion model. This setup was mutually requested by other reviewers and, we believe, is also relevant to the current question. Namely, while velocity-parameterized models are not identical to flow-matching approaches, they are closely related. In fact, the distinction between them can often be seen as a subtle difference in the noise schedule [5]. We perform a grid search over guidance weights for different methods with SD v2.1 and report the best FID $\downarrow$ and CLIP $\uparrow$ scores for each method in the table below.

We will perform thorough evaluations on REG for flow-matching based generative models and report them in the final version if the paper is accepted.

---

References

[1] Tuomas Kynkäänniemi et al., 'Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models,' Neurips, 2024.

[2] Tero Karras et al., 'Guiding a Diffusion Model with a Bad Version of Itself,' Neurips, 2024.

[3] Tianwei Yin et al., 'One-step Diffusion with Distribution Matching Distillation,' CVPR, 2024.

[4] Xi Wang et al., 'Analysis of Classifier-Free Guidance Weight Schedulers,' TMLR, 2024.

[5] Ruiqi Gao et al., 'Diffusion Meets Flow Matching: Two Sides of the Same Coin', https://diffusionflow.github.io/.

Thank you so much for acknowledging our contribution and the constructive feedback.

---

Q1. Runtime and memory cost.

A1. Thanks for the great question. The tables below summarize runtime and peak memory usage of CFG and REG on a single NVIDIA A40 GPU. Runtime is reported using example batch sizes, while memory is measured with batch size 1 to isolate per-image cost. Since REG introduces one extra gradient computation on top of vanilla CFG, a moderate increase in runtime and memory usage is expected. Similar inference-time gradient calculations have also been explored in Universal Guidance (Arpit Bansal et al., 2024), albeit in a different context.

We emphasize that our main contribution lies in correcting the CFG theory, and REG serves as an empirical validation. Its practical deployment depends on the specific application and acceptable overhead. We will include these tables in the updated paper.

---

Q2. Experiments on diverse model architectures, applications, and datasets.

A2. Thanks for the valuable feedback. To address architecture concern, we conduct extra experiments using SD-V2.1, a velocity-parametrized diffusion model; results can be found in A3 of Reveiwer BefU. Below is a summary of the models used, covering a wide range of settings.

Regarding applications and datasets, we respectfully clarify that our choices are consistent with current standards in the literature. For examples, DiT (William Peebles and Saining Xie, 2023) and EDM2 (Tero Karras et al., 2024) primarily conduct experiments on ImageNet. In addition to ImageNet, Interval Guidance (Kynkäänniemi et al., 2024) and Autoguidance (Tero Karras et al., 2024) perform qualitative experiments on text-to-image tasks using SD models. Due to limited time and resource constraints, we will explore additional datasets in future work.

---

Q3. Comparison with recent guidance enhancements.

A3. Thanks for the constructive suggestion. We respectfully note that the proposed REG method has already been compared with SOTA guidance techniques, such as Interval Guidance (Kynkäänniemi et al., 2024) and Autoguidance (Tero Karras et al., 2024), both of which are strong and recent baselines. As shown in Table I and Figure 7, REG is still able to improve upon these methods. We greatly appreciate suggestions from the reviewer on any specific missing guidance methods that we should compare to.

---

Q4. REG implementation details and open-source code.

A4. Thanks for the constructive remark. We will open source our code to ensure full reproducibility, and add implementation details in the updated paper.

---

Q5. Other comments: (i) Add intuitive explanation for marginal scaling in Section 3. (ii) Add a summary table of results and architectures in numerical result section. (iii) REG with different sampling schemes and different parametrizations.

A5. Thanks for the constructive remarks.

(i) We will update our manuscript accordingly.

(ii) We want to respectfully point out that Table 2 and 3 have already included all numerical results and Table 4 (in supplementary) have summarized model architectures.

(iii) We respectfully note that our experiments already cover a wide range of settings. Please refer to the summary table in A2 for details.

Thank you so much for acknowledging our contribution and the constructive feedback.

---

Q1. Runtime and memory cost.

A1. Thanks for the great question. The tables below summarize runtime and peak memory usage of CFG and REG on a single NVIDIA A40 GPU. Runtime is reported using example batch sizes, while memory is measured with batch size 1 to isolate per-image cost. Since REG introduces one extra gradient computation on top of vanilla CFG, a moderate increase in runtime and memory usage is expected. Similar inference-time gradient calculations have also been explored in Universal Guidance (Arpit Bansal et al., 2024), albeit in a different context.

We emphasize that our main contribution lies in correcting CFG theory, and REG serves as an empirical validation. Its practical deployment depends on the specific application and acceptable overhead. We will include these tables in the updated paper.

---

Q2. Extra visualization results.

A2. Thanks for the constructive remark. We respectfully point out that Figures 1, 2, 5, 8, 9, and 10 already provide qualitative visualizations for synthetic 1D/2D cases and real benchmarks. To address this concern, extra visualizations (e.g., standard 2D toy dataset, text-to-image and class-conditioned generation) are performed and added in this anonymous link.

Please note that due to OpenReview rebuttal format constraints, we are unable to include figures directly in this response. However, anonymous links are permitted under ICML guidelines, and we will include these visualizations in the revised manuscript.

---

Q3. Standard public 1D and 2D datasets and 2D generation accuracy.

A3. Thanks for the helpful feedback. Our 1D setup follows standards from prior works, such as Interval Guidance (Kynkäänniemi et al., 2024) and Autoguidance (Karras et al., 2024). For 2D, we use custom shapes with fine-grained structures, more challenging than standard 2D toy datasets like “two moons” or “Swiss rolls.” Extra results on standard 2D datasets are performed and can be accessed via this anonymous link (anonymous links permitted by ICML guidelines).

About generation accuracy, we clarify that Figure 2(b) appears reasonably well when compared to the target in Figure 2(a). We acknowledge that the results may not be perfect, as we use a relatively small diffusion model in order to compute the golden gradient $\nabla \log E_t$ efficiently. We also clarify that the metric in Table 1 indicates how often REG achieves lower error compared to no REG. A value x%>50% suggests consistent improvement.

---

Q4. Cosine and linear CFG perform worse than vanilla in SD models and DiT-XL/2.

A4. Thanks for the great question. We first want to clarify that our experiments are designed to verify whether REG can enhance a given guidance method. This argument has been validated by the results shown in Table 2 and 3. Our aim is not to compare “cosine + REG” vs. “vanilla CFG,” since such a comparison conflates the base method’s performance with the effect of REG. Hence, we believe the essential question here is that cosine and linear CFG perform worse than vanilla CFG in SD models and DiT-XL/2 (they do perform better in EDM2).

This trend is consistent with results shown in related works, such as the analysis of CFG weight schedulers in Xi Wang et al., TMLR 2024 --- Both our Figure 4 (right) and their Figure 7(c) show that vanilla CFG achieves the lowest FID, linear CFG shifts slightly upward, and cosine CFG shifts it further upward in CLIP-FID space.

Two factors likely contribute: (i) all these methods are heuristic, and their effectiveness can vary across model architectures and datasets. (ii) linear and cosine CFG require tuning two hyperparameters, while vanilla CFG uses only one—making linear and cosine CFG more expensive and less robust to optimize.

We will include this clarification in the updated paper.

Thank you so much for acknowledging our contribution and the constructive feedback.

---

Q1. Reference [1]: Krunoslav Pavasovic et al., arXiv 2025.

A1. Thanks for bringing [1] to our attention. We are aware of this work and will include it in our references. As the reviewer kindly noted, this paper was published on arXiv on February 11th, after the ICML submission deadline. [1] shows that while CFG may reduce diversity in low-dimensional settings, it becomes effective in high-dimensional regimes due to a "blessing of dimensionality." The authors identify two phases: an early phase where CFG aids class selection, and a later phase where it has minimal impact. They propose non-linear CFG variants that deactivate in the second phase, improving quality and diversity without extra computational cost.

---

Q2. Include samples with the golden guidance in Figure 2.

A2. Thanks for the great question. We respectfully clarify that Figure 2(a) already displays the golden/target samples that we aim to generate, and Figure 2(d) presents the golden guidance $\nabla \log E_t$. To the best of our understanding, these are all the golden cases available. We are more than happy to include any missing visualizations to address this concern.

---

Q3. Extra results on diffusion models using velocity-parametrization.

A3. Thanks for the constructive feedback. We conduct extra experiments on SD-V2.1, a velocity-parametrized text-to-image diffusion model. Due to time and resource constraints, we perform a grid search over guidance weights and report the best FID $\downarrow$ and CLIP $\uparrow$ scores for each method in the table below. A more thorough evaluation (e.g., a full FID-CLIP curve) will be included in the final version if accepted.

---

Q4. Diagonal Jacobian matrix assumption.

A4. Thanks for the constructive remark. We agree that a diagonal Jacobian is more precise. We originally refer to a diagonally dominant Jacobian for a writing reason. From Eq. (22) to Eq. (21), three approximations are introduced, as detailed in Lines 220–224 on the right side of the paper, with the Jacobian assumption being the third. The first two approximations are essential—(2) is self-explanatory, and (1) is explained in our response to Q5. Given these, the transition from Eq. (22) to Eq. (21) is already approximate, regardless of whether the Jacobian is diagonal or diagonally dominant.

We will revise the text to refer to a diagonal Jacobian where appropriate.

---

Q5. Apply Vector-Jacobian-Product (VJP) in Eq. (22).

A5. Thanks for the great question. In short, the application of VJP requires that the function $\log R_0(\hat{x}_0, y)$ be differentiable with respect to $\hat{x}_0$. However, in current CFG-like frameworks, $R_0(\cdot, y)$ is not arbitrary—it is specifically defined as shown in the second line of Eq. (6). This definition renders $\log R_0(\cdot, y)$ non-differentiable with respect to its first argument (or very consuming to evaluate). Consequently, VJP is not applicable in this case.

Note that to address it, we use chain rule and approximate $\nabla_{\hat{x}_0} \log R_0(\hat{x}_0, y)$ with $\nabla{x_t} \log R_t(x_t, y)$, which can be further simplified via Eq. (7) and (9).

---

Q6. Runtime and memory cost.

A6. Thanks for the great question. The tables below summarize runtime and peak memory usage of CFG and REG on a single NVIDIA A40 GPU. Runtime is reported using example batch sizes, while memory is measured with batch size 1 to isolate per-image cost. As expected, REG introduces minor overhead due to the extra gradient computation. We will add these tables in the updated paper.

# net(xt, y, t) is the trained conditional noise prediction network
uncond_pred = net(xt, None, t) # unconditional noise prediction without labels
cond_pred = net(xt, y, t) # conditional noise prediction with labels
pred = cond_pred + w * jvp(net, xt, cond_pred - uncond_pred)