Towards a Golden Classifier-Free Guidance Path via Foresight Fixed Point Iterations

Thank you for the valuable feedback, here's our response to the concerned problems.

---

Quality W1/W2: FID, Diversity Metrics, and Class-Conditional Generation

A1: FSG improves sample quality and diversity, as demonstrated by the FID and Vendi scores in Tab. 1 below.

We conducted experiments on the ImageNet 256×256 conditional generation task, generating 1K images per class (totaling 50K images). We report FID and Vendi at NFE=25/50. The results show:

1. FSG achieves better FID than baselines in class-conditional generation, and even slightly improves diversity.
2. CFG/CFG++$\times 2$ (NFE=50) also benefit from increased fixed-point iteration steps compared to CFG/CFG++ (NFE), improving both generation quality and diversity, indicating that fixed-point iterations do not cause mode collapse.

We hypothesize that this is because fixed-point iterations are performed locally in the neighborhood of $x_t$, preserving randomness. Intuitively, the initial noise for $x_t$ is $\epsilon$, and fixed-point iteration finds a better $\epsilon'$ in the vicinity of $\epsilon$, enhancing generation quality without sacrificing the inherent randomness of $\epsilon$.

We will provide detailed experimental settings and results in the Appendix. Tab. 1: FID and Vendi scores for different methods on ImageNet 256×256 conditional generation (lower FID and higher Vendi are better).

---

Clarity W1: Improved Generation Performance When $f^c _ {t\to 0}(x_t)$ Aligns with $f^u _ {t\to 0}(x_t)$

A2: Thank you for your constructive feedback. We provide further evidence to support this statement.

First, we generate data points $x_t$ such that $f^u _ {t\to 0}(x_t) \approx f^c _ {t\to 0}(x_t)$. Specifically, we generate images $x^c$ conditioned on $c$, and obtain $x_t$ via DDIM inversion using unconditional noise $\epsilon^u(x_t)$, i.e., $x_t = f _ {0\to t}^u (x^c)$. These $x_t$ can be regarded as consistent conditionally / unconditionally generation on $c$, i.e., $x^c \approx f^u _ {t\to 0}(x_t) \approx f^c _ {t\to 0}(x_t)$. We then randomly sample a $c' \ne c$ from prompts except $c$, so $x^c \approx f^u _ {t\to 0}(x_t) \ne f^{c'} _ {t\to 0}(x_t)$.

Tab. 2: Generation performance of FSG from $x_t$ under matched ($c$) and mismatched ($c' \ne c$) conditions.

---

It can be seen that when $f^{c/u} _ {t\to 0}(x_t)$ is mismatched, both the alignment and quality of the generated images decrease significantly. Intuitively, the less $f^u _ {t\to 0}(x_t)$ matches the condition $c$, the more components in $x_t$ are unsuitable for condition $c$, making it harder for the diffusion model to correct these components within the $t\to 0$ interval.

We will add a section in the Appendix to introduce the golden path, along with detailed experimental settings and visualization results.

---

Clarity W2: Satisfaction of Manifold Constraints

A3: Here we clarify that satisfying the manifold constraint is meaningful; otherwise, score estimation outside the probability manifold may be inaccurate. In practice, we avoid violating the manifold assumption via empirical settings.

• As shown in Fig. 2(b) (main text), the norm of $\Delta\epsilon (x_t):=\epsilon^c(x_t)-\epsilon^u(x_t)$ in CFG and CFG++ updates is relatively small. Therefore, as long as the guidance strength $w$ is not set too large, $x_t+w\xi_t\Delta\epsilon (x_t)$ can still be regarded as remaining on the probability manifold $P_t(x_t)$.

• For forward/backward operators, we consider that a suitable ODE/SDE sampler $f_{a\to b}$ can sample $x_b\in \mathcal{M}_b$ from $x_a\in \mathcal{M}_a$, thus satisfying the manifold constraint.

Our paper follows these empirical settings and does not focus on strict satisfaction of the manifold constraint. For rigor, we will remove the strong constraint $s.t. x_t \in \mathcal{M}_t$ from the formulas in the main text and instead discuss it in the text.

---

Clarity W3: $n_{t\to t+1}(x_t)$ as an approximation of $f^u_{t\to t+1}(x_t)$

A4: The use of noise addition as a backward operator was proposed in [1], where the authors argue that $x_t$ is obtained from $x_{t+1}$ via conditional denoising, and that the injected conditional signals are not completely removed by the noise process $n _ {t\to t+1}$. Therefore, $f^{\gamma} _ {t+1 \to t} \circ n_{t\to t+1}(x_t)$ contains more conditional semantics compared to $x_t$.

In our view, $n _ {t\to t+1}$, like $f^u _ {t\to t+1}$, samples $x _ {t+1} \in \mathcal{M}_{t+1}$ from $x_t$ (differing only in whether the sampler is stochastic or deterministic). However, there is still a gap between $n _ {t\to t+1}$ and $f^u _ {t\to t+1}$, which causes Resample requires more iterations to improve the generation quality.

---

Originality W1: FSG as a collection of engineering techniques

A5: Within our fixed-point iteration framework, FSG can be viewed as a composition of settings like the forward-backward operator and the CFG++ scheduler. However, the novelty of our paper lies in proposing a unified fixed-point iteration framework that enables the engineering-composable design of guidance algorithms. The effectiveness of the CFG/CFG++$\times N$ instances constructed under our fixed-point iteration framework has also been experimentally validated. We anticipate that the fixed-point iteration framework will provide deeper insights for future guidance design.

---

Reference

[1] Repaint: Inpainting using denoising diffusion probabilistic models.

Thank you for the detailed reply! While the authors' rebuttal has addressed my concerns regarding

• Quality [W1], [W2],
• Clarity [W1], [W2],

there are still remaining concerns with whether $n_{t \rightarrow t+1}$ a valid approximation of $f^{u_{(-1)}}\_{t+1 \rightarrow t}$, and I am still ambivalent about the originality of this work. Hence, I have raised the score to borderline accept.

Thank you for your valuable questions. Please find our responses below.

---

Q1: Wall clock time of FSG

A1: Our method (FSG) has identical wall-clock time to baselines (Tab. 1).

Tab. 1: Running time (seconds per image) for different methods at NFE=50/100/150.

For a fair comparison, all NFEs in the main text count a single backbone call. Although FSG employs longer-interval ODE operators, we reduce its computational time by using single-step ODE solver. The potential discretization error is acceptable, as fixed-point iterations occur mainly in the early denoising phase. As a result, FSG delivers superior performance while maintaining computational efficiency equivalent to the baselines.

---

Q2: Can we learn the interval schedule?

A2: Our interval scheduler can be set relatively easily and does not require extensive fine-tuning or learning.

For example, when NFE=100, we allocate the NFEs in a 3:2:1 ratio across three stages: $1 \rightarrow 0.67$ (early), $0.67 \rightarrow 0.33$ (mid), and $0.33 \rightarrow 0$ (late), with iterations uniformly distributed within each stage.

As shown in Tab. 2, as long as the overall ratio is not violated (e.g., 1:1:1 or 3:2:1), FSG is robust to hyperparameter choices: perturbing the intra-stage distribution or operator intervals leads to stable results. We conducted 5 trials with random perturbations to the position of points intra-stage ($\pm 1$) or interval lengths ($\times 70\\%-130\\%$), and observed stable performance.

Tab. 2: Performance under hyperparameter settings

---

Q3: FID Metric and Mode Collapse

A3: FSG improves sample quality and diversity, as demonstrated by the FID and Vendi scores in Tab. 3 below.

Since FID is challenging to use for large-scale text-to-image models, we conducted experiments on the ImageNet 256×256 conditional generation task, generating 1K images per class (totaling 50K images). We report FID and Vendi (diversity metric)[1] at NFE=25/50. The results show:

1. FSG achieves better FID than baselines in class-conditional generation, and even slightly improves diversity.
2. CFG/CFG++$\times 2$ (NFE=50) also benefit from increased fixed-point iteration steps compared to CFG/CFG++ (NFE), improving both generation quality and diversity, indicating that fixed-point iterations do not cause mode collapse.

We hypothesize that this is because fixed-point iterations are performed locally in the neighborhood of $x_t$, preserving randomness. Intuitively, the initial noise for $x_t$ is $\epsilon$, and fixed-point iteration finds a better $\epsilon'$ in the vicinity of $\epsilon$, enhancing generation quality without sacrificing the inherent randomness of $\epsilon$.

We will provide detailed experimental settings and results in the Appendix.

Tab. 3: FID and Vendi scores for different methods on ImageNet 256×256 conditional generation (lower FID and higher Vendi are better).

---

Q4: Convergence rate in synthetic settings

A4: FSG accelerates convergence under the Gaussian mixture setting.

We simulated the convergence rates of FSG and CFG under a Gaussian mixture setting. Following [2], the target distribution is a Gaussian mixture $\mathcal{N}(\pm \mathbf{m}, \sigma^2I)$ with two equally weighted components ($\mathbf{m}\in \mathbb{R}^d, \lVert \mathbf{m} \rVert_2^2=d$). The conditional sampling target is the component with mean $+\mathbf{m}$. Assuming the score is estimated accurately, [2] shows that CFG achieves faster convergence than using only the conditional prediction $\epsilon^c(x_t)$, as quantified by $\mathbb{E}(x_t)\cdot\mathbf{m}/\sqrt{d}$.

Building on this, we simulated FSG with an interval of 0.1 at $t=0,0.1,...,0.9$, and observed significantly faster convergence. The results indicate that FSG provides stronger and more effective guidance, especially in the early stages.

Tab. 4: Convergence speed under the Gaussian mixture setting ($\times 10^{-2}$), using the same metric as in [1] (higher values indicate better performance).

We will provide additional details and visualizations of the simulation experiments in the Appendix.

---

Q5: Qualitative failure cases with a discussion on limitations

A5: When excessive iterations are performed, sample quality may degrade due to violation of the manifold constraint $x_t \in \mathcal{M}_t$. We will include qualitative failure cases and discuss these limitations in the Appendix.

---

Reference

[1] The Vendi Score: A Diversity Evaluation Metric for Machine Learning

[2] Classifier-Free Guidance: From High-Dimensional Analysis to Generalized Guidance Forms

Thank you for your valuable suggestions regarding our work. Please find our responses below.

---

Q1: Specific design choices for the foresight guidance: fixed point operator and scheduler.

A1:

1. In FSG, we select the forward-backward operator with a larger interval $\Delta t$ because it exhibits a smaller contraction rate, thereby accelerating the convergence of the fixed-point algorithm. We empirically measure local contraction rates for operators $F$ under $\ell_2$ norm: $\hat{r}=\mathbb{E}_{x_0\sim p(x)} \frac{\lVert F(x_t)-F(x_t') \rVert_2^2}{\lVert x_t-x_t' \rVert_2^2}$, where $x_t,x_t'$ are obtained by mixing different noises with $x_0$ (See Tab. 1).

Tab. 1: Empirical contraction rates $\hat{r}$ for different operators across denoising timesteps ($t\in[0,1],dt=0.05$)

We observe that, compared to the linear fixed-point operator ($\text{id}-w_t\Delta\epsilon$), the contraction rate of the forward-backward operator decreases as the interval $\Delta t$ increases. Therefore, to balance operator complexity and convergence speed, we choose the forward-backward operator $f^u _ {t - \Delta t \to t}\circ f^{\gamma} _ {t\to t - \Delta t}$ with a larger $\Delta t$.

2. In FSG, we adopt the CFG++ scheduler because it provides more stable guidance strength during the early stages.

The schedulers for CFG and CFG++ are given by $\xi_t = \sqrt{1-\bar{\alpha} _ t} - \sqrt{\alpha_t - \bar{\alpha} _ t}$ and $\hat{\xi}_t = \sqrt{1-\bar{\alpha} _ t}$, respectively. Using the $\alpha_t$ setting in DDIM as an example, the derivatives of $\xi_t$ and $\tilde{\xi}_t$ at different timesteps are shown in Tab. 2:

Tab. 2: Derivatives of $\xi_t$ and $\tilde{\xi}_t$ at different timesteps

It can be observed that, in the early stages where the guidance has a greater impact on generation ($t \in [0.67, 1]$), the guidance strength provided by $\xi_t$ decays rapidly, which limits the generation quality.

---

Q2: Enhancing the comprehensibility of Tab.1 (main text)

A2: Thank you for your valuable suggestions. To improve clarity, we will expand the description of Tab.1 in the main text. This will include complete definitions of all abbreviations in the caption (e.g., $\text{id}$: identity mapping, $\Delta\epsilon(x_t):=\epsilon^c(x_t)-\epsilon^u(x_t)$), as well as clear explanations for "Interval" and "Iter."

Specifically, the Interval $a\to b$ indicates that the method aims to satisfy $f^c_{a\to b} = f^u_{a\to b}$, while Iter describes whether the original version of the method supports increasing the number of fixed-point iterations (Iter=1: not supported; Iter=N: supported). The default versions of CFG, CFG++, and Z-sampling do not mention increasing iterations to improve performance. The author of Resampling noted that iterations can be increased, but did not provide an explanation from the fixed-point perspective.

In contrast, our fixed-point iteration framework naturally enables this enhancement, and we validate the performance of CFG/CFG++ with additional iterations as CFG/CFG++ ($\times 2/3$) in Tab.2 (main text). These clarifications will be incorporated into the main text as well.

Thank you for the valuable feedback, here's our response to the concerned problems.

---

Q1: Computational Overhead
A1: Our method (FSG) has identical wall-clock time to baselines (Tab. 1).

Tab. 1: Running time (seconds per image) for different methods at NFE=50/100/150.

For a fair comparison, all NFEs in the main text correspond to a single backbone call. Although FSG employs longer-interval ODE operators, we reduce its computational time by using single-step ODE solver. As fixed-point iterations occur mainly in the early denoising phase, the potential discretization error is acceptable. As a result, FSG delivers superior performance while maintaining computational efficiency equivalent to the baselines.

---

Q2: Golden Path Proxy

A2: Thank you for your questions, which help improve the clarity of our work. We clarify that while we use $\lVert \epsilon^c(x_t) - \epsilon^u(x_t) \rVert_2^2$ for visualization in Fig. 2 (main text), the golden path proxy actually minimized in FSG is $f^c _ {t-\Delta t}(x_t) = f^u _ {t-\Delta t}(x_t)$ (see line 180 in the main text).

The rationale for this proxy is as follows: (1) directly targeting $f^c _ {t\to 0}(x_t) = f^u _ {t \to 0}(x_t)$ is suboptimal for optimizing $x_t$, as the long-range generation process causes the dependence of $f^{c/u} _ {t\to 0}(x_t)$ on $x_t$ to diminish. (2) Minimizing $\lVert \epsilon^c(x_t) - \epsilon^u(x_t) \rVert_2^2$ only regularizes the trajectory locally around $x_t$. Therefore, FSG minimizes $f^c _ {t-\Delta t}(x_t) = f^u _ {t-\Delta t}(x_t)$, which balances these two extremes by selecting an appropriate interval length $\Delta t$.

For visualization, we use $\lVert \epsilon^c(x_t) - \epsilon^u(x_t) \rVert_2^2$ because it is equivalent to the expectation of clean image gap $\lVert \mathbb{E}^c (x_0 \mid x_t) - \mathbb{E}^u(x_0 \mid x_t) \rVert_2^2$, where $\mathbb{E}^{c/u} (x_0 \mid x_t)$ denotes the conditional/unconditional expectation of the clean image $x_0$ given $x_t$. This expectation approximates $f^{c/u} _ {t\to 0}(x_t)$, with the equivalence following from Tweedie’s formula: $\mathbb{E}^{c/u} (x_0 \mid x_t) = [x_t - \sqrt{1-\bar{\alpha} _ t} \epsilon^{c/u}(x_t)] / \sqrt{\bar{\alpha} _ t}$.

Empirically, FSG-guided $f_{t\to 0}^{c/u}(x_t)$ also exhibits stronger semantic similarity, indicating improved full-path consistency. To further support this, we will provide additional visualization examples in the Appendix and revise our paper to enhance clarity.

---

Q3: Contraction Assumption

A3: We empirically measure local contraction rates for operators $F$ under $\ell_2$ norm: $\hat{r}=\mathbb{E}_{x_0\sim p(x)} \frac{\lVert F(x_t)-F(x_t') \rVert_2^2}{\lVert x_t-x_t' \rVert_2^2}$, where $x_t,x_t'$ are obtained by mixing different noises with $x_0$.

Tab.2 Empirical contraction rates $\hat{r}$ for different operators across denoising timesteps ($t\in[0,1],dt=0.05$)

• Notably, long-interval operators (e.g., $f^u _ {0\to t}\circ f^{\gamma} _ {t\to 0}$) demonstrate lower contraction rates ($\hat{r} < 1$) compared to short-interval operators such as Z-sampling or Resample. To balance convergence speed and operator complexity, we select an intermediate interval $\Delta t$ ($dt < \Delta t < t$) for the operator $f^u _ {t-\Delta t\to t}\circ f^{\gamma} _ {t\to t-\Delta t}$.

• The contraction rate of an operator depends on the choice of metric. While operators in CFG/CFG++ are non-contracting under the $\ell_2$ norm, they may exhibit contraction in other carefully designed metrics. Empirically, as shown in Tab. 2 (main text), these methods also benefit from additional iterations, supporting the fixed-point framework.

---

Q4: Hyperparameter Selection

A4: Our scheduler assigns more fixed-point iterations to the early denoising stage. Specifically, we distribute NFE in a 3:2:1 ratio across the stages $1 \to 0.67$ (early), $0.67 \to 0.33$ (mid), and $0.33 \to 0$ (late), with iterations uniformly distributed within each stage.

Alternative ratios (such as 1:1:1 or 1:2:3) lead to degraded performance (see Tab. 3). Under the 3:2:1 setting, FSG demonstrates robustness to hyperparameters: perturbing the intra-stage distribution or operator intervals yields stable results. We conducted 5 trials, randomly perturbing the position of points intra-stage ($\pm 1$) or the interval length ($\times 70\\%-130\\%$) on our chosen hyperparameters.

Tab. 3: Performance under different hyperparameter settings