SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity

We thank Reviewer 4Z6k for appreciating our theoretical reevaluation of SDS and VSD. Regarding your questions, please see our detailed responses below:

1. Why a lower variance during optimization is helpful for the final quality?

In optimization theory, variance plays a crucial role in determining the convergence rate of SGD algorithms (refer to Theorem 5.3 in [1]). With a finite number of optimization steps and a standard learning rate, maintaining low variance is pivotal to ensure convergence. Training with noisy gradients introduces high instability, potentially resulting in suboptimal solution or even divergence, as illustrated in Fig. 4(a).

[1] Garrigos & Gower, Handbook of Convergence Theorems for (Stochastic) Gradient Methods

2. Quantitative results.

We present a qualitative comparison among DreamFusion, ProlificDreamer, and SteinDreamer across all showcased scenes, utilizing the CLIP score as our metric.

Our observations indicate that SteinDreamer consistently outperforms all other methods, which improves CLIP score by ~0.5 over ProlificDreamer. This superior result suggests our flexible control variates are more effective than the one adopted by ProlificDreamer.

3. The relationship between the baseline function and the final performance.

Various baseline functions can yield differing correlations between the constructed control variate and the target random variable for estimation. As outlined in Section 2.2, it is advisable to employ a baseline function that exhibits a stronger correlation with the 2D diffusion model to minimize variance. Typically, we notice that baseline functions with a more robust prior tend to demonstrate higher correlations. From another perspective, the baseline function $\phi$ can be regarded as a guidance introduced into the distillation process. Intuitively, enforcing priors and constraints on the gradient space can also stabilize the training process by regularizing the optimization trajectory. In our empirical design, the inclusion of geometric information expressed by a pre-trained MiDAS estimator is expected to result in superior variance reduction compared to SSD and VSD.

We appreciate Reviewer XeQE for recognizing our motivation and the clarity of our mathematical exposition. We have fixed all the typos and addressed your concerns as follows.

1. More experiment results.

We additionally showcase four qualitative results on scene generation in Appendix C.4. Aligning with our prior empirical observations, our method consistently produces sharp and visually coherent results.

2. DreamFusion results seem not converged.

Every qualitative result, including DreamFusion's, was obtained after 25k training steps, which follows the standard training configuration in the threestudio repository. To be more convincing, we train DreamFusion with the prompt "a car made out of sush" for an additional 10k steps, there were no observable improvements in the results. See our Appendix C.3 for more details.

3. Why does the variance of SteinDreamer gradually increase after 12k steps.

As depicted in Fig. 5, not all scenarios witness an increase in variance, as seen in examples like 'a blue tulip' and 'a plush dragon toy'. We emphasize that the optimization trajectory of score distillation exhibits high non-smoothness, leading to a typical fluctuation phenomenon in variance. Even in instances where variance appears to rise, its final value remains upper bounded, ensuring the convergence of our results. In contrast, other methods, such as ProlificDreamer with the prompt 'a plushy dragon toy', display potential variance explosion, evident in the training instability depicted in Fig. 4(a).

4. Compare convergence after more training steps.

We express our apologies for the mistake in Fig. 7, where the maximum of x-axis was incorrectly set to 100 instead of the intended 25k steps. This initial error occurred as we mistakenly used the indices of data points for the x-axis. We have rectified this issue in our revised version. It is important to note that 25k is the default setting suggested by threestudio, typically resulting in converged results with ProlificDreamer. To verify this claim, we train ProlificDreamer for an additional 10k steps. Both visual quality and the CLIP score exhibit negligible changes. See Appendix C.3 for more details.

We are grateful to Reviewer D8i1 for recognizing our motivation. We have conducted benchmarking of the wall-clock time and provided detailed per-point answers below.

1. Wall-clock time comparison between different methods.

Below, we list the average per-iteration wall-clock time for all methods, each benchmarked using the same GPU device with 10k training steps.

In summary, SteinDreamer exhibits comparable per-iteration speed to DreamFusion while significantly outperforming ProlificDreamer in terms of speed. The trainable component $\mu$ in SSD comprises only thousands of parameters, which minimally increases computational overhead and becomes much more efficient than tuning a LoRA in VSD. Notably, given that SSD can reach comparable visual quality in fewer steps, SteinDreamer achieves significant time savings for 3D score distillation.

2. Quantitative comparison with other methods.

We present a qualitative comparison among DreamFusion, ProlificDreamer, and SteinDreamer across all showcased scenes, using the CLIP score as our metric.

Our observations indicate that SteinDreamer consistently outperforms all other methods, demonstrating an approximate 0.5 improvement in CLIP distance over ProlificDreamer.

3. Missing citations.

Thank you for highlighting these two works. We have updated our citation to include mention of these references.

[1] Roeder et al. Sticking the landing: Simple, lower-variance gradient estimators for variational inference. NeurIPS 2017.

[2] Gorham et al. Measuring sample quality with Stein's method. NeurIPS, 2015.

We thank Reviewer HqJT for the time and effort in reviewing our paper. Per your questions, please see our response below:

1. Lack of novelty and idea coincides Kim et al.

Before submission, we were aware of Kim et al.'s work [1]. However, we respectfully disagree regarding the similarity of their method to ours. While both methods are grounded in Stein's method, the underlying principles significantly differ. In their approach [1], the SVGD-based update takes the form of the Stein discrepancy: $\max_{\phi \in \mathcal{F}} \mathbb{E}\_{x\sim q(x)} [\phi(x) \nabla \log p(x) + \nabla \phi(x)]$, where $\phi(x)$ is often interpreted as an update direction constrained by a function class $\mathcal{F}$ (RBF RKHS in [1]). In contrast, our update rule appends a zero-mean random variable via the Stein identity after the raw gradient of the KL divergence: $\mathbb{E}_{x \sim q(x)} [\nabla \log p(x) + \phi(x) \nabla \log q(x) + \nabla \phi(x)]$, where $\phi(x)$ typically represents a pre-defined baseline function. The potential rationale behind [1]'s approach to reducing variance lies in introducing the RBF kernel as a prior to constrain the solution space by modeling pairwise relations between data samples. Our approach to reducing variance is centered around constructing a more general control variate that correlates with the random variable of interest, featuring zero mean but variance reduction. We have cited [1] and included the above discussion in our revised submission (Sec. 4.3).

[1] Kim et al., Collaborative Score Distillation for Consistent Visual Editing. NeurIPS 2023.

2. Clarification on training steps and quantitative results.

The qualitative results presented in this study were generated using a default setting of 25k training steps, consistent with the threestudio repo's default configuration. An error was identified in the original Fig. 7, where the x-axis required scaling to 25k due to our oversight in using the number of data points as the x-axis reference. We have rectified this error in the updated Fig. 7. To verify 25k is sufficient for convergence, we extended the training of ProlificDreamer by an additional 10k steps. During this extension, the CLIP score ranged between 0.74 and 0.75. Furthermore, a quantitative comparison is provided below.

The notably superior performance suggests that our method not only enhances quality at the early stages but also improves overall quality with full training.

3. Correction on Janus results.

We apologize for the inclusion of an incorrect video demo in the previous supplementary material, an oversight that occurred during the manuscript preparation rush. The updated supplementary material now accurately demonstrates that our method can potentially mitigate the Janus problem, although it is not the primary goal of this paper.

4. The number of examples is limited.

To enhance our experiment results, we present an additional set of four qualitative results on scene generation in Appendix C.4. As previously noted, our method consistently delivers smooth and visually consistent outcomes. Their video demos can be found in our updated supplementary materials.