Two-Steps Diffusion Policy for Robotic Manipulation via Genetic Denoising

We thank the reviewer for their feedback. We are pleased that they find our method well-motivated and practical. We respond to their comments and questions below.

1. Inference runtime

(same answer as to reviewer fdQZ) We measured the step-wise overhead for GDP using a standard RTX 3080 GPU without specific optimization beyond calling the model batch-wise on the population.

The relatively low cost is due to the under-utilization of the GPU computation and memory abilities. As the model grows bigger, the population may be more constrained. We argue the following.

• In production, the observation embedding may represent a significant portion of the computational budget but since it can be shared across the population, its impact on the overhead is limited.
• Training is typically done with a batch size larger than one; thus, assuming training and inference are done on the same hardware, inference is likely to leave room for a non-trivial population.
• Fully-fused models, weight quantization, and other low-level optimization would allow for a larger batch size (hence a population) during inference.

These points will be discussed in a dedicated section in the appendix.

2 . Method Scalability

Our experiments include Robomimic and Adroit, which span low to medium action dimensionality, 9 and up to 30 DoF respectively. Results indicate that the method scales well and tends to be more beneficial in complex tasks, particularly those with more convoluted action distributions, even at higher prediction horizons. As noted in answer to reviewer fdQZ, despite the low action space dimensionality, our experiments with long horizons show the method is efficient for extrinsic distribution dimension comparable to CIFAR or ImageNet-64.

We have not tested the method on humanoid-level tasks (~60 DoF). Still, the demonstrated scalability up to 30 DoF and 200-step horizons suggests that the method would extend to such domains, at least for short horizons.

3. Real world experiment

We tested our method on a pick-and-place task using a Franka Emika Panda arm. All methods—DDPM with 100 steps, 5 steps, 2 steps (with adapted schedules and zero variance), and GDP—achieved a 100% success rate. Although this does not highlight performance differences, it demonstrates that our approach is robust and deployable on real hardware for such tasks. Results and experimental details will be included in the paper.

However, showcasing dynamic benefits from faster inference in real-world manipulation remains impractical for now. Such demonstrations would require a robotic platform capable of ultra-fast closed-loop control and streamlined integration between perception, model inference, and actuation, which are beyond our current resources.

Nevertheless, we argue that reducing inference cost provides real benefits:

• Enabling more reactive or frequent control cycles;
• Freeing GPU compute for parallel tasks (e.g., perception or planning);
• Allowing deeper models.
• Allowing deployment on less expensive hardware.

4. Mode Collapse

We agree that mode collapse may harm sample diversity, particularly in guided generation. We also acknowledge that guidance effectiveness typically declines as the number of denoising steps is reduced.

Our proposed mechanism enables a diversity trade-off: possibly lowering the diversity with $\gamma$ while compensating with a larger population. It is important to note that lowering $\gamma$ distorts the generated distribution, but it does not necessarily reduce mode coverage, just like increasing it does not necessarily increase mode coverage.

In future work, we intend to explore a more integrated use of guidance with genetic denoising. For instance, applying guidance before selection and then scoring candidates with a Stein metric may preserve distributional support while promoting goal satisfaction—even with few denoising steps.

5. Theoretical Contribution

The theoretical motivation for Stein score-based selection is detailed in our response to Reviewer UvdB, point 4.

We also see promising directions for further theoretical development. One avenue is to formalize the diversity gain induced by our population mechanism. We conjecture that in the limit of many denoising steps, population-based selection approximates the effect of increasing the noise level $\gamma$, thus promoting broader exploration. A formal proof of this claim is left for future work.

To the best of our knowledge, there is no theoretical description of the noise injection scale distortion in the current literature. Partial results are likely achievable in the coming week such as distribution support preservation using the line of argumentation of Pidstrigach's "Score-Based Generative Models Detect Manifolds".

6. Quality

We will conduct a thorough proofreading and will improve the overall writing quality and clarity of our paper.

We thank the reviewer for their comments. We appreciate that they find the topic interesting and our method well established. Below we address the points raised in the review.

1. Necessity of Clipping in Diffusion

Each DDPM denoising step predicts a fully denoised sample, which is then re-noised at a lower noise level. Clipping is applied only to this fully denoised sample prior to re-noising. Its purpose is twofold: to enforce bounded action constraints and to avoid numerical instabilities at high noise levels—particularly with schedules like cosine. From this perspective, clipping may be viewed as a form of universal guidance (Bansal et al. "Universal Guidance for Diffusion Models") with a "potential well" guidance.

While such clipping is helpful in standard DDPM settings, alternative noise parameterizations, such as the variance-exploding linear schedule used in EDM, are less prone to numerical instabilities. Notably, EDM avoids rescaling the original sample, ensuring that predictions remain within the convex hull of the data distribution. Similarly, flow-matching models inherently satisfy this constraint and do not require clipping.

Although we considered a clipping schedule (loosening the constraint over time), we ultimately abandoned it: clipping is already a heuristic justified primarily by empirical results, and further engineering on top of it lacked theoretical grounding. Instead, our approach offers a principled solution by improving alignment between the denoising trajectory and the data distribution, without compounding ad-hoc mechanisms.

---

2. Is a Stochastic Policy Still Necessary if γ = 0 Performs Best?

While the denoising process may become deterministic as γ → 0, the policy remains stochastic due to the random initialization from Gaussian noise. This setting mirrors standard DDIM and Flow Matching sampling: randomness exists only at the start, and is then followed by a deterministic process.

Moreover, stochasticity is essential during training, even if the final policy is effectively deterministic at inference time. Stochastic training encourages mode coverage, which helps prevent mode collapse—a critical concern in robotics. For example, in a 2D pick-and-place task with obstacles, multiple disjoint paths may exist. If the obstacle changes location, a deterministic policy may fail catastrophically if it relies on a single, now-infeasible mode. A stochastic policy can maintain robustness by supporting multiple valid behaviors, from which guidance or downstream selection mechanisms can choose.

---

3. Tradeoff Between Conditioning Complexity and Distribution Complexity (L256)

We address this fully in our response to Reviewer fdQZ, Point 2, under the "Horizon Tradeoff Hypothesis." In brief: shorter horizons simplify the output distribution but complicate conditioning (as the model must infer context from a single observation). Longer horizons ease conditioning (since the observation can often be assumed to be the initial state) but require modeling a more complex multi-step distribution. The intermediate horizon regime thus suffers from the compounded difficulties of both.

---

4. On the Lack of Theoretical Analysis

We acknowledge the value of theory and are actively working toward formal guarantees. A first result we aim to establish is a bound of the form:

$$\mathbb{E}_{t, x_t} \left( \left. \| \epsilon^\theta(t, x_t) - \epsilon(t, x_t) \| \right| \| \epsilon(t, x_t) \| \right) \leq C(\|\epsilon\|) \cdot \mathcal{L},$$

where $\mathcal{L}$ is the training loss and $C(\|\epsilon\|)$ is an explicit increasing function vanishing at zero. This would characterize how the denoising error increases with noise magnitude, supporting the intuition that OoD predictions at high noise lead to degraded performance. Our goal is to include the full results in the paper.

A more ambitious goal is to relate the expected sample density under the learned distribution to the Stein score norm, potentially bounding it from below by a decreasing function of the score norm. This remains speculative, but we are exploring this direction.

5. Quality

We will conduct a thorough proofreading and will improve the overall writing quality and clarity of our paper.

We thank the reviewer for their constructive comments and insightful questions. The requested experiments were omitted due to space limitation and the focus of the paper. However, we agree with the reviewer that they are valuable and therefore added them as requested (see Point 5).

1. Experimental details - Shortcut model training

We re-implemented the shortcut model from Algorithm 1 in Frans et al., 2024 in PyTorch, using their official GitHub repository (which is in Jax). For each (task, horizon) pair, we trained 10 models via random hyperparameter search. Each reported score on a given task is the best-performing out of all 10 models. All models use the same 65M-parameter UNet backbone to ensure fair comparison.

2. Models Retraining

All models were trained once using 100 denoising steps (128 for the shortcut model). At inference, we vary the number of denoising steps using the cosine schedule from Diffusion Policy. We also experimented with linear schedule and found that the impact of our method was similar but the performance of linear schedule models was significantly lower compared to cosine schedule both for baseline DDPM and GDP. We therefore decided to focus on cosine schedule.

3. Clarification on Step Range in Experiments

The set {1, 2, ..., 10} ∪ {20, 30, ..., 100} refers to inference-time denoising steps only. All base models were trained using the full 100-step schedule (or 128 for shortcuts), and inference was conducted by subsampling the training schedule (Please refer to Equation (2) and lines 85–86 in the paper).

4. DDIM results

The values for DDIM are compiled in the table below and integrated in tables 1 and 2 of the submission.

5. Thorough ablation

Our method modifies the denoising process in three ways: adapting inference schedule, reducing noise injection scale and genetic denoising. For completeness, we added the missing lines to tables 1 and 2 of the submission. For 2-step we provide the normalized scores for

• DDPM,
• DDPM + adapted schedule,
• DDPM + adapted schedule + reduced noise,
• DDPM + adapted schedule + reduced noise + GDP.

2-step DDPM alone always yields minimal score as shown in the table below,

Note that on Robomimic, the difference between $\gamma=0$ and $\gamma=1$ scores is not statistically signifiant as the difference is within 1-sigma.

6. Noise injection scales

Figure 3 and Appendix's Figure 9 include intermediate $\gamma$ values. We chose not to include them in the tables for brevity, as $\gamma= 0$ consistently yielded the best results.

7. Which OoD score was used for GdP experiments

Both Stein and clip-based scores were tested. The Stein score consistently performed better, both in reducing clipping and improving success rate. Thus, we report results using the Stein score.

8. Figure 5 and 6

Figures 5 and 6 are averaged over the remaining two dimensions (e.g., horizons and tasks). We updated the captions to clarify this.

9. Quality

We acknowledge that the current writing should be improved and we are working to improve writing quality and overall clarity.

We thank the reviewer for the thoughtful and constructive feedback. We appreciate the precision provided and the points raised. Below, we address the reviewer’s concerns in detail.

---

1. Weaknesses and Clarifications

Generalizability Across Task Types

We agree with the reviewer’s observation. Our intuition is that for tasks like those from Robomimic, the conditional action distributions are simpler and more localized. As our method enhances the denoising process but does not modify the base model, its benefit is limited when the target distribution is easily covered, even with a few steps. We infer that the limiting factor for the Robomimic tasks is a conditioning bottleneck. The impact of out-of-distribution (OoD) trajectories is then smaller, leading to limited performance gains of our method despite latency reduction. While we did not formally validate this hypothesis, it is consistent with the empirical trends observed across all Robomimic tasks.

---

2. Baseline Comparisons

DDIM Scores

We added the DDIM results, see point 4 of the rebuttal to reviewer PHeJ.

Shortcut Model and γ

Shortcut is a self-consistency variant of Flow-Matching (FM), as such it does not inject noise during inference. Stochasticity only comes from the choice of starting point in latent space. The $\gamma$ parameter has no direct equivalent but may be related to the standard deviation of the latent space normal distribution during inference. We did not experiment with this possibility. Guo et al.'s "Variational Rectified Flow Matching" introduces stochasticity along the inference process via VAE step-model; however, we did not find an adequate equivalent to $\gamma$. Finally, combining both self-consistency (shortcut FM) and distributional step-model (variational FM) would likely motivate a publication on its own.

Shortcut Performance on Adroit

Shortcut models require the network to learn both a vector field and its integral. With a fixed architecture (same UNet backbone), this leads to capacity bottlenecks that likely explain the inferior performance observed.

Shortcut on Robomimic

We agree that comparing the shortcut with the 2-step GDP is relevant for Robomimic tasks; however, no shortcut model checkpoints were publicly available for Robomimic tasks, so the comparison would be unfair to the off-the-shelf diffusion models we benchmarked.

---

3. Conceptual Clarifications

Horizon Tradeoff Hypothesis

Thank you for highlighting this point. A thorough analysis of the horizon-dependence was initially intended to be included in the present work. However, we concluded that a proper treatment reveals deeper phenomena that merit a separate follow-up article. Below, we summarize our current insights.

Empirically, we observe three distinct regimes based on the action horizon $\mathbf{h_\mathcal{A}}$:

• $\mathbf{h_\mathcal{A}} \leq 5$: Low performance
• $24 \leq \mathbf{h_\mathcal{A}} \leq 50$: High performance, then decreasing
• $\mathbf{h_\mathcal{A}} \geq 100$: High performance resumes

We hypothesize that these regimes arise from competing factors:

1. For very short horizons (< 5): the action distributions are heavily overlapping and "blob-like", leading to mode confusion. The model struggles to disambiguate modes due to the compressed embedding space. Increasing the horizon expands the dimensionality of the action sequence, which helps disambiguate modes and better leverage conditioning.

2. Distribution Complexity: As the horizon increases, the model must represent more complex and multi-modal action distributions. Fortunately, diffusion models are expressive enough to accommodate this growing complexity.

3. Conditioning Complexity: Shorter horizons also require the model to infer contextual information (e.g., temporal position) from the observation, which makes conditioning harder. For example, in short-horizon settings, the model must deduce which timestep is being referenced based solely on the observation. In contrast, long-horizon models can typically assume that the observation corresponds to the first timestep.

Hence, performance is governed by a trade-off between distribution complexity (which increases with horizon) and conditioning complexity (which decreases with horizon). We believe that the performance drop observed in the intermediate horizon regime $50 \leq \mathbf{h_\mathcal{A}} \leq 100$ results from both complexities being simultaneously high.

To further analyze this phenomenon, we performed a preliminary investigation of the structure of the learned action distributions. Specifically, we estimated:

• the intrinsic manifold dimension,
• the intrinsic linear dimension, and
• the number of distributional modes.

The ratio between the intrinsic linear and manifold dimensions offers a geometric measure of complexity, which complements standard mode-counting approaches.

Looking forward, we aim to explore more principled Wasserstein-based complexity measures, which can capture both the shape and spread of distributions:

• Expected Wasserstein Distance to Gaussian (as a proxy for distributional complexity),
• Wasserstein Barycenter Distance distance to Gaussian (another proxy for distribution already complexity), and
• Wasserstein Variance (quantifying the distributionnal diversity across conditioning).

Role of the Genetic Algorithm

Our goal was to validate the utility of population-based denoising without introducing algorithmic complexity. We did run comparative experiments Top-k vs. Multinomial Selection: At 2 steps, top-k outperforms multinomial slightly. However, it quickly collapses diversity in multi-step settings, leading to mode collapse.

While a greedy selection heuristic could achieve similar performance in the very low-step regime, it would fail to scale beyond that. The population-based design is essential for maintaining robustness as the denoising depth increases. If diversity is what the current task requires, more sophisticated population-based metaheuristics like Particle Swarm Optimization or Ant Colony Optimization might be better suited and could be applied, but we leave such an exploration to future work.

---

4. Practical and Computational Considerations

Memory and Wall-Clock Cost

We measured the step-wise overhead for GDP using a standard RTX 3080 GPU without specific optimization beyond calling the model batch-wise on the population.

The relatively low cost is due to the under-utilization of the GPU computation and memory. As the model grows bigger, the population may be more constrained. We argue the following.

• In production, the observation embedding may represent a significant portion of the computational budget but since it can be shared across the population, its impact on the overhead is limited.
• Training is typically done with a batch size larger than one; thus, assuming training and inference are done on the same hardware, inference is likely to leave room for a non-trivial population.
• Fully-fused models, weight quantization, and other low-level optimization would allow for a larger batch size (hence a population) during inference.

These points will be discussed in a dedicated section in the appendix.

Hyperparameter Sensitivity

Our coarse grid search served both as an ablation and robustness check. We found GDP to be relatively insensitive to population sizes in [8, 32] and temperature T in [1, 1000]. Increasing temperatures caused performance to vary from optimal at low temperatures to vanilla DDPM at high temperatures. While additional hyperparameters are introduced, we show that good performance is obtainable across a wide range without fine-tuning. That said, we acknowledge that future work should explore more principled tuning strategies or adaptive methods.

---

5. Limitations

Dependence on Fitness Score

Indeed, the success of GDP depends on the choice of fitness function. For robotic tasks with saturation constraints, our clipping-based score is effective. For tasks where clipping is rare (e.g., end-effector control, image domains), this heuristic is less useful. Still, whatever the task, Stein-based scores are always available and valid candidates. Furthermore, we note that it may be possible to learn a task-specific fitness function in a supervised or self-supervised manner, which is a promising direction for future work.

Scalability to High-Dimensional Action Spaces

While our action spaces have ~30 DoFs, our experiments span horizons up to 200, leading to effective dimensions over 6000. This is higher than image datasets like CIFAR. Furthermore, many humanoid benchmarks such as HumanoidBench have ≤61 DoFs, keeping GDP within practical scaling bounds for whole-body control tasks.

We will note this point in the limitations.

6. Quality

We will conduct a thorough proofreading and will improve the overall writing quality and clarity of our paper.