Falcon: Fast Visuomotor Policies via Partial Denoising

We sincerely thank the reviewer for the detailed and thoughtful feedback. We appreciate your recognition of Falcon’s novelty and practical motivation, and we are grateful for your constructive suggestions on deployment, which helped us strengthen the final version of our work.

Real-World Validation

To address the concern about practical utility, we conducted a real-world dexterous grasping using a physical robotic setup (see in Fig. 1 of the link) . Falcon, DDPM, and SDP were evaluated using the same Unet backbone with planning horizon $T_p=16$. As shown in Table 1 on our link, all methods achieved 100% success, while Falcon reduced runtime by 3.07x compared to DDPM, and outperformed SDP in both runtime (0.14s vs. 0.23s) and memory usage (3730MB vs. 3808MB), demonstrating strong real-world performance.

Memory and Computational Overhead

Falcon maintains a small buffer of 20–50 actions (see Appendix D in our submission), while SDP requires a much larger horizon × noise-level buffer (e.g., 16×100). As shown in Table 7 of the link, Falcon’s memory overhead is +12MB, significantly lower than SDP’s +26MB, making Falcon more suitable for real-time and resource-constrained deployment.

Hyperparameter Sensitivity and Usability

Falcon introduces two scalar hyperparameters: $\epsilon$ (reuse threshold) and $\delta$ (exploration rate), which control the balance between inference speed and action accuracy.. While both influence the trade-off between speed and accuracy, our ablations (Figure 3, left and middle in submission) show that Falcon performs robustly when ε ∈ [0.001, 0.01] and δ ∈ [0.1, 0.2], across all tested environments. These parameters are intuitive, require no retraining or fine-tuning, and generalize well across tasks. We will include these practical guidelines in the revision.

Comparison with Streaming Diffusion Policy (SDP)

We thank the reviewer for highlighting SDP [1], which is indeed an important related work. We would like to clarify that SDP was already cited and briefly discussed in Section 5 of our submission. We appreciate the opportunity to strengthen this comparison further.

While both Falcon and SDP leverage partial denoising for policy acceleration, Falcon offers 2 core advantages:

• Training-free and plug-and-play: Falcon does not require task-specific training or modifications to the diffusion model. In contrast, SDP must be retrained on each task with a specially designed noise corruption scheme (see SDP Section 3.3) to enable recursive denoising. This limits SDP’s adaptability and ease of deployment.
• Lower memory overhead: Falcon uses a threshold-based selection mechanism to maintain a compact buffer (20–50 entries). SDP, by design, stores a full horizon × noise-level buffer (e.g., 16×100 entries). As shown in Table 7 on our link, Falcon adds only +12MB of memory compared to DDPM, while SDP adds +26MB—a significant difference for resource-constrained settings.

In both simulated tasks (Tables 4–8 in our link) and real-world experiments (Table 1 in our link), Falcon achieves comparable or higher success rates than SDP, while also achieving higher speedups and lower runtime (e.g., 0.14s vs. 0.23s in real-world grasping, . These results demonstrate that Falcon not only accelerates diffusion policies but also transfers effectively to physical robot systems.

Figure Improvement (Regarding Figure 1)

Thank you for the helpful suggestion. We have revised Figure 1 to improve clarity and visual structure, now included in our updated submission and in the link (Figure 3).

The new diagram adopts a cleaner modular layout and color-coded elements to distinguish the two-stage mechanism: (1) reference action estimation and (2) threshold-based candidate selection. We believe this revision significantly improves readability.

Closing Remarks

We once again thank the reviewer for their valuable feedback. If our responses have addressed your concerns, we would be sincerely grateful for your consideration in revising the score. Please let us know if further clarification would be helpful—we would be happy to provide it.

References

[1] Fast Policy Synthesis with Variable Noise Diffusion Models

We sincerely thank the reviewer for the thoughtful suggestion and for the generous score. We deeply appreciate your recognition of our work, and your feedback has helped us identify opportunities to clarify and strengthen our contributions.

On the Relationship Between $\epsilon$ and $\delta$

In Falcon, $\epsilon$ and $\delta$ play complementary roles in controlling the reuse of partial denoised actions.

• $\epsilon$ serves as a selection threshold that determines whether a previously denoised action is temporally aligned enough to be reused. Smaller ε enforces stricter matching (favoring accuracy), while larger $\epsilon$ allows more aggressive reuse (favoring speed).

• $\delta$ is the exploration rate, specifying the probability of discarding the reused action and instead sampling from standard Gaussian noise. This mechanism injects diversity into the action sequence and prevents over-reliance on suboptimal reuse.

While our ablations (Figure 3, left and middle in our submission) analyze these parameters independently, we agree that their joint effect is important. In our experiments, we typically fix $\delta = 0.1$ or $\delta = 0.2$ , and find that $\epsilon \in [0.001, 0.01]$ works robustly across tasks. This setting consistently balances acceleration and success rate. We will explore their interaction more formally in future work.

On the Definition of $\alpha$

Thank you also for pointing out the missing definition of $\alpha$. It refers to the variance schedule in the forward diffusion process, as commonly defined in DDPM[1] or DDIM[2]. We will include the formal definition and cite the appropriate reference in the revised version.

We once again thank the reviewer for their thoughtful and constructive feedback. Your comments have been instrumental in helping us refine the clarity and usability of our method.

References

[1] Denoising Diffusion Probabilistic Models

[2] Denoising Diffusion Implicit Models

We sincerely thank the reviewer for the thoughtful and constructive feedback. We appreciate your recognition of Falcon’s motivation and your suggestions for strengthening the comparison and evaluation. Below, we address each concern in turn.

On the Effectiveness of the Threshold Mechanism (DDIM+Falcon)

The performance drop in Table 1 for DDIM+Falcon stems from the threshold $\epsilon$ being set relatively high in our original configuration to maximize speedup. This allowed partial denoised actions with weaker temporal dependency to be reused, leading to performance degradation. This does not indicate a flaw in the mechanism itself, but rather reflects a trade-off between efficiency and accuracy.

To address this, we conducted additional experiments with smaller $\epsilon$ values. As shown in Table 9 in the link, reducing $\epsilon$ (e.g., 0.005 → 0.001, 0.01 → 0.003) significantly improved success rates in Transport (0.74 → 0.81) and Tool Hang (0.51 → 0.54), while retaining acceleration.

We also note that Figure 3 (left) in our main paper presents an ablation study showing this trade-off trend. For more task-specific results, Figure 5 in the link further supports this behavior. We will incorporate the updated results in the revised version.

Comparison with SDP[1]

We would like to clarify that SDP was already cited and briefly discussed in Section 5 of the original submission. We appreciate the opportunity to provide a more detailed comparison.

Falcon offers two key advantages over SDP:

• Training-free

  Falcon requires no retraining or noise-corruption design. In contrast, SDP must be retrained per task with a handcrafted noise corruption scheme (SDP Sec. 3.3), limiting flexibility.

• Less Memory Cost

  Falcon’s thresholding mechanism allows it to maintain a small buffer (e.g. 50, see Appendix D), while SDP stores a full horizon × noise-level buffer (e.g., 16×100). As shown in Table 7 on our link, Falcon adds only +12MB of memory compared with DDPM, but SDP adds + 26M.

Finally, we implemented SDP under our setting and provide comparisons in Tables 4–8 on our link. Falcon achieves comparable task performance while offering higher speedups and lower memory overhead.

On Sensitivity to $\epsilon$ and Deployment Concerns

We agree that $\epsilon$ is a key hyperparameter, and we have already provided a detailed analysis of its impact in Figure 3 (left) in our submission and Figure 5 in link. These results show that:

• Falcon’s performance remains stable within a broad $\epsilon$ range (e.g., 0.001–0.01) across tasks;
• The accuracy-speed trade-off is smooth and interpretable, making ε easy to configure in practice;
• The same $\epsilon$ values generalize well across different environments, reducing per-task tuning burden.

Although Falcon does rely on $\epsilon$ , it is training-free, and introduces only a few parameter to adjust—a significantly lighter requirement compared to training based acceleration methods.

We believe this controllable trade-off offers a practical balance between deployment flexibility and performance.

On Real-World Deployment Evaluation

To assess real-world applicability, we conducted a dexterous grasping experiment with a physical robotic platform (Figure 1 in link), composed of a RealMan 7-DoF arm, a PsiBot 6-DoF hand, and dual RealSense cameras.

DDPM, SDP(DDPM), and Falcon+DDPM were deployed using the same Unet architecture, with $T_p=16,T_o=1,T_a=8$. As shown in Table 1 on our link, all methods achieved 100% success rate, but Falcon achieved 3.07× speedup and outperformed SDP in both runtime (0.14s vs. 0.23s) and memory consumption (3730MB vs. 3808MB CPU).

These results confirm that Falcon not only accelerates diffusion policies in simulation but also transfers efficiently to real-world robotic execution.

Others

We consider Paper [2] as concurrent works.

Closing Remarks

We once again thank the reviewer for the detailed comments. If our responses have addressed your concerns, we would be sincerely grateful for your consideration in revising the score. Please let us know if further clarification is needed.

References:

[1] Fast Policy Synthesis with Variable Noise Diffusion Models

[2] Responsive Noise-Relaying Diffusion Policy

We sincerely thank the reviewer for the thoughtful and detailed comments. We appreciate the constructive feedback and address the key concern below.

Clarifying the Motivation and Positioning of Our Method

Our contribution is not to replace distillation or ODE solvers, but to offer a practical, training-free alternative that leverages temporal structure for faster inference in diffusion policies.

While we discuss limitations of prior methods, Falcon is not motivated by them. As stated in our abstract, “The core insight is that visuomotor tasks exhibit sequential dependencies between actions at consecutive time steps.” Falcon leverages this via partial denoising to reduce sampling steps.

We fully recognize that distillation methods are powerful, particularly in settings with large data and compute [2,4,5,6]. In such settings, they deliver substantial speedups while maintaining performance. We respect this line of work and do not position Falcon as a replacement.

Instead, Falcon instead targets low-resource regimes. In these cases, distillation may be infeasible, and ODE solvers can suffer from large discretization errors under very low NFE [1]. Falcon addresses this gap by offering a plug-and-play solution that also integrates well with ODE solvers in low-step regimes (Section 4.3).

We acknowledge that our initial wording may have been overly strong regarding distillation. While methods like CP [2, Sec. 5] may face challenges in preserving multimodality, this is task-dependent. We will revise accordingly, and thank the reviewer for pointing this out.

Lack of Theoretical Foundation

Although Falcon lacks a full theoretical derivation, its core components are grounded in well-established principles.

Falcon is built on the insight that in sequential decision-making, when past actions are strongly correlated with the current one, initializing from them can reduce sampling steps. Falcon exploits this via reusing past denoised actions.

Crucially, the choice of which past action to reuse is not heuristic. It is determined by Falcon’s thresholding mechanism, based on Tweedie’s formula, a foundational result in empirical Bayes theory. As noted in Remark 1 of [7], Tweedie’s formula yields the Bayes-optimal posterior mean under Gaussian noise, guiding our selection of the cleanest available prior. This forms the theoretical core of our partial denoising module.

Thus, while Falcon lacks a full pipeline of theory, its main mechanism—Tweedie-based partial action reuse—is mathematically justified and validated through strong empirical performance.

On Expressing Multimodal Distributions

1. The BlockPushing task has inherently multimodal expert trajectories (BeT [3], Table 2).
2. We added a Gaussian baseline using the same network. It fails on both goals (p1: 0.02, p2: 0.01), while Falcon succeeds (p1: 0.99, p2: 0.97), (see in Table 3 in the link) showing the need for multimodal modeling.
3. We evaluated Consistency Policy on PushT and found it biased toward one mode, indicating that some distillation methods may struggle to retain multimodality see in Figure 4 in the link.

Comparisons with Reduced-Step DPMSolver

To directly assess whether lower-step DPMSolver leads to performance degradation, we matched its NFE to that of DPMSolver+Falcon. Under this constraint, DPM-Solver* exhibited significant performance drops—10.6% in Square_ph, 35.6% in Square_mh, and 6.8% in Transport_ph—while Falcon maintained high success rates (see Table 2 in link). This confirms that aggressive NFE reduction in ODE solvers can hurt performance, and Falcon helps mitigate this degradation.

Missing or Incorrect References

We will add the missing citation for Progressive Distillation [6], and correct the mistaken reference on Line 432.

Closing Remarks

We are grateful for the reviewer’s thoughtful comments, which helped us improve both the clarity and positioning of our work. If our responses have addressed your concerns, we would greatly appreciate a reconsideration of the score. Please let us know if further clarification is needed.

References

[1] PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores

[2] Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation

[3] Behavior Transformers: Cloning k Modes with One Stone

[4] Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning

[5] Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion

[6] Progressive Distillation for Fast Sampling of Diffusion Models

[7] Diffusion Posterior Sampling for General Noisy Inverse Problems