ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation

We sincerely thank Reviewer 7Vfu for their extremely thorough and detailed review. We appreciate the positive comments on our motivation and method design. Your critical feedback on task design, evaluation, and the framing of our contributions is invaluable, and it has pushed us to significantly clarify and strengthen our paper. We address each of your concerns below.

1. On Task Design, Difficulty, and Generalization

This section addresses the reviewer's primary concerns about the quality and design of our tasks and generalization experiments.

Regarding Task Difficulty and Realism (Weakness 1 & 2): We appreciate the reviewer's concerns about task design and would like to provide more context.

• On the "Plug Insertion" setup: We understand the concern that starting with the plug half-inserted reduces complexity. We initially attempted a full "pick-move-insert" sequence. However, due to hardware limitations, our gripper's frictional force was insufficient to hold the plug firmly against the large insertion forces required. We therefore opted for the "push-to-insert" task design. This setup introduces its own unique challenges that are highly relevant to force control: without a firm grasp, the plug is prone to wobbling and shifting, requiring the policy to be highly sensitive to instantaneous force changes to maintain a stable push. We will add a discussion of this hardware limitation and the resulting design choice to the paper. When hardware permits (e.g., with a stronger gripper), we plan to explore the full sequence in future work.
• On "Unsmooth Trajectories": We apologize for this impression. The videos on our website were sped up 2-3x for brevity. The original trajectories are smoother, and we will clarify this on the website. Regarding "trial-and-error," we acknowledge this reflects the current capabilities of the model and that achieving smoother, more "ballistic" motions is a key challenge for future work.
• On "Board Wiping": We agree that if the task could be solved by simply applying a large force, its value would be diminished. However, this is not feasible in our setup. Applying excessive force pushes the whiteboard out of its fixed position and beyond the robot's workspace, leading to task failure. Therefore, controlled force application is necessary to wipe the board effectively while keeping it in place.
• On a "More Demanding Task": The suggestion of wiping a bristle bottle is excellent. This is a great idea for a more challenging benchmark, and we will consider it for future work.

Regarding the Benefits of Force Modeling in Generalization (Weakness 2): Thank you for pointing out the need to better highlight how our generalization experiments test force modeling. While some experiments involve visual changes, many are designed to create novel physical dynamics:

• For Pump Bottle, generalization to bottles of different heights and pumps with different travel distances requires the policy to use force feedback. For shorter bottles or longer pumps, the robot must press lower, using the upward reaction force to determine when the press is complete or if it has exceeded a safe limit.
• For Plug Insertion, using plugs of different sizes and shapes introduces complex new force dynamics. Smaller plugs demand more precise directional control during contact, while taller plugs require fine control over the timing and magnitude of the force. The "unstable socket" generalization specifically introduces random, unpredictable dynamic forces (sliding, wobbling), forcing the model to rely on precise force sensing and immediate adjustments to counteract the instability. The visual occlusion task for this setup similarly forces a greater reliance on force modeling over vision.

We will revise our experimental section to make these force-specific challenges much clearer.

2. On Methodological & Training Details

This section addresses the reviewer's specific questions about our implementation.

Regarding Teleoperation Details (Question 1): Although the human operator does not receive direct haptic feedback, they are provided with real-time visual feedback of the forces within their Quest3 VR headset. The 6D wrench is visualized as a dynamic force vector at the TCP, allowing the operator to see and react to contact forces. The main challenge during teleoperation (e.g., for USB insertion) is the low resolution of the headset's display, which can make fine details hard to see from a distance. Operators sometimes mitigate this by looking at the real setup with their own eyes. We have not formally tracked the teleoperation success rate, but anecdotally, we perform about 60 teleoperation trials to collect 50 successful demonstrations, suggesting a success rate of approximately 85%.

Regarding Single vs. Multi-task Training (Question 2): Thank you for this question. In our main experiments (Figure 5), we trained separate, task-specific policies. However, we also conducted multi-task joint training, and these results are reported in Section D (Table 5) of our supplementary material. The results show that our model performs well both as a single multi-task policy and as separate task-specific policies. We will add a pointer to these results in the main paper.

Regarding Base Policy (π₀) Usage (Question 4): We initialize our model with the pre-trained weights of π₀. During training, we use the LoRA method to fine-tune the original π₀ parameters (such as the Gemma model and action expert), while we train the new parameters of our proposed FVLMoE module from scratch. We will clarify this training procedure in the methodology section. Regarding Dataset Size and Generalization (Question 5): We agree that a few hundred trajectories is a relatively small dataset. We believe our strong generalization results demonstrate that the model is not merely overfitting. We encourage this generalization through two main mechanisms:

1. Implicit Regularization from Pre-training: Fine-tuning a powerful VLA like π₀ provides a strong prior that helps prevent overfitting.
2. Data Diversity: During data collection, we consciously introduce variations in object starting positions, interaction angles, and approach directions to enrich the training data. We also randomize initial object positions during testing.

3. On Positioning, Originality, and Broader Comparisons

This section addresses the higher-level concerns about our work's framing and its relation to other methods.

Regarding the VLA Framing and Language Component (Question 3 & Limitations): This is a fair point. While the language instructions in our main experiments are for a predefined set of tasks, our goal is to build towards general-purpose robots that can handle diverse tasks at a macro level while also executing fine-grained, contact-rich motions at a micro level. Our multi-task training results (in Appendix D), which require language understanding to differentiate tasks, show that our model has potential for open-world generalization. We chose the VLA framework to align with this long-term vision, rather than developing a more narrowly-focused, vision-force policy. While we acknowledge we have not demonstrated large-scale language generalization due to time constraints, we believe our work is a critical step in that direction.

Regarding Originality (Limitations): While the core components we use (feature fusion, MoE) have been explored in other contexts, we believe our primary novelty lies in the specific architectural design and first-of-its-kind demonstration of integrating these techniques for online, 6D force-based VLAs operating in complex, real-world contact tasks. This specific problem domain and our successful application remain underexplored.

Regarding Comparison to Diffusion Policy (Question 6): Thank you for the suggestion. We agree that Diffusion Policy and 3D Diffusion Policy are strong baselines for visuomotor control. For this initial work, we prioritized a direct comparison with the π₀ VLA family, as our method is a direct extension of the VLA paradigm. Given the time constraints of the rebuttal period, we were unable to implement and evaluate these new baselines, but we recognize their importance and will include them in future iterations of this work.

We also thank the reviewer for the feedback on our project website. We will update it with a more organized and polished design to better present our methodology and results.

We hope these detailed responses have addressed all your concerns. We are grateful for the thorough review, which will undoubtedly help us produce a much stronger final paper.

We thank Reviewer 4bXA for the constructive feedback and for raising important questions about the interpretation of our results and the broader applicability of our work. We address each point below.

1. On Simulation Experiments, Reproducibility, and Scalability(Weakness 1, Question 1)

We appreciate the reviewer's point regarding simulation experiments. Our primary focus for this work was to tackle the challenges of force-aware manipulation directly in the real world, as physical interaction and multi-body contact dynamics are central to our problem. We believe that using real-world data ensures our findings are directly applicable to physical deployment scenarios. While we agree that simulation offers significant benefits for scalability and reproducibility, developing a high-fidelity simulation for these diverse, contact-rich tasks was beyond the scope of our current resources for this submission. However, we concur that large-scale training with generated data in simulation is a very promising direction for expanding the range of tasks our model can address. We see this as a key area for future work and will add this discussion to the paper.

2. On the Performance of π₀-fast in the "Unstable Socket" Task(Weakness 2, Question 2)

This is a very keen observation, and we thank the reviewer for asking for a deeper insight into this interesting result. Our analysis reveals a nuanced reason for this behavior. We have consistently observed that π₀-fast models, due to their architecture, often exhibit stuttering, pausing mid-air, or significant delays between actions. Paradoxically, this frequent pausing becomes an advantage for the specific "Unstable Socket" task. When the end-effector pauses while in contact with the plug, the likelihood of the unstable socket sliding or wobbling decreases significantly. This effectively reduces the difficulty of this particular generalization task to that of a stable socket, creating the impression of superior performance for the π₀-fast model. However, for general contact-rich, fine-manipulation tasks, we argue that π₀-fast is not as competitive. Its autoregressive decoding nature limits its inference speed, which in turn hinders its ability to react promptly to real-time force changes and adjust its trajectory flexibly. We will add this important nuance to our analysis section to prevent misinterpretation of this result.

3. On Generalization to Low-Cost Robots without Precise F/T Sensors(Question 3)

Thank you for this forward-looking question about practical deployment. Due to time and hardware constraints, our current experiments are limited to a single robot platform. However, we strongly agree that demonstrating generalization to more common, lower-cost platforms is crucial. To promote wider practical deployment and help democratize force-aware manipulation research, we are actively assessing ForceVLA’s adaptability and performance on lower-cost platforms equipped with external or retrofitted force sensors.

We hope our responses have clarified these points and addressed your concerns. Thank you again for your valuable feedback.

We sincerely thank Reviewer 1t4m for the thoughtful and detailed feedback. Your comments on architectural clarity, experimental analysis, and missing comparisons are invaluable. They have helped us identify key areas for improvement, and we have addressed each point below.

1. On Architectural & Conceptual Clarity

Regarding MoE Specialization (Weakness 2, Question 1): While it is challenging to definitively assign a single modality to each expert, our analysis reveals clear specialization patterns based on task and execution phase. As detailed in Section C of our supplementary material, we observe that the router learns to dynamically allocate different experts for different stages of a task. For example, Figure 9 shows that the expert load varies across different task completion percentages. This indicates that the router has learned a strategy to leverage different experts based on the evolving context of the task, rather than a static modality assignment.

Regarding the Definition of "Force Observation" (Weakness 3, Question 2): Thank you for this question highlighting a point of ambiguity. To clarify, a "force observation" is a reading from a single frame/timestep, not a sequence over time. Specifically, at each observation step, this single reading is a 6-dimensional vector representing the estimated external wrench applied on the TCP, expressed in the world frame ($^0F_{ext}\in \mathbb{R}^{6\times 1}$). This vector consists of a $\mathbb{R}^{3\times1}$ force component and a $\mathbb{R}^{3\times1}$ moment component: $[f_x,f_y,f_z,m_x,m_y,m_z]^T$. The units are Newtons [N] for force and Newton-meters [Nm] for moments. We will revise the paper to make this distinction between a single-step reading and a time-sequence explicit to avoid any confusion.

Regarding Force-Sensitive Data Collection (Weakness 4, Question 3): Although the human operator does not receive direct haptic feedback, they are provided with real-time visual feedback of the forces within their Quest3 VR headset. The 6D external wrench readings from the robot's sensor are streamed to the headset and visualized as a dynamic force vector originating from the robot's TCP. This vector continuously changes its length and orientation based on the measured force, allowing the operator to see and react to contact forces during teleoperation, thus enabling the collection of force-reactive demonstrations.We acknowledge that teleoperation systems with direct haptic force feedback are now available. Due to hardware constraints in our current setup, we did not integrate this capability. We agree that this is a valuable direction and will certainly consider incorporating direct haptic feedback in our future work to potentially enhance the quality and reactivity of the demonstrations.

2. On Analysis of Experimental Results

Regarding Typical Failure Modes and Low Success Rates (Weakness 5, Question 5): We have compiled a detailed breakdown of the failure modes for each task and model:

• Plug Insertion: w/o force models generally exhibit cruder behavior during insertion, unable to adjust their strategy based on interaction state changes. w/ force models can generally perceive changes in external forces but are not flexible enough in adjusting their trajectory. ForceVLA can better maintain continuous contact, follow the plug's surface, make fine adjustments to the force angle and duration within a more precise range, and terminate the action more promptly upon full insertion.
• USB Insertion: The overall success rate for this task is low. A common problem across all five models is the difficulty in aligning with the USB port, which we attribute to insufficient visual clarity and the backbone model's inability to process fine-grained visual information. However, ForceVLA's success rate is slightly higher because it exhibits clear autonomous adjustment or re-attempt behaviors upon feeling external force from contact with the socket.
• Bottle Pumping: Simply pressing a seen bottle could be completed with 100% success by the pi0-base models and ForceVLA. Therefore, during testing for this task, we introduced additional visual occlusions and background changes. Most of the failure modes occurred under these specific variations. w/o force models often missed the pump or did not press it fully. ForceVLA was more robust but sometimes pressed off-center.
• Cucumber Peeling: w/o force models were significantly more prone to breaking the peel mid-way, unable to peel continuously from end to end, and sometimes peeled too deeply, indicating a lack of control over the peeling force. w/ force models peeled more stably with a wider peel, but still had issues with breaking the peel and not following the cucumber's curvature well. ForceVLA could overcome these issues.
• Board Wiping: A common problem for all models was the inability to pick up the eraser, possibly because its placement was far from the base camera, resulting in low resolution, and its color was similar to the black table, making precise localization difficult. In the remaining trials, if a small part of the writing wasn't erased, we classify as a failure. Additionally, if a model didn't stop after 5 minutes, we counted it as a failure. The w/o force models started wiping in the air or pressed too hard, leaving scratches, due to their inability to perceive force. ForceVLA made closer contact with the board and applied a more appropriate wiping force than other models.

We will integrate this detailed failure analysis into the appendix to provide a much richer context for our quantitative results.

Regarding π₀ Performance with Force Input (Weakness 7, Question 4): The performance degradation of π₀ with force in generalization tasks (Table 2) stems from an out-of-distribution data problem. The base π₀ model was pre-trained on state information that did not include force data. When we concatenate force information to the state input for fine-tuning, the model performs well on in-distribution tasks. However, during generalization to unseen objects and scenarios, the model must rely more heavily on its pre-trained knowledge. Since the force data is an unfamiliar and out-of-distribution signal for the pre-trained π₀, it disrupts its existing priors, leading to a drop in performance. Our ForceVLA architecture is designed to mitigate this by handling multimodal signals more robustly.

3. On Additional Comparisons and Ablations

Regarding Comparison to MoE without Force (Weakness 6, Question 6): To directly assess the value of the force modality within our MoE architecture, we conducted the suggested ablation study where the force input to the ForceVLA model was masked.

The results are as follows: The success rates were 20% for Plug Insertion, 0% for USB Insertion, 30% for Pump Bottle, and 26.7% for Wipe Board. The Peel Cucumber task was not reported due to time constraints. Compared to the success rates of ForceVLA with force inputs, these results represent a drop of 60% for Plug Insertion, 25% for USB Insertion, 37% for Pump Bottle, and 0.3% for Wipe Board.

In this ablation study, the common failure modes for ForceVLA w/o force inputs included:

• During Plug Insertion, the end-effector could only push a short distance in the insertion direction, and the trajectory's curvature was insufficient to properly insert the plug into the socket; alternatively, the end-effector would only move to the plug's surface and perform an arcing push motion near the contact point without making actual contact, similar to overfitting the insertion trajectory.
• In the USB Insertion task, the gripper's closing width was too large, making it unable to grasp the USB drive, possibly due to overfitting the gripper width and preventing a firm grasp.
• When pumping the bottle, it repeatedly applied excessive force, causing the gripper to bend, or the off-center press caused the bottle to spring away.
• When wiping the board, the number of attempts to erase the writing was reduced, leading to most cases where the board was not wiped clean.

We summarize the likely cause of these failure modes as follows: Without force input, using the FVLMoE module increases the model's total number of parameters, making it more prone to overfitting. During inference, it exhibits many stereotypical behaviors that are misguided by vision, which represents an overfit to the action distribution of the training data. Conversely, the fact that the full ForceVLA model with the additional force modality does not suffer from this overfitting strongly indicates that our architectural design is effective at leveraging force signals to regularize the policy and achieve robust, reactive behaviors instead of merely memorizing training trajectories. We will add this crucial ablation study and analysis to the paper.

Regarding Related Work on VTLA and FuSe (Weakness 1): We sincerely thank you for pointing out these highly relevant works. We have studied both papers and agree they are essential for situating our work. We will add a detailed comparison to our Related Work section.

• Comparison with VTLA: VTLA focuses on sim-to-real transfer using a new simulation dataset and fingertip tactile sensors. Our work is fundamentally different as it is a real-world-first approach, using a new real-world dataset with 6D force-torque data, and our novelty lies in the MoE architecture for online force modulation, not sim-to-real.
• Comparison with FuSe: FuSe introduces a finetuning recipe to adapt existing generalist models to new modalities. Our contribution is a complete, end-to-end policy architecture. Our focus is on investigating a new architectural paradigm (the MoE) for deep integration.

We hope these clarifications, analyses, and new experimental results have addressed your concerns. Incorporating these changes will significantly improve the quality and clarity of our paper. Thank you again for your valuable guidance.

We sincerely thank Reviewer hqeq for their positive assessment and very insightful feedback. We are encouraged that the reviewer found our work to be a "timely and important direction" and a "meaningful advancement." Your constructive comments have been instrumental in helping us clarify and strengthen our contributions. We address your points below.

1. Regarding the Discussion and Positioning of the Dataset (Weakness 1, Question 1)

We thank the reviewer for this detailed and extremely helpful suggestion. We agree completely that a more thorough comparison with existing datasets will better situate our work and help the community appreciate its unique utility. As suggested, we have performed a detailed analysis and will incorporate the following discussion into the final version of our paper.

• Comparison with RH20T (Fan et al.): While RH20T provides a valuable, large-scale dataset, its primary offering is raw, unformatted data with basic reading tools. Our work significantly advances usability and integration by providing not only the raw data but also a pre-processed dataset in the SOTA Lerobot format and the conversion tools themselves. Datasets in the Lerobot format are "plug-and-play," meaning they can be directly loaded to train most models within the powerful Lerobot library. This out-of-the-box compatibility removes significant data-wrangling overhead, greatly accelerating research in this area.
• Comparison with Factory (Narang et al.): Factory is a powerful toolkit focused on simulating contact-rich interactions at scale, providing virtual assets and controllers to train policies in a simulated environment. Our approach is fundamentally different, as our data is collected entirely in the real world. By doing so, our pipeline and dataset bypass the well-known sim-to-real gap. Our work thus offers a complementary, reality-first methodology that is more directly applicable to physical robot deployment.
• Comparison with IndustReal (Tang et al.): The approach of IndustReal is notable as it explicitly avoids F/T sensors, instead relying on other cues to manage contact. Our work is built on a contrasting philosophy. We posit that explicit, high-resolution force feedback is a critical signal for dexterous manipulation. Therefore, our ForceVLA-Data prioritizes the deep integration of 6D F/T sensor data. This positions our dataset as an essential resource for developing and benchmarking a distinct class of policies that explicitly leverage force.

In summary, while these pioneering datasets have pushed boundaries in task diversity (RH20T), simulation (Factory), and sim-to-real (IndustReal), our ForceVLA-Data offers unique contributions in three key areas: out-of-the-box usability with the Lerobot framework, a reality-first methodology, and a focus on explicit 6D force sensing.

We will add this detailed discussion to a new subsection in our Related Work section. We thank the reviewer again for prompting this valuable clarification.

2. Regarding the Discussion of Failure Modes(Weakness 2, Question 2)

We thank the reviewer for pointing out the need for a more detailed failure analysis. Below, we describe the typical failure modes for our ForceVLA model on the mentioned tasks and address the question about alternative fusion strategies.

• For "Pump Bottle": The most common failure mode for ForceVLA was making contact with the pump head off-center. It is important to note that for the standard task with a bottle seen during training, our model achieved a 100% success rate. The observed failures occurred primarily when we introduced challenging visual variations (occlusions and background changes) during testing to probe the model's robustness.
• For "Wipe Board": The failures for ForceVLA fell into two categories:

1. Grasping Failure: In some cases, the model failed to pick up the eraser. We attribute this to visual perception challenges, as the eraser was placed far from the base camera, resulting in a low-resolution image, and its color was similar to the black table, making precise localization difficult.
2. Timeout due to Meticulousness: Interestingly, some failures were due to timeouts. The model would successfully erase ~90% of the writing but would then spend a prolonged period making repeated, fine-grained attempts to erase a tiny remaining smudge. While this is technically a failure by our timeout metric (5 minutes), this behavior suggests the policy is highly sensitive to the task goal, perhaps overly so.

• On Different Fusion Strategies: This is an excellent question. For the failures we observed (e.g., grasping the low-resolution eraser, off-center pressing under visual distraction), we believe the root cause lies more in the limitations of the visual perception backbone rather than the FVLMoE fusion module itself. Our MoE is designed to dynamically weigh inputs, but it cannot overcome fundamental ambiguity or a lack of fine-grained detail in the upstream visual stream. We hypothesize that an alternative fusion strategy would likely face similar challenges if the input perception is noisy. However, we agree that exploring fusion mechanisms that can explicitly model visual uncertainty is a very promising direction for future work. We will add this discussion to the limitations section of our paper.

Once again, we sincerely thank you for your thoughtful and constructive review. Your feedback has been invaluable in helping us strengthen the paper, and we hope our responses and planned revisions have fully addressed your concerns. Thank you for your time and guidance.

I appreciate the authors’ efforts in addressing the comments raised in my original review. The rebuttal addressed them. Please ensure to incorporate these comments in the final manuscript.

Overall, the work is important to the community. Specifically,

1. This paper explores a timely and important direction—integrating force signals into vision-language-action (VLA) models for contact-rich manipulation tasks.
2. The authors benchmark their model against state-of-the-art approaches such as pi-0 and demonstrate superior performance in tasks like “Insert USB,” “Insert Plug,” and “Peel Cucumber.”
3. The authors conduct ablation studies on various architectural designs of ForceVLA, with a particular focus on the integration of force feedback. This analysis is valuable for researchers interested in incorporating temporal signals into vision-language models.

Thus, I recommend accepting the paper.