VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning

Thank you for your detailed review and constructive feedback. We address your concerns point by point below:

Clarity of Explanation. You highlighted the need for more detailed explanations of key processes, particularly how predicates condition on previous actions. We have addressed this by:

• Adding an annotated example to Appendix B.1 that demonstrates how prior actions, observations, and truth values are leveraged in the predicate evaluation.
• Expanding Section 5.3 with a clearer explanation of how the predicates are validated.

Reliance on VLMs and Practicality. We agree that the reliance on VLMs introduces potential variability and would like to highlight that our system is designed to handle this in two additional ways:

1. Neuro-symbolic reasoning: Our method does not exclusively rely on VLMs. Instead, we combine symbolic reasoning with VLM question answering to combine the strength of both processes.
2. Iterative Optimization: The iterative predicate proposal-and-selection process allows the agent to refine its abstraction implementation, optimizing for robustness and reliability. We see further improvement in this as an exciting area for future research and have added Appendix B.5 to discuss this. This section outlines the system’s current limitations to handle variability and suggests future extensions.

Time Complexity and Performance. You rightly emphasized the importance of computational efficiency, particularly for online learning in robotics. Our approach is efficient because each predicate invention iteration completes within minutes on a single GPU, and once abstractions are learned, planning is extremely fast due to the use of domain-independent heuristics and symbolic planners from the AI planning community. To further clarify, we have included a more detailed time complexity analysis in Section 5.3 and Appendix B.2 and C.3, illustrating the scalability of our method across increasing task complexities. We believe this demonstrates the approach’s feasibility for real-world applications.

Robust, diverse yet task-relevant predicate set. Diversity and relevance are key characteristics of candidate predicates, and our method explicitly promotes these qualities: On one hand, we propose predicates using three complementary strategies (described in Section 5.2 and detailed in Appendix B.4): discrimination, transition modeling, and logical derivation. On the other hand, our predicate selection process balances task relevance and simplicity through the two predicate score functions. Section 5.3 has been revised for clarity on this process.

Mechanism for leveraging failure. We can think of two orthogonal ways to leverage infeasible plans to inform future planning decisions. The first is to learn from failed plans by inventing predicates and learning the operators, and the second is to improve the planner used. Our approach is an instance of the first one, and we see the second as an interesting direction for future research. Specifically, our approach invents predicates (in particular with the prompting strategy #1) and learns operators to prevent the controllers that failed in previous planning attempts from being selected again in similar states. On the other hand, within the planner--specifically for the A* search--we can try to learn a node expansion policy, where the policy could decide how to expand the existing partial plans at the current state, based on the knowledge of previous infeasible plans.

In summary, we have revised the manuscript to address your concerns and to better articulate the contributions of our work. We hope these improvements clarify the robustness and practicality of our approach. Thank you again for your thoughtful review and for helping us refine this work!

Thank you for your thoughtful feedback. We appreciate your positive comments and have carefully addressed each of your concerns and questions below. Regarding the weaknesses identified by you, for example:

Have you considered applying D-TSN to real-world tasks or environments with higher-dimensional observation spaces, such as language or vision?

We think that it's possible the weaknesses section of your review was meant for another paper you might be reviewing (about D-TSN, not NSP) which doesn't involve vision (as we learn from images). We suspect this was an innocent copy-paste mistake. We're really happy to have your support, and we'd welcome discussion regarding limitations of our paper.

Observation Noise and VLM Reliability. To address questions 1 and 2 together, we fully agree that the reliability of visual language models (VLMs) under noisy observations could pose challenges for our approach. However, we have designed the system to handle this in the following 2 ways:

1. Neuro-symbolic reasoning: Our method does not solely rely on VLMs. Instead, we combine VLM outputs with symbolic reasoning, improving both robustness and flexibility.
2. Iterative Optimization: The iterative predicate proposal-and-selection process allows the agent to refine its abstraction implementation, optimizing for robustness and reliability. We recognize that further improvements in this area are crucial and have added a discussion of these limitations and future directions in Appendix B.5. This section outlines the current challenges and suggests extensions for handling observation noise and VLM inaccuracies more effectively.

Assumption about skills. We agree that relaxing the assumption of fixed low-level motor skills is a fascinating direction for future research. Learning skills can leverage advances in closed-loop policy writing (e.g., Code as Policies), reinforcement learning for manipulation skills (e.g., RMA$^2$, Eureka), and motion planning (e.g., ReKep). Additionally, the interplay between symbols and skills is an exciting area for exploration. For example, predicates can serve as subgoals for controllers or as components of reward functions for learning new skills. We see this virtuous cycle as a promising avenue for future work. We have included some discussion of this in the revised manuscript.

Thank you for your positive assessment and valuable comments. Below, we address each of your concerns in detail.

VLM reliability. We agree that reliance on VLMs can introduce variability, and addressing this issue has been a key consideration in our design. Our system mitigates this through:

1. Neuro-symbolic reasoning: Our method does not exclusively rely on VLMs. Instead, we complement them with deterministic symbolic computations, enhancing generalization and improving the system’s overall robustness.

2. Iterative Optimization: The iterative predicate proposal-and-selection process allows the agent to refine its abstraction implementation over time, optimizing for robustness and reliability. We acknowledge this as an open challenge and an exciting area for future research. We have added a discussion in Appendix B.5, where we outline the limitations and propose potential extensions.

Clarification for low-level skills. We design hand-coded closed-loop controllers for each environment and these are used across all the approaches. We have also clarified this under the environment paragraph of Section 6.

Oracle performance. Thank you for noting the anomaly in the Oracle baseline’s performance in the Blocks. We identified this is due to the suboptimal implementation of the place controller. We have fixed this issue and updated the results accordingly in Figure 4 and the corresponding sections of the manuscript.