Thank you for highlighting the importance of comprehensive evaluations. While we agree that additional datasets could help assess generalization capabilities, we want to emphasize that ALFWorld and its extended dataset, ALFWorld-Afford, are particularly well-suited to our experimental setup. We need the benchmark to have a large number of objects, actions, and affordance types to effectively evaluate our approach. ALFWorld possesses several critical characteristics that make it an ideal open-world environment for embodied agent research. First, it features partial observability, where the agent operates with an egocentric view and must actively explore to discover new objects. This mirrors real-world scenarios where agents do not have complete knowledge upfront. Additionally, the environment includes 125 objects and 6 binary affordance actions, requiring reasoning over 126×2×6=93,750 potential affordances. This level of complexity aligns directly with the focus of our work, which emphasizes affordance reasoning for open-world planning. To further strengthen the benchmark, we introduced the ALFWorld-Afford dataset. This extension increases task complexity, expands the diversity of affordances, and incorporates multi-affordance tasks, thereby providing a more robust and comprehensive evaluation framework for affordance reasoning in open-world scenarios.

“We assume observation predicates \mathcal{P}_o are given but not affordance predicates \mathcal{P}_a.” Why? why can we ground \mathcal{P}_o—for instance, p(hold(potato_1) | image)—but not p(canHeatStuff(microwave_1) | image)?

For embodied agents that have a symbolic model, we assume they must possess the ability to convert observations to symbolic forms. For example, the agent needs to identify any object and relationship within a goal for a symbolic model to reason about. If the goal is "Give me a cleaned bowl," the agent can represent it as "isBowl(?o) and isCleaned(?o)." However, defining affordance knowledge cannot often be reasoned by observation alone. Knowledge such as affordances can be learned via prior experiences, through a manually defined database or in our case using LLM common sense reasoning. Defining affordance knowledge manually or learning such knowledge via past experience can be challenging when the number of potential objects is large. A single binary affordance can contain N^2 possible combinations for N objects. For instance, ALFWorld features 125 distinct objects and 6 affordances predicates, which require the agent to reason over a combinatorial space of 125 ^ 2 x 6=93,750 potential affordances.

If \mathcal{P}_o, which is the set of grounded propositions, is already given, then we don’t have any unknown objects?

\mathcal{P}_o does not contain all possible grounded propositions; instead, it represents a small subset that we assume the agent can ground through observation (as illustrated in the previous examples). When the agent encounters a new object that cannot be grounded through observation, we leverage a large language model (LLM) and plans generated via lifted regression to determine whether the object is relevant to the agent's goal.

I don’t understand how it is possible to get better results than the LLM-Affordance+FD baseline. Both the proposed method and this baseline use the affordance information generated from the LLM. So if I get this correct, the only difference in results should possibly be the planning times, since the FD should get correct plans if the operators are correct. The only way we get a difference in these results means that the domain definition should be different, which in turn means operators and/or the used propositions are different, which would further mean that the affordance-related propositions generated by the LLM are not consistent in these two approaches, LLM-regress and FD+LLM Afford.

While both LLM-Regress and FD+LLM Afford both rely on LLMs to generate object affordances, the methods for generating these affordances differ significantly. FD+LLM Afford employs a grounded forward search that requires complete knowledge of the domain to generate feasible plans. In our experiments, the LLM was provided with a complete list of objects to generate affordances for FD+LLM Afford, which need to generate 15,624 grounded affordances for a single action. This exhaustive approach often caused the LLM to struggle to produce feasible domains, resulting in frequent incorrect affordances and syntax errors.

In contrast, LLM-Regress uses regression planning to identify the affordance types only relevant to the goal. LLM based affordance reasoning happens when new objects are discovered, which significantly reduces the number of affordances that need to be generated. By narrowing the scope of affordances and objects, LLM-Regress avoids the combinatorial explosion faced by FD+LLM Afford, leading to a more focused and efficient process.

This efficiency is the primary reason LLM-Regress consistently outperformed FD+LLM Afford in our experiments. A detailed comparison of both methods, along with examples, is included in the Appendix.

references

[1] Liu, Yongmei, and Gerhard Lakemeyer. "On first-order definability and computability of progression for local-effect actions and beyond." Schloss-Dagstuhl-Leibniz Zentrum für Informatik, 2010.

We thank the reviewer for the insightful suggestions and criticisms, we want to use this opportunity to better explain some of the key concept and design choices we made in the paper.

“LLMs do not require structured inputs or explicit knowledge modeling, which theoretically makes them well-suited for open-world planning” I don’t quite get why this would make LLMs theoretically well-suited.

Thank you for highlighting the inappropriate use of the word "theoretical." A more suitable term would be "technically." The rationale behind this is that LLMs process natural language as both input and output. Therefore, as long as observations and actions can be translated into natural language, LLMs are capable of reasoning about unknown objects and relationships that were not previously specified. We will update the wording accordingly in the final draft of the paper.

”However, they require complete problem descriptions, which are not readily available in open-world scenarios.” While classical planning guarantees completeness and consistency in its solutions, it requires detailed domain descriptions, which may be unrealistic in real-world settings. There are some works that focus on learning abstractions to be used for planning [1-8].

Thank you for sharing the related works on learning abstractions for planning. We agree that these are very important contributions in model learning, and we have included a discussion of these works in the related work section. However, we aim to distinguish our approach from methods that focus on learning abstract models through past data. Our work focuses on tasks where the agent’s action model is provided but lacks information about object affordances. This mirrors robotic tasks where a robot is equipped with a set of predefined skills but does not have complete knowledge about the environment or the objects it contains.

”Our approach provides complete and verifiable plans, guaranteeing the discovery of a feasible plan if one exists.” Complete with respect to the affordance performance of LLMs?

While LLMs may bottleneck our model's performance in terms of affordance reasoning, the completeness we refer to is First-Order (FO) completeness. This means our approach can generate a complete lifted plan if one exists, a property that does not hold for FO progression-style methods [1].

The definition of \theta_o and \theta_a—grounding functions—are vague. It’s not clear what kind of input they take and output as a result. I can only guess.

\theta_o takes a set of object name strings and grounds them according to predefined goal predicates. For example, if the agent has the goal "pick up a plate" and is given the natural language description of the environment, "you are in a kitchen and you see a microwave, plate, and potato on the table,". We can use a parser of LLM to parse a list of objects O = {microwave_1, plate_1, potato_1}. \theta_o grounded O into goal-related predicates P_o = {isHolding(?o), isPlate(?o)}. Theta_o(O) = {isPlate(plate_1)}.

“The main affordances reasoning in this environment include determining whether an object can be heated, cooled, cleaned, or turned on.” This seems like pre-defining affordance classes.

The affordance types are defined under the assumption that the agent has access to a set of known action schemas. In this work, we define these actions using a PDDL-like syntax. The preconditions of these actions include affordance predicates, which determine whether an action can be applied to specific objects. This framework reflects the nature of embodied agent tasks, where the agent operates with a predefined set of high-level skills.

I believe one implicit assumption is that \theta_o should also output the labels of objects, e.g., egg_1, microwave_2, etc. Does this mean that there is a pre-defined set of object names?

We assume that \mathcal{P}_o​ is given because under our setup, the agent is capable of representing goals in symbolic form. For example, if the goal is "Give me a cleaned bowl," the agent can represent it symbolically as isBowl(?o)∧isCleaned(?o). When the agent encounters a new object in a room, it can translate its observation into text and extract the object's name. Since we use name strings for reasoning with a large language model (LLM), there is no need for a predefined set of object types. For instance, if the agent receives a natural language description of the room such as, "There is a table, chair, fridge, and XYZ in the room," it stores the set of names {’table’,’fridge’,XYZ’ }as objects. Using the LLM, the agent can reason about action affordances such as canCool(?r,potato) by grounding these names in \mathcal{P}_o​. For example, we can use LLM to evaluate canCool(fridge,potato) or canCool(XYZ,potato).

We thank the reviewer for the insightful suggestions and criticisms, we want to use this opportunity to better explain some of the key concept and design choices we made in the paper.

The paper emphasizes that this method is for “open-world planning,” but it is unclear to me how this approach relates to open-world concepts. How does the paper define “open-world?”

We adopt the formal logic definition of "open-world," where "the facts asserted in a model are not assumed to be complete" [3]. In our setup, this concept translates to assuming that certain elements of the model (such as object types, locations, and affordances) are incomplete or unknown. In our setup, we assume knowledge of the agent's action models in a symbolic form. This mirrors many real-world robotic tasks, where robots are typically programmed with a predefined set of skills but lack comprehensive knowledge of their environment, including the objects they interact with, their locations, and the affordances required for every task. By addressing this partial knowledge scenario, our approach aligns with the principles of open-world reasoning while tackling practical challenges in robotic planning.

Is the evaluation environment used, ALFWorld, truly an open-world environment?

ALFWorld exhibits several characteristics that make it an open-world environment well-suited for embodied agent research. First, it incorporates partial observability: the agent operates with an egocentric view and must actively explore its surroundings to discover objects, reflecting the incomplete knowledge typical of open-world settings. Furthermore, the environment features 125 distinct objects and 6 binary affordance actions (e.g., canHeat(toaster, egg), canCool(fridge, bread)), requiring reasoning over a combinatorial space of 125×6×2=93,750 potential affordances. This complexity underscores the open-world nature of ALFWorld, as the agent must interact with objects in a partially observable environment with incomplete knowledge.

In my view, open-world scenarios often imply that the objects or predicates within the environment are unknown or theoretically infinite. How would this method function in such a scenario, and what would the complexity be?

It is an intriguing prospect to build systems that can act in an infinitely large environment with no prior knowledge of any objects and relationships. The method we proposed in the paper is more suitable for high-level reasoning for any agent than other formal methods. The key benefit of lifted regression is that we produce plans containing objects and predicates that are relevant to a goal. Also, since the plan we produce is in a lifted form, our method only focuses on objects that satisfy certain properties (which can be reasoned about using LLMs). As a result, our method greatly reduces the number of objects and predicates the agent needs to reason about. This is in contrast to grounded forward search methods, where the agent needs to plan over all possible objects in the environment. (A benefit of our approach is highlighted with SayCan[2] in the next two questions.)

What is “grounding”? The paper lacks a clear definition.

The grounding we refer to is the first-order logic definition of grounding: “Grounding is the task of taking a problem specification, together with an instance, and producing a variable-free first-order formula representing the solutions to the instance" [4]. In the context of ALFWorld, it is the process of determining which objects the agent has seen to make a subgoal true. For example, the agent sees a set of objects {apple, microwave, plate}, and the agent needs to determine which object can ground so the predicate canHeat(?x, ?y) to make it true. In grounded forward search methods, actions also need to be grounded first before planning. For example, the action pick(?x) with 500 objects will produce 500 grounded actions {pickup(apple), pickup(plate), ...}. We will use SayCan as an example in the next question to contrast it with our approach.

We thank the reviewer for the insightful suggestions and criticisms, we want to use this opportunity to better explain some of the key concept and design choices we made in the paper.

More benchmarks should be adopted. Albeit the reputation of ALFWorld in planning and LLM agents, it is not very "open-world" as the number of objects, functionalities, tasks, etc, can be relatively limited.

We thank the reviewer for the insightful comment. However, in this work, we focus on the formal logic definition of "open-world," where "the facts asserted in a model are not assumed to be complete" [1]. In our setup, we assume the object types, locations and affordance of these objects are incomplete parts of our model. ALFWorld demonstrates several key characteristics that make it an open-world environment suitable for embodied agent research. First, it features partial observability, where the agent operates with an egocentric view and must explore to discover new objects. This setup mirrors real-world scenarios where agents lack complete knowledge upfront. More importantly, the environment includes 125 objects and 6 binary affordance actions(i.e canHeat(toaster, egg)), requiring reasoning over 125 ^ 2 * 6 =93,750 potential affordances. This complexity aligns with the focus of our work, which centers on affordance reasoning for open-world planning. To further enhance the benchmark, we introduced the ALFWorld-Afford dataset, which increases task complexity, expands affordance diversity, and incorporates multi-affordance tasks, providing a more robust evaluation framework for affordance reasoning.

My suggestion is to extend to some truly open-world environments, ex. Minecraft. It adds more complexity as the environment is more volatile and necessitates much more background knowledge. I would like to see some results on this benchmark.

We thank the reviewer for pointing out the comparison, we added [2] in the related work section. While Minecraft[2] offers a rich environment with diverse objects types, it is less suitable for our focus on affordance reasoning. Minecraft’s primary affordances revolve around mining and crafting actions. Mining assumes tools can mine predefined objects(that are domain specific to Minecraft), which limits the need for affordances reasoning. Similarly, crafting relies on recipes that specify exact item combinations which is also specific to the Minecraft domain. This focus on domain specific reasoning which contrasts with ALFWorld's household setup that mimic real-world senarios. We recognize Minecraft’s potential to complement our work in multi-modal multi-modal reasoning, enabling us to explore the integration of LLMs with visual and textual inputs. These extensions would allow us to evaluate our framework in richer and more complex open-world scenarios.

More baselines, especially LLM planning methods should be compared. Notably, the community has already noticed that straightforward planning with LLMs does not usually work very well and several remedies have been proposed: interactive planning with self-check or reflections[1], adding memory as a means of learning[2], retrieval-augmented planning with LLMs[3], etc. I think [1] also has results on ALFWorld. A more comprehensive comparison of these approaches could help strengthen the merit of the proposed method

We thank the reviewer for highlighting these related works. We also included a discussion about memory-based methods, such as those in [3,4], in the related work section. We also included them in the discussion and incorporate the reported ALFWorld results for comparison in our revised manuscript. DEPS shares a similar framework with the ReAct methods used as baselines, as both adopt a think-plan-reflect loop. The use of external memory as a mechanism to learn or reason with LLMs is an intriguing and important approach for enhancing LLMs' planning capabilities. However, our method focuses on model-based reasoning, which does not rely on external memory, making it not directly comparable to memory-based methods. That said, we believe extending our work to incorporate the extraction of common-sense knowledge from past data presents an intriguing direction for future research. This could potentially bridge the gap between model-based reasoning and memory-augmented approaches, further improving planning capabilities in open-world environments.

Citation

[1] Computer and information security handbook. Newnes, 2012.

[2] "Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents." arXiv preprint arXiv:2302.01560 (2023).

[3] "Jarvis-1: Open-world multi-task agents with memory-augmented multimodal language models." arXiv preprint arXiv:2311.05997 (2023).

[4] "Rat: Retrieval augmented thoughts elicit context-aware reasoning in long-horizon generation." arXiv preprint arXiv:2403.05313 (2024).