AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents

We greatly appreciate your feedback. It's heartening to learn that you recognize our work's contributions of proposing the widely applicable idea of action and observation space optimization of large language model (LLM) agents, different from previous work's focus on designing other modules. Your positive remarks about the inspiration from our action optimization, akin to effective tool learning, are highly valued. We have updated the manuscript according to your feedback, and have added code for all experiments and new experiment trajectories to the supplementary materials. We organize our responses to your questions in the order they are presented and hope they address any concerns you may have.

AgentOccam's Domain Generalizabiliy

We are very delighted that you perceive the inherent generalizability of our proposed action and observation space alignment with LLM training to other agent tasks. However, our agent policy introduced in the paper is specifically applied to executing general-purpose web tasks, because in those tasks the agent's action/observation space deviates significantly from the LLMs' training data, and thus the task representation alignment could yield the most remarkable performance gain. Our methods proposed in Sections 4.1 and 4.2 are completely automated with deterministic rules, and we upload AgentOccam's code as the rebuttal supplementary materials in case you would like to play it in any web environment.

Additionally, we help conduct experiments on the general-purpose web task benchmark WebVoyager and compare AgentOccam's performance with the benchmarks' SOTA Agent-E. We kindly refer you to the general response section for the details. In short, AgentOccam performs better than Agent-E on the WebVoyager's definite-answer-subset. Specifically, both agents have their specificities. We find that Agent-E has strong performance on websites like Wolfram Alpha with more delicate plans issued by its "planner," and AgentOccam outperforms it on websites like Google Search and Hugging Face which need more accurate task interpretation and information retrieval. The impressive results across 10+ websites and many types of tasks can show AgentOccam's generalizability in the web domain.

Regarding how the principles of action and observation space alignment might help in other agent tasks, we provide a concrete example here for illustration.

The ALFWorld benchmark (https://arxiv.org/abs/2010.03768) includes text-based virtual household tasks. It requires the agent to reason over the text description of the house environment to complete the tasks, with an example as follows:

You are in the middle of a room. Looking quickly around you, you see a drawer 2, a shelf 5, a drawer 1, a shelf 4, a sidetable 1, a drawer 5, a shelf 6, a shelf 1, a shelf 9, a cabinet 2, a sofa 1, a cabinet 1, a shelf 3, a cabinet 3, a drawer 3, a shelf 11, a shelf 2, a shelf 10, a dresser 1, a shelf 12, a garbagecan 1, a armchair 1, a cabinet 4, a shelf 7, a shelf 8, a safe 1, and a drawer 4.  
Your task is to: put some vase in safe.

The original observation is represented in sequential texts, enumerating the items around. This implicitly assigns an order to the objects, defying our intuitive impression on the biologically plausible environment perception, which should be that all the sensings happen and affect our decision making simultaneously. This can be corrected by representing the observation in the form of structured code:

class Env():
	def __init__():
		self.drawer_idxs = {2, 1, 5, 3, 4}
		self.shelf_idxs = {5, 4, 6, 1, 9, 3, 11, 2, 10, 12, 7, 8}
		…
def househould_task():
	# define the function of putting the vase in the safe here.
	pass

Afterwards, we can prompt the LLMs to generate household procedures following the househould_task() functions defined.

As we didn't conduct experiments on agent benchmarks other than web interaction, we will avoid overclaiming the generalizability of AgentOccam to those tasks in the revised paper.

Alignment automation

We appreciate your insights regarding the automation of observation and action space alignment. We would like to clarify that our observation space alignment is realized with hard-coded rules that can be automatically applied to any web environment, which we believe is a wide enough range of applications. Our action space alignment follows the exploration-first and less-embodiment heuristics that improves agent performance significantly and thus will potentially inspire future alignment automation research.

To be more specific, when we are aligning the observation space, our input is the web page's DOM, which enables us to flexible merge or reshape elements based on their role (e.g., button, static text, table etc.), name (e.g., the text description of a button's functionality like submit or cancel, the content in a table cell, etc.), and other properties (e.g., focused, selected, etc.). All the procedures detailed in 4.2 "simplifying web page observations" can find a corresponding code snippet in the AgentOccam/obs_opt.py file that automates the observation processing. For example, merging the elements corresponds to the action_merge_statictext_to_parent, action_merge_menuitem_and_option etc. functions, and reformatting tables corresponds to the action_reformat_table function. Moreover, for the action space, our manually-designed alignment strategies are grounded in action statistics in Table 3 and the empirical action's exploration returns, which means we decide to retain or remove actions based on whether the action type encourages the agent to interact with the environment that leads to success, and whether it poses a challenge for the LLMs to interpret and use. These heuristics have been proven practical with experiment results, and we expect that they can guide action alignment automation principle designs in the future work.

We thank you for pointing out plenty of promising future directions (especially the web agent's evolving tool learning) in our spirit, and we want to highlight that AgentOccam can serve as the basic step one for many web agents of this kind, including using LLM to process and generate observations and actions.

Miscs.

If you have any additional questions or concerns, please let us know so we can resolve them before the discussion period concludes. Otherwise, it would be greatly appreciated if you could raise your score to show that the existing concerns have been addressed. Thank you!

Selectively Replaying Web Elements on a Page with Pivotal Nodes (Figure 4 Step 2) (Contd.)

AgentOccam's Workflow Regarding the Pivotal Nodes

0. The task started at the Google.com, with the observation to be:

RootWebArea 'Google'
	link [29] 'About'
	link [30] 'Store'
	link [277] 'Gmail'
	link [275] 'Search for Images'
	button [282] 'Google apps'
	link [152] 'Sign in'
	IframePresentational [153]
	search [6]
		combobox [12] 'Search' [required: False]
		button [294] 'Search by voice'
		button [295] 'Search by image'
		button [272] 'Google Search'
		button [273] "I'm Feeling Lucky"
	contentinfo
		link [83] 'Advertising'
		link [84] 'Business'
		link [85] 'How Search works'
		link [89] 'Our third decade of climate action: join us'
		link [93] 'Privacy'
		link [94] 'Terms'
		button [100] 'Settings'
			generic [102] 'Settings'

1. The agent typed the keyword into the search box and identified element 12 (combobox [12] 'Search' [required: False]) and 272 (button [272] 'Google Search') as the pivotal nodes. The web transit to a Google search page with the searched entries listed. Now, the agent would be prompted with task history and web observation for issuing the next action, where the task history was constructed based on the pivotal nodes selected by the agent previously, and any pivotal-node-associated nodes. We take the automatically-generated interaction history clip from the prompt at this step:

<step_0_interaction>
OBSERVATION:
RootWebArea 'Google'
	search
		combobox 'Search' [required: False]
		button 'Search by voice'
		button 'Search by image'
		button 'Google Search'
		button "I'm Feeling Lucky"
REASON FOR ACTION:
To progress towards finding the email address of the Dean of the School of Engineering at Stanford University, the first logical step is to search for relevant information. Using the search input box to enter a query such as "Dean of the School of Engineering Stanford University email" and then submitting it using the 'Google Search' button is a direct approach to gather this information.
ACTION:
type [12] [Dean of the School of Engineering Stanford University email] [1]
</step_0_interaction>

2. We can observe that the "pivotal node" technique helps keep a focused memory attention by only retaining the context of crucial information and omitting irrelevant noise. Based on the interaction history summary and current observation, the agent executed the following steps, with the history section constantly updated with the information from the ongoing steps till trajectory ended.

"Replaying Observation History Selectively" Clarifications

To prevent confusion, we separate the following explanations for the independent techniques of "selectively replaying web elements on a page" and "selectively replaying entire pages." We have updated the paper in Section 4.2 and in the appendix with these clarifications to make it a clearer version.

Selectively Replaying Web Elements on a Page with Pivotal Nodes (Figure 4 Step 2)

We will use how AgentOccam performed a simple customized task as an example for showing how we get the pivotal nodes and how those nodes help in selecting web elements for replaying.

Intuition and Background

We could view selectively replaying web elements on a page as a focused memory mechanism, where only the information that's relevant to the task would be recorded and replayed as the agent history traces. As all the web pages could be framed into a (DOM) tree structure, any web elements are nodes in this representation. We define pivotal nodes as the web elements, be it interactable or not, that are potentially useful for completing the task. Those pivotal nodes might present information (e.g., customer reviews in a review summary task) or the agent could interact with them to navigate to crucial states (e.g., the search box and the search button in a search task). We obtain these nodes at each step by prompting the agent to generate the page's highlights based on which they issue the action, or any web elements they will attend to if they fail at future steps and restore to the current state. After identifying the pivotal nodes with the agent's efforts, our code supports automatically parsing the web page's DOM tree and retaining nodes that are associated with the pivotal nodes (i.e., pivotal nodes' ancestor, sibling, and descendant nodes, as shown in Figure 4), to get a more succinct but less noisy version of the observation. This version would be used for constructing the agent's focused working history.

Prompt, Pivotal Node Section

Generate the response in the following format:
…
OBSERVATION HIGHLIGHT:
List the numerical ids of elements on the current webpage based on which you would issue your action. Also include elements on the current webpage you would attend to if you fail in the future and have to restore to this step. Don't include elements from the previous pages. Select elements at a higher hierarchical level if most their children nodes are considered crucial. Sort by relevance and potential values from high to low, and separate the ids with commas. E.g., `1321, 52, 756, 838`.

Example Task Objective

What is the email address of the Dean of the School of Engineering at Stanford University?

"Replaying Observation History Selectively" Clarifications (Contd.)

Selectively Replaying Entire Pages with the Planning Tree (Figure 4 Step 3)

We will use a planning tree generated by our agent during the development stage as an example for explaining how the planning tree takes shape and helps in selectively replaying past pages.

Intuition and Background

If we make an analogy of the single-thread web task completion to code execution, the agent's issuing sub plans mimic nested function calling, the active plan is like the uppermost function in the execution stack, and pruning the planning tree plays as popping failed functions off the stack. There constantly exists a planning tree and the planning tree operation instructions in the agent's prompt. We enable the agent to organize the planning tree with branch and prune commands, where the branch action creates new subplans, and the prune action restores the task progress to a previous state. It's noteworthy that those planning commands are in the same position as navigation prompts for the agent.

Prompt, Planning Tree Section

If you think you should refine the plan, use the following actions:
branch [parent_plan_id] [new_subplan_intent]: To create a new subplan based on PREVIOUS PLANS. Ensure the new subplan is connected to the appropriate parent plan by using its ID. E.g., `branch [12] [Navigate to the "Issue" page to check all the issues.]`
prune [resume_plan_id] [reason]: To return to a previous plan state when the current plan is deemed impractical. Enter the ID of the plan state you want to resume. E.g., `prune [5] [The current page lacks items "black speaker," prompting a return to the initial page to restart the item search.]`
Otherwise, use the following actions:
{navigation_specifications}

Example Task Objective

(WebArena 174) Open my latest updated issue that has keyword "feature" in its title to check if it is closed.

AgentOccam's Workflow Regarding the Planning Tree

0. At the beginning, we used the task objective as the root plan:

[0] (Active Plan) Find the solution to "Open my latest updated issue that has keyword "feature" in its title to check if it is closed"

1. The agent added a subplan to planning node 0 by issuing branch [0] [Navigate to the Issues page to search for the latest issue with the keyword "feature" in the title.]. Now the planning tree changed into:

[0] Find the solution to "Open my latest updated issue that has keyword "feature" in its title to check if it is closed"
	[1] (Active Plan) Navigate to the Issues page to search for the latest issue with the keyword "feature" in the title.

2. The agent navigated to the project's issue page.
3. The agent decomposed the plan by generating branch actions branch [1] [Search for the latest issue with the keyword "feature" in the title and check if it is closed.] and branch [1] [Open the latest issue with the keyword "feature" in the title.]. It performed navigation steps for each active plan before the next planning command was executed. In this example, after branch [1] [Search for the latest issue with the keyword "feature" in the title and check if it is closed.] was proposed, the active plan turned into "Search for the latest issue with the keyword "feature" in the title and check if it is closed." The agent then typed the keyword "feature" into the search box and sorted the issues by operating the sort icon before generating the next plan "Open the latest issue with the keyword "feature" in the title." In other words, all the navigation commands (e.g., search and sort) it issued were intended for the current active plan (e.g., search for the latest issue with the keyword "feature" in the title and check if it is closed). Just like a function call only needs to consider the function's scope, this allows the agent to only attend to the navigation actions dedicated to the active plan and the corresponding web observation as the playing history, which helps selectively replay past pages for the agent. Note that in this case, it assigned the two new sub plans to the same parent plan [1], which automatically shaped the planning tree's structure. Finally, the planning tree reformed into (the content enclosed in "[]" means comments, which is intended for illustration and didn't appear in the agent's prompt):

[0] Find the solution to "Open my latest updated issue that has keyword "feature" in its title to check if it is closed"
	[1] Navigate to the Issues page to search for the latest issue with the keyword "feature" in the title. [Plan's action scope: navigate.]
		[2] Search for the latest issue with the keyword "feature" in the title and check if it is closed. [Plan's action scope: search and sort.]
		[3] (Active Plan) Open the latest issue with the keyword "feature" in the title.

4. The agent executed the following steps to complete the task.

Dear Reviewer,

Thank you once again for your time and effort in reviewing our paper.

As the discussion period is nearing its end, we kindly remind you that there are two days remaining for any further comments or questions. We would be grateful for the chance to address any additional concerns before it concludes.

Thank you again!

Best regards,

Authors

Miscs.

We appreciate your suggestion of swapping the visual illustration of the ablation study with the corresponding result table in the appendix. We will incorporate this in the final version. If you have any additional questions or concerns, please let us know so we can resolve them before the discussion period concludes. Otherwise, it would be greatly appreciated if you could raise your score to show that the existing concerns have been addressed. Thank you!

AgentOccam's Technical Novelty

We thank you for recommending web IE papers released before LLMs came out, which could complete the research background of web page information retrieval, and potentially enlighten future web-agent observation space optimization work. We will include those papers in related work in a future version of our paper..

We would like to argue that in contrast to existing LLM-based agent designs, which emphasize compound agent policies as listed in Table 1, our AgentOccam is the first to propose aligning task representation with LLM training to enhance agents' task understanding and reasoning, with extensive experiments and ablation studies on web tasks, where the agent's action/observation space deviates significantly from the LLMs' training data, and thus the task representation alignment could yield the most remarkable performance gain. In short, our primary focus is on optimizing task representation for LLM-based web agents, which has the potential to inspire future work such as agent action selection automation and observation attention mechanisms. This marks the novelty of our work as the problem was overlooked by the LLM-based agent community, in 10+ existing web agent papers.

Previous web agent papers, which rely on additional modules, in context examples, task-specific strategies, offline data or online search, might need to adapt the agent's action and observation space to facilitate their policy designs, but their exploration is limited, or otherwise it cannot be explained that our AgentOccam outperforms them by a large margin without in context examples but with a consistent agent role and single LLM call at each step.

In addition, compared with static web information extraction (IE) in the pre-LLM era, the agent's action and observation space refinement poses new challenges, because i) Web IE works on extracting web attributes of interest, typically from the text field, the leaf node of the DOM tree, as implied in the paper you recommend (https://arxiv.org/abs/2101.02415), but the web agent is supposed to cater to all interactable/non interactable elements on a webpage, as it's required to not only extract relevant information but also dynamically interact with the web environment; and ii) LLM-based agent's observation space optimization covers more topics other than single-page refinement, such as agent planning and history organization, as those are included in prompt presented to the functioning LLMs. To address the above challenges, we propose to i) simplify web page elements by removing repetitive functionality- and layout-indicative tokens and reframing page content block in DOMs (not restricted to leaf nodes as in web IE), and replay observation history selectively with pivotal nodes and the planning tree, as detailed in Section 4. We also conduct extensive ablation studies to account for how each component of the action and observation space helps in agent task completion.

Thank you for your feedback. We are delighted to know that you recognize the contributions of our web agent's action and observation space alignment with the training of large language models (LLMs), and our introduction of branch and prune actions to enhance agent planning and task decomposition. We also appreciate that you conclude our experiments' key results and achievements. We would like to highlight that when we were designing and evaluating AgentOccam, the state-of-the-art (SOTA) method was SteP (https://arxiv.org/abs/2310.03720), and for a fair comparison with SOTA then, we reused their GPT model version (i.e., gpt-4-turbo). Additionally, it's noteworthy that all our experiments are conducted with gpt-4-turbo, whose fundamental performance is slightly worse than those versions the concurrent SOTA method use (i.e., gpt-4o for WebPilot (https://arxiv.org/abs/2408.15978)) (we refer to OpenAI's official model performance comparison: https://github.com/openai/simple-evals). We have updated the manuscript according to your feedback, and have added code for all experiments and new experiment trajectories to the supplementary materials. Your remaining questions are answered in sequence as raised.

AgentOccam is Generalizable to Real-world Websites

We kindly refer you to the general response section for the details. In short, AgentOccam performs better than Agent-E on the WebVoyager's definite-answer-subset. Specifically, both agents have their specificities. We find that Agent-E has strong performance on websites like Wolfram Alpha with more delicate plans issued by its "planner," and AgentOccam outperforms it on websites like Google Search and Hugging Face which need more accurate task interpretation and information retrieval. The impressive results across 10+ websites and many types of tasks can show AgentOccam's generalizability in the web domain.

AgentOccam is Generalizable to different LLM Families

In response to your question about AgentOccam’s LLM generalizability, we conduct the full set of ablation studies on a WebArena development subset with Gemini-1.5-flash (https://blog.google/technology/ai/google-gemini-ai/#sundar-note), a model trained with different data from the GPT model family. We also refer you to the general response section for the experiments' details. In conclusion, each alignment component proposed by AgentOccam proves beneficial to the agent system and can be generalized to different LLMs.

Dear Reviewer,

Thank you once again for your time and effort in reviewing our paper.

As the discussion period is nearing its end, we kindly remind you that there are two days remaining for any further comments or questions. We would be grateful for the chance to address any additional concerns before it concludes.

Thank you again!

Best regards,

Authors

We appreciate your insightful feedback and suggestions. We are glad to read that you find our alignment of observation and action space between web tasks and the training of large language models (LLMs) to be sound, pertinent to the current web agent literature, and contributing to the LLM-agent community. We have updated the manuscript according to your feedback, and have added code for all experiments and new experiment trajectories to the supplementary materials. We arrange our responses in the order your questions are raised, and hope they clarify your concerns.

AgentOccam's Performance on Real-world Websites (Benchmark WebVoyager)

As per your suggestion, we conduct experiments on the general-purpose web task benchmark WebVoyager and compare AgentOccam's performance with the benchmarks' state-of-the-art (SOTA) Agent-E. We kindly refer you to the general response section for the details. In short, AgentOccam performs better than Agent-E on the WebVoyager's definite-answer-subset. Specifically, both agents have their specificities. We find that Agent-E has strong performance on websites like Wolfram Alpha with more delicate plans issued by its "planner," and AgentOccam outperforms it on websites like Google Search and Hugging Face which need more accurate task interpretation and information retrieval. The impressive results across 10+ websites and many types of tasks can show AgentOccam's generalizability in the web domain.

Miscs.

Thank you for carefully checking our paper and pointing out the typo. We will fix it in the later versions. If you have any additional questions or concerns, please let us know so we can resolve them before the discussion period concludes. Otherwise, it would be greatly appreciated if you could raise your score to show that the existing concerns have been addressed. Thank you!

Planning Tree Clarifications

We will use a planning tree generated by our agent during the development stage as an example for explaining how the planning tree takes shape and helps in selectively replaying past pages. We have updated the paper in Section 4.2 and in the appendix with these clarifications to make it a clearer version.

Intuition and Background

If we make an analogy of the single-thread web task completion to code execution, the agent's issuing sub plans mimic nested function calling, the active plan is akin to the uppermost function in the execution stack, and pruning the planning tree plays involves popping the failed functions off the stack. There constantly exists a planning tree and the planning tree operation instructions in the agent's prompt. We enable the agent to organize the planning tree with branch and prune commands, where the branch action creates new subplans, and the prune action restores the task progress to a previous state. It's noteworthy that those planning commands are in the same position as navigation prompts for the agent.

Prompt, Planning Tree Section

If you think you should refine the plan, use the following actions:
branch [parent_plan_id] [new_subplan_intent]: To create a new subplan based on PREVIOUS PLANS. Ensure the new subplan is connected to the appropriate parent plan by using its ID. E.g., `branch [12] [Navigate to the "Issue" page to check all the issues.]`
prune [resume_plan_id] [reason]: To return to a previous plan state when the current plan is deemed impractical. Enter the ID of the plan state you want to resume. E.g., `prune [5] [The current page lacks items "black speaker," prompting a return to the initial page to restart the item search.]`
Otherwise, use the following actions:
{navigation_specifications}

Example Task Objective

(WebArena 174) Open my latest updated issue that has keyword "feature" in its title to check if it is closed.

AgentOccam's Workflow Regarding the Planning Tree

0. At the beginning, we used the task objective as the root plan:

[0] (Active Plan) Find the solution to "Open my latest updated issue that has keyword "feature" in its title to check if it is closed"

1. The agent added a subplan to planning node 0 by issuing branch [0] [Navigate to the Issues page to search for the latest issue with the keyword "feature" in the title.]. Now the planning tree changed into:

[0] Find the solution to "Open my latest updated issue that has keyword "feature" in its title to check if it is closed"
	[1] (Active Plan) Navigate to the Issues page to search for the latest issue with the keyword "feature" in the title.

2. The agent navigated to the project's issue page.
3. The agent decomposed the plan by generating branch actions branch [1] [Search for the latest issue with the keyword "feature" in the title and check if it is closed.] and branch [1] [Open the latest issue with the keyword "feature" in the title.]. It performed navigation steps for each active plan before the next planning command was executed. In this example, after branch [1] [Search for the latest issue with the keyword "feature" in the title and check if it is closed.] was proposed, the active plan turned into "Search for the latest issue with the keyword "feature" in the title and check if it is closed." The agent then typed the keyword "feature" into the search box and sorted the issues by operating the sort icon before generating the next plan "Open the latest issue with the keyword "feature" in the title." In other words, all the navigation commands (e.g., search and sort) it issued were intended for the current active plan (e.g., search for the latest issue with the keyword "feature" in the title and check if it is closed). Just like a function call only needs to consider the function's scope, this allows the agent to only attend to the navigation actions dedicated to the active plan and the corresponding web observation as the playing history, which helps selectively replay past pages for the agent. Note that in this case, it assigned the two new sub plans to the same parent plan [1], which automatically shaped the planning tree's structure. Finally, the planning tree reformed into (the content enclosed in "[]" means comments, which is intended for illustration and didn't appear in the agent's prompt):

[0] Find the solution to "Open my latest updated issue that has keyword "feature" in its title to check if it is closed"
	[1] Navigate to the Issues page to search for the latest issue with the keyword "feature" in the title. [Plan's action scope: navigate.]
		[2] Search for the latest issue with the keyword "feature" in the title and check if it is closed. [Plan's action scope: search and sort.]
		[3] (Active Plan) Open the latest issue with the keyword "feature" in the title.

4. The agent executed the following steps to complete the task.

A Detailed Comparison of AgentOccam and Agent-E

We appreciate that you bring Agent-E to our attention. We will include the discussion about this paper in the later versions, as well as the WebVoyager result table to show AgentOccam's web environment generalizability. We want to emphasize that while Agent-E might look similar to AgentOccam, there exists differences in those two agents' policy construction.

First, Agent-E designs a planner-browser-reflector agent architecture on top of the system, and like many previous work, any compound agent architecture is susceptible to error propagation, which means the former module's erroneous behaviors would affect later performance (ii and iv). Though Agent-E cleverly finds a balance between the planner, browser, and reflector role by strategic prompting, many of its failures are induced by those agent roles' poor collaboration. For example, in Google Search task "Find the software requirements for iPhones that support AirDrop's ability to continue transmitting over the web when out of range." (Google Search--7), the planner starts by making a plan that requires the browser to navigate to Apple.com and find the answer there, complicating the problem and leading to failure. In contrast, AgentOccam simply types an appropriate query into the search field and finds the answer. Another example is the Cambridge Dictionary task "Search for the word 'sustainability' on the Cambridge Dictionary, what is the translation of sustainability into Chinese and French in the dictionary." (Cambridge Dictionary--6), where the reflector mistakes the Spanish translation of "sustainability" as its French translation, misleading the following agent roles' interaction and causing failure. Additionally, there are cases where the reflector accidentally terminates the trajectory when it wrongly thinks the task has ended. In summary, as different agent roles have varied responsibilities (e.g., reflector needs to provide verbal feedback) and control (e.g., planner can delegate navigation task to browser) over the trajectory flow, different roles' prompting biases are prone to induce specific misconduct modes, which will negatively affect the task completion sequentially and accumulatively. On the contrary, we keep AgentOccam's policy simplicity by using the same agent role (i.e., agent prompts) throughout the play, which, based on the empirical results, has the minimum misleading role-prompting biases compared to compound agent architectures.

Second, AgentOccam and Agent-E are concurrent works and propose or implicitly follow many similar LLM-agent refinement principles inspired by those that have been proven useful in classical robotics learning (i and iii). Notably, AgentOccam introduces the concept of action alignment that enhances agent navigation focus by only retaining essential actions (i.e., click, type, note, stop, and go_back) in the agent action space, with each action space refinement component being shown helpful by ablation studies. Similarly, Agent-E pre-defines its submodule browser's skills to be click, enter text, open url, and press key (e.g., submit, pagedown). Moreover, AgentOccam proposes observation space alignment that revises the web environment observation to be more succinct (e.g., removing repetitive functionality- and layout-indicative tokens, reframing page content block in DOMs, organizing history with pivotal nodes and the planning tree) but as informative, turning out to both downsize the LLM consumed tokens and improve agent's performance by trajectory statistics and ablation studies. Likewise, Agent-E allows its browser to select from three types of DOMs (i.e., text-only, input fields, and all fields) that best suit the task. With Agent-E setting SOTA on WebVoyager and AgentOccam excelling on both WebArena and the WebVoyager subset, it follows that shared principles—specifically, action and observation space adaptations to the agent architecture design—enhance LLMs' understanding and reasoning on web tasks.

Dear Reviewer,

Thank you once again for your time and effort in reviewing our paper.

As the discussion period is nearing its end, we kindly remind you that there are two days remaining for any further comments or questions. We would be grateful for the chance to address any additional concerns before it concludes.

Thank you again!

Best regards,

Authors

AgentOccam is Generalizable to different LLM Families

We conduct the full set of ablation studies on a WebArena development subset with Gemini-1.5-flash (https://blog.google/technology/ai/google-gemini-ai/#sundar-note), a model trained with different data from the GPT model family.

Due to the time and cost constraints, we construct a representative subset from the original 812 tasks in WebArena. Specifically, we sample one task from each task cluster instantiated with the same intent template (e.g., “What is the top-{{n}} best-selling product in {{year}}”, where “{{n}}” and “{{year}}” would be replaced by instantiation tokens) in WebArena, forming a development set with 190 tasks. We use Gemini-1.5-flash for all experiments. We run each task one time and restart the experiment if the WebArena simulator fails (i.e., login expires, reddit post limit exceeds, map malfunctioning, etc.). We include the code and trajectory logs in the supplementary materials. The results are as follows, with the performance from the GPT-4-turbo counterpart on the same development task set for comparison:

The main action statistics:

AgentOccam is Generalizable to different LLM Families (Contd.)

The average observation tokens per step:

The average number of steps per task:

We can observe a similar trend with the Gemini model that each alignment component introduced by our paper contributes to the web agent’s overall performance. To be specific, removing non-essential actions (↓ Actions) encourages the agent to explore more actively with commands click and type; disabling scrolling (Above + X Scrolling) proves advantageous in tasks where key information is not presented on the first browser sight; simplifying web page elements (Above + Obs Opt.) reduces the observation token number; selectively replaying web element in one page (Above + History) reduces the steps required to accomplish the task; and planning and selectively replay past pages (AgentOccam) enables the agent to self-organize task workflow and quickly restore to a previous stage after several failed attempts. In conclusion, each alignment component proposed by AgentOccam proves beneficial to the agent system and can generalize to different LLMs.

As a general comment, your evaluation between Agent-E and your results are not a fair comparison for two reasons:

• You are evaluating only on a subset as you mentioned.
• The subset that you are measuring is the ones with "golden" subset that seemingly has deterministic answer. However, that assumption is wrong, for example, one of the ESPN task is (ID: ESPN--17) : "Check out the NBA Basketball Power Index 2023-24 to see which teams are in first place and which are in last place." The "golden answer" is "Boston Celtics; San Antonio Spurs". However, this is not true (https://www.espn.com/nba/bpi/_/season/2024) and there are 10s of such "wrong" golden answers. When evaluating in the above comparison table, for example, Agent-E performs poorly on ESPN in comparison to AgentOccam, but it would have been considered wrong according to this golden answer. Hence, a fair comparison would be with a full set and using manual annotation. These results should not be published until such a rigorous evaluation is performed.

Thank you for the additional comments. We respectfully disagree that the comparison is unfair; please find our arguments below. Additionally, on the reviewer’s insistence, we performed a thorough manual evaluation, the process and results for the same are included below as well.

On Selection of 129 Tasks

For benchmarks with a large volume of tasks, it is a common practice to test a statistically significant subset of tasks for assessment efficiency. For instance, VisualAgentBench introduced WebArena-Lite, and WorkArena also selected a limited set of tasks for testing on their proposed benchmark. Our selection of 129 subtasks is entirely based on the golden-answer labels from the WebVoyager benchmark dataset, as detailed in their dataset repository. This categorization and labels are not introduced by us, thus eliminating any subjective task selection bias or unfairness to either our or the baseline method. We chose to evaluate on this subset because of a shortage of time during the rebuttal phase. We would be happy to include results on all the 643 tasks for the camera ready.

On Incorrect Evaluators in WebVoyager

Firstly, we would like to point out that any issues with the benchmark's labels would likely affect all agents' performance evaluations similarly, making the success rate figures a statistically fair reflection of agent performance. However, in the spirit of a thorough evaluation, we performed manual labeling of trajectories from the 129 tasks with the help of three different human annotators. We observe that all human assessments show improvements in the performance of both agents compared to hard-coded evaluators (with improvements not exceeding 5%), with AgentOccam consistently leading by a slight margin. The labeling details are described below.

In summary, we argue that our evaluation supports a statistically significant "fair comparison." We want to clarify that we did not directly compare with the results from the Agent-E paper. Instead, we replicated the method on the same subset, using the same reference answers and evaluators/annotators as assessing our method. As noted above, neither subset selection nor errors from the original benchmark are introduced or leveraged by either agent policy. Therefore, they should be orthogonal to the agent performance comparison and pose no bias towards either method.

Labeling Details:

We asked three human labelers to mark the correctness of trajectories, two of whom (Alice and Bob) had no prior involvement in web agent projects. We provided them with the WebVoyager questions, the start URLs, and the responses to the WebVoyager questions by Agent-E and AgentOccam (with the agents' identities concealed), asking them to give a binary success/failure evaluation and their answers if they were to respond. They were also provided the reference answer link from WebVoyager and warned that the answers might contain errors. To avoid biasing their choice with the agent's identity, for questions difficult to judge based solely on the responses, we only provided details of the agent trajectory logs at the end, as needed by the human labelers. We would like to, but we cannot add more to the supplementary materials at this moment due to the rebuttal schedule. We will attach the assessment files to the supplementary materials once policy allows. The evaluation statistics are detailed as follows:

The overall impressions of the two web agents by the three reviewers are copied below (for ease of reading, we have replaced references to Agent A and Agent B with their actual identities):

Alice: They make quite endearing mistakes. Overall, AgentOccam seems better because, although it often claims it cannot find an answer and makes some humorous errors, Agent-E returns incorrect answers.

Bob: The most direct feeling is that AgentOccam has less redundant information. Agent-E always seems to give everything it knows to avoid errors, like in a question that only asked for a number, Agent-E, as if not understanding, still listed a lot.

Colin: Agent-E's answers tend to be longer, while AgentOccam's are more concise. Agent-E's answers require careful examination as they might contain errors, whereas AgentOccam simply states it does not know.

Note: AgentOccam's statement that it cannot find an answer is due to us applying the same configuration from WebArena, which limits the step count to 20 steps; this limitation does not affect Agent-E's performance. Observing some of AgentOccam's failed trajectories reveals that tasks are interrupted midway toward success due to the step limit. If required for the camera-ready version to complete the WebVoyager benchmark, we will rerun these 129 experiments, relaxing AgentOccam's (ours) step limit.