Thank you for citing the importance and novelty of our idea, as well as the detailed and constructive suggestions!

Quality: Ablating Skill Verification and Episode Cleaning in ASI

In Section 3.3 (Table 4), we have ablated the critical skill verification component, where unverified induction yields a 6.4% lower success rate (SR), showing the benefit of doing skill verification.

We also appreciate your suggestion on ablating on the data processing (episode cleaning) step. We additionally experiment with AWM and ASI without episode cleaning, which get 35.5 (AWM) and 38.5 (ASI) SRs. In general, episode cleaning improves AWM and ASI by 0.8% and 1.9%. Nonetheless, episode cleaning is not necessary for AWM/ASI to work, as they still substantially outperform vanilla baseline by 2.8 and 5.8 points, while bringing bonus benefits such as efficiency because cleaned episodes have fewer tokens.

Quality: Statistical Analysis of ASI-AWM

Thank you for your suggestion. We conduct paired t-testing between AWM and ASI results, and get t-stats of -2.21 and p-value of 0.027. both |t|>2 and p<0.05 justify that ASI performs statistically significantly better than AWM. We would like to note that WebArena enforces a particular order such that earlier tasks do not interfere with later tasks, thus we cannot flexibly change example ordering. So we conduct all experiments following the original WebArena order. Further, AWM’s ablation on the Mind2Web dataset shows that inducing reusable skills can be robust to example ordering, which could be extrapolated to our program skill setup. Nonetheless, we do agree that example ordering is an interesting angle to analyze the ASI method in future works.

Clarity: concepts and descriptions

Thank you for your detailed read and feedback!

For concepts, thank you for your help clarifying. Yes, your understanding was largely correct, and here are our updated definitions:

• task: An NL instruction that the agent attempts to accomplish.
• action: an executable program function, this can be pre-defined (e.g., the 'click' primitive), or later induced high-level skills (e.g., search_product).
• step: refers to a discrete timestamp during the task-solving process, for step at time t, this step is associated with the observed state $s_t$, the action taken $a_t$, and the state after action execution $s_{t+1}$.
• skill: correct. Further for program-skill and text-skill, semantically they contain the same information, just present it in different formats.
• function: this is just the general meaning of a program function, not a specialized term.

Lines 139-140: A program function here means a function written in a Python program. We add "executable" to emphasize the fact that programs can be executed (thus verified), in contrast to the baseline AWM method using "non-executable" texts.

Lines 132-138: We write a script to automatically clean the input episodes. The browsergym platform returns the error message if an action is not correctly executed on the environment. The chains-of-thought are very long (average 87.9 tokens) and redundant (see example below), which may interfere with the induction process/quality, so our script leverages an LLM to summarize them for both quality and efficiency considerations (reduced length by 6.6x).

Hope this helps clarify the terminology! We will add these clarifications in the revised paper.

Originality

The mentioned SkillWeaver is a concurrent (later) work, so we do not think that it affects the originality of our work. Furthermore, SkillWeaver creates APIs offline, which means they have access to supervision signals, a costly learning process from excessive offline episodes, and potentially suffer from offline-online task mismatches. In contrast, we do not require additional offline compute and run directly online, thus we are supervision-free, efficient learning with just the online agent episodes, and distributionally match the test queries.

Thank you for recognizing the importance of our tackled problem, and ASI’s effectiveness in improving agent correctness and efficiency.

R1: More analysis of ASI failure cases

In cross-web generation tests, we showcased one of the most common failure cases of ASI in Figure 4 – a skill induced on one website is incompatible with the design of other websites. For in-domain web activities, in both basic (sec3) and scaled-up (sec 4) scenarios, the main bottleneck of ASI is skill program quality, which we attempted to guarantee via the method described in section 2.3. We will add more failure case examples to the appendix of the revised version.

Q1: Is skill induction chaining existing skills?

Not necessarily. Although many skills are sequentially executing actions (i.e., existing skills), some skills leverage more program primitives (e.g., if-else, for/while loops), such as:

def browse_category_by_navigation(menu_id_sequence: list):
   """Browse products by navigating through a sequence of menu IDs.
   This function allows navigation through a series of menu interactions.
   Args:
       menu_id_sequence (list): A list of menu IDs to interact sequentially, using hover actions followed by a click.
   Example usage:
       browse_category_by_navigation(['735', '786', '797']) # Navigates Home & Kitchen -> Storage & Organization -> Racks, Shelves & Drawers
   """
   for idx, menu_id in enumerate(menu_id_sequence[:-1]):
       hover(menu_id)
   click(menu_id_sequence[-1])  # Click the final id to land in the predefined category

def navigate_and_sort_category(category_id: str, subcategory_id: str | None = None):
   """Navigate to a product category and sort items by price.
   Args:
       category_id: ID of the main category link
       subcategory_id: Optional ID of the subcategory link, if needed
   Returns:
       None
   Examples:
       navigate_and_sort_category('1241', '1873')  # For PS4 under Video Games
       navigate_and_sort_category('1245')  # For main category only
   """
   click(category_id)  # Click main category
   if subcategory_id:
       click(subcategory_id)  # Click subcategory if provided
   select_option("1553", "Price")  # Sort by price ascending

We are happy to incorporate these examples into the revised paper.

Thank you for your recognition of the importance of our method, as well as its effectiveness in accuracy and efficiency.

R1: Position relative to prior work

Compared to AWM, the main difference of ASI is representing skills as programs, thus (i) integrating skills into agent action space instead of memory, and (ii) enabling execution-based verification to improve the quality of induced skills. According to our results in Table 1, this design change brings significant success rate increases by another 11.3%; further, it boosts agent efficiency by up to 85.2% compared to AWM in long-horizon web activities (Table 1,4,6).

R2: Details about Scaled-Up and Cross-Domain Settings

1. How tasks are selected: We create tasks of two scenarios by extending WebArena examples: For scaled-up we take "search for coffee product" in WebArena and expand it to "search for coffee, mug, and coffee machine" (as exemplified in Table8), in a natural way that search for multiple relevant items. For cross-domain, we simply change the search website from sandboxed OneStopMarket to Target (see all website mappings in Table 5). We create 10 examples per website per scenario, and provide all examples in Appendix B.1 and B.2.

2. How performance is measured: as stated in lines 268--270, we manually create evaluation checkpoints for each task and report the checkpoint-average success rate for each task. We also provide detailed checkpoint lists for all examples in Appendix B.1 and B.2.

3. What controls are used: As partially mentioned in point 1 above, we control the task (scale-up the processed items) and domain (switch to in-domain same-purpose website), to study the scale-up and cross-domain scenarios with minimal task transfers.

We couldn't provide all details in the main content due to page limits, but we are happy to specify these more in the revised paper!

R3: Add citation

Thank you for the paper suggestions. We will add and discuss it in the related works.

Thank you for your recognition of the effectiveness of our ASI method, comprehensive experiments, and clarity in paper writing!

R1: How Agents Learn Not to Use Incompatible Skills

Agents often spot the incompatibility of skills based on the discrepancy of web design (e.g., opens a filtering sidebar) and skill implementation (e.g., a dropdown menu for sorting options), as we illustrated in Figure 4. This could be more pronounced when the agent has been trained on the website. Even if the agent cannot spot this incompatibility beforehand, the agent will receive execution errors when trying to use this skill; with these error messages included in context, the agent is less prone to use the incompatible skill in later steps.

R2: ASI assumes deterministic environments

We start with single-episode based skill induction to demonstrate the effectiveness of the programmatic format, and it took careful design to make the single-episode scenario work already. We agree that enabling multi-episode skill induction, or domain randomization, could be an interesting follow-up to potentially enable greater reusability. Nonetheless, our current method allows agents to refer to existing skills to induce new skills. For the pop-up case, our agent can write an extended version of the previous skill with extra lines closing the pop-up window (exemplified below), thus making it reusable and generalizable across episodes.

def click("Search textbox...")
    if has_popup_window(): click("Close")  # added
    fill("Search textbox", name)
    click("Search button")

R2.1: Rate of reusing and inducing skills

The number of examples that (i) attempt to create a new skill, (ii) successfully create a new skill, and (iii) reuse a skill is shown in the table below. Overall, the agent reuses at least 1 skill for 42.5% of the cases, pretty frequently across websites; while these skills are created from 7.6% cases, demonstrating the high scalability of our skill learning approach.

R2.2: Shuffling example ordering

We did not compare different example ordering, because the implementation of the WebArena benchmark evaluation enforces a particular order so that conducting earlier tasks does not contaminate the environment to do later tasks, thus we cannot flexibly change example ordering. We conduct all experiments following the original WebArena order. We do agree that example ordering is an interesting angle to analyze the ASI method in future works.

R3: Induction input We provide the reasoning chain of thought and action associated with each action step for the entire episode as the skill induction input. We find reasoning chains-of-thought are sufficient augmentation to actions to induce high-quality skills, where adding additional state information (e.g., webpage accessibility tree) is long and noisy, thus often hurting end-task success rate. We provide the whole episode for induction, because identifying sub-episodes first and then applying skill induction to them would take extra compute, not practically simple, and often hurt performance in our preliminary studies.

R4: Skill verification

Our goal is to test if agents can successfully use induced skills, to achieve this we first need to have the agent (i) reuse the skills, which we encourage via prefix trajectory (with steps rewritten using new skills), and (ii) correctly use the skills, by our evaluation of the agent task re-solving trajectory. So our skill verification does tell if agents can (i) reuse the skill, and (ii) correctly, to solve the task. Note that we do not add the induced skill to the agent’s action space if (i) or (ii) is not passed, to avoid low-quality skills affecting later task-solving processes.

R5: Baseline: add previous episodes directly in context

Thank you for the suggestion. We recognize this is a reasonable baseline and acquire its performance on WebArena – 35.6 in success rate — 4.8 lower than ASI; furthermore, this baseline agent uses an average of 6.3 steps to solve tasks, 26.0% more than ASI, because the agent cannot call high-level skills to achieve more scalable trajectories. We will additionally report this number in the revised paper.