We appreciate your insightful feedback and constructive comments!

W1. The approach involves several components. It would be helpful to understand how robust the overall method is to each individual component.

Cago consists of five main modules: (1) a capability-aware goal sampler, (2) a BC explorer, (3) a goal-conditioned policy, (4) a world model, and (5) a goal predictor. At a high level, Cago trains the world model and goal-conditioned policy in an end-to-end manner. The world model is updated using real trajectories collected by the goal-conditioned policy, whose goal is set by the capability-aware goal sampler (Algorithm 2, Line 18). Upon goal reaching, a pretrained BC explorer takes over for the remaining time steps in a rollout. Simultaneously, the goal-conditioned policy is trained using imagined rollouts generated by the world model (Line 19). These two components are tightly coupled and co-adapt during training, making their joint performance critical to overall success. The goal predictor is trained separately on demonstrations to infer the final goal state from an initial observation. It is used only during evaluation and enables the goal-conditioned policy to generalize to test environments where the goal state is not explicitly provided.

We assess the importance of each component as follows.

(1) To ablate the capability-aware goal sampler, we designed two additional goal sampling strategies:

• Step-based curriculum: This baseline samples goals from demonstrations in proportion to training steps (i.e., current training step / predefined total training steps). However, it does not assess the agent's actual capabilities, and may therefore sample goals that are either too easy or too difficult for the agent at its current learning stage.
• Final-goal sampling (Line 329): This strategy always selects the final observation from a demonstration in the goal phase of our Go-Explore sampling paradigm, ignoring the agent's current goal-reaching capability. The BC Explorer takes control from the goal-conditioned policy halfway through each rollout. The table below reports the final success rates averaged over 5 random seeds (each evaluated over 100 test episodes):

Cago's goal sampling strategy significantly outperforms the alternatives, highlighting its critical role in the overall effectiveness of our approach.

(2) As for BC-Explorer, we provide detailed ablation studies in Section 4 (Line 328). Specifically, Cago-NoExplorer removes the BC Explorer. It does not further explore after reaching a sampled goal. We also evaluate Cago-RandomExplorer, which replaces the BC Explorer with a uniformly random policy during the exploration phase of our Go-Explore-style rollout strategy. The results are in Fig. 4. The table below reports the final success rates averaged over 5 random seeds:

The results highlight the importance of BC Explorer for generating relevant exploratory rollouts to accelerate learning.

(3) For the goal-conditioned policy, we observed that its critic loss decreases over time in correlation with the success rates reported in Fig. 3, underscoring its crucial role in overall performance.

(4) In early development, we attempted a model-free variant of Cago by training a goal-conditioned policy directly from environment rollouts starting at demonstration initial states using actor-critic methods. However, due to the limited number of demonstrations, this approach failed to generalize. We therefore adopted the model-based strategy that leverages a world model to generate diverse imagined rollouts (Line 173), enabling more effective and sample-efficient policy learning.

(5) Empirically, we find that the Goal Predictor reliably infers final goals from held-out demonstrations, enabling effective zero-shot execution in unseen test tasks.

W2. The discussion on how the method could be applied to broader scenarios—such as real-world settings with diverse initial states and uncertain dynamics—could be expanded.

Cago can be extended to support diverse initial states and uncertain dynamics.

• For diverse initial states in real-world settings, Cago can gradually prioritize those where the agent has the most learning potential—starting from demonstration initial states and expanding to nearby ones where the agent can still benefit from the demonstration trajectories for dynamic capability tracking and progressive goal selection. As training progresses, we incorporate successful rollouts from these nearby initial states into the demonstration set, broadening its coverage. This enables a natural curriculum that eventually transitions to diverse, real-world initial states.
• For uncertain, stochastic dynamics, Cago remains robust by not requiring exact reproduction of actions in demonstration trajectories. Instead, it adapts goals based on observed agent successes, making the curriculum self-correcting in the face of noise or variability. If the agent consistently fails to reach a specific demonstration state, Cago's capability metric naturally shifts goal sampling toward more achievable targets. Additionally, goal selection can be extended with probabilistic estimates—such as goal-reaching likelihood or future state predication uncertainty from an ensemble of world models—which we view as a promising direction for high-stochasticity domains.

Q1. Why not initialize the goal-conditioned policy with behavioral cloning (BC) pre-training?

We considered the possibility of initializing $\pi^G$ with behavioral cloning (BC) on demonstration data, and conducted preliminary experiments with BC pretraining in Cago’s early development stage. However, our findings led us to deliberately decide against using BC initialization in our final framework, for the following reasons:

• (1) Risk of overfitting under sparse demonstrations: Cago is specifically designed for the low-data regime, where only a small number of demonstrations (typically 5 or 10) are available. In such cases, BC pretraining tends to overfit to the limited demonstration data, resulting in poor generalization when the agent starts to explore beyond the demonstrated trajectories.

• (2) Misalignment with the training framework of Cago: Cago’s goal-conditioned policy $\pi^G$ is trained jointly with a Temporal Distance Network (TDN), which estimates how many steps are needed to reach a goal. The TDN serves as a dense feedback signal and is itself trained on imagined trajectories from the world model. Crucially, all three components—the policy, world model, and TDN—are updated jointly in an end-to-end manner. In the early phase of training, the TDN’s estimates are often inaccurate. Therefore, even a well-behaved BC-initialized policy would receive noisy or misleading training signals, causing it to quickly deviate from the pretraining distribution. This undermines the value of BC initialization and can sometimes even slow down learning. These observations informed our design decision to exclude BC pretraining.

Q2. How does the method’s performance scale with the number of demonstrations? ... It would be helpful to justify the choice of using only 10–20 demonstrations.

Our method is explicitly designed for the low-demonstration regime, which is typical in real-world robotic settings where expert trajectories are expensive, time-consuming, or risky to collect. We intentionally evaluate Cago with only 10–20 demonstrations to highlight its practical relevance, emphasizing sample efficiency and generalization from limited supervision, enabled by the use of a world model to generate imagined rollouts that enrich training. A more accurate model around the demonstration distribution can benefit policy generalization. We conducted ablation study in the Supplemental Material (Figure 1) using 5, 10, and 20 demonstrations. We further conducted an experiment using 50 demonstrations to evaluate how Cago performs under a large set of expert data. The table below reports the final success rates averaged over 5 random seeds (each evaluated over 100 test episodes):

We found that with 50 demonstrations, Cago performs exceptionally well, exhibiting significantly better generalization and achieving higher final success rates compared to training with fewer demonstrations. However, we also found that larger demonstration sets may delay learning onset due to longer warm-up times to fit world models. Studying this trade-off is an interesting direction.

We appreciate your insightful feedback and constructive comments!

1. This methodology critically relies on "high-quality demonstration trajectories that adequately cover the task space." The methodology lacks experimental analysis or solutions to noisy, incomplete, or suboptimal expert data.

We clarify that Cago only assumes access to "high-quality demonstration trajectories that adequately cover the task space" (Line 246) in order to derive the theoretical guarantee stated in Theorem 1. In practice, the demonstrations used in the Adroit environments (door, hammer, and pen) are in fact noisy and suboptimal, and yet Cago remains effective. This highlights the robustness of our method to imperfect demonstrations.

To further address this concern, we have conducted additional experiments specifically designed to evaluate how Cago performs when provided with noisy, incomplete demonstrations. We collected three different types of suboptimal demonstrations:

• (1) Missing Observations: We randomly removed 20% of the observations from each original demonstration trajectory to simulate incomplete data.

• (2) Noisy Expert Actions: We added Gaussian noise (scaled by 0.1) to each action output by the expert policy during expert trajectory collection.

• (3) Random Actions: We replaced the expert’s action with a randomly sampled action with a probability of 20% at each timestep during expert trajectory collection.

These settings simulate three kinds of suboptimality: missing state information, noisy expert behavior, and corrupted decision-making. Using these demonstrations, we re-evaluated Cago on three benchmark environments: AdroitPen, Stick-Push, and Disassemble. Because we do not have the expert policy for Adroit, we only conducted the "Missing Obs" experiment for Adroit-Pen. The table below reports the final success rates after 1M training steps, averaged over 5 random seeds (each evaluated over 100 test episodes):

The results suggests that Cago is robust to the suboptimality in demonstrations. This is because:

• First, Cago only uses demonstration trajectories to extract sequences of observations that serve as curriculum goals. It does not attempt to imitate the demonstration directly, nor does it rely on demonstrations to infer a reward function. This significantly reduces Cago's dependency on high-quality demonstrations.

• Second, even when the demonstration trajectories are suboptimal—as long as they represent a successful sequence of observations—Cago can still learn well. We observe that in the Disassemble environment, Cago achieves significantly higher success rates when using the "Random Actions" type of suboptimal demonstrations compared to using the original demonstrations. Those imperfect demonstrations—with pauses or detours—cover a broader region of the task space, thereby improving the final policy's generalization. More importantly, Cago is guided by a reward function shaped using a Temporal Distance Network (see Main Paper Appendix B.2), which estimates how many steps are needed to reach a goal state. This enables Cago to discover more efficient, often shorter, alternative paths for goal reaching. Therefore, even when the demonstration are suboptimal, Cago may still learn the optimal paths. We will revise the paper to highlight Cago's robustness to suboptimal demonstrations by including these results.

2. Here is a clear lack of discussion or evidence regarding its scalability to situations where goals are difficult to define directly from demonstration trajectories.

Our work specifically targets robotic manipulation tasks, where it is both common and effective to define the goal space directly in terms of the observation space. This approach has been widely adopted in prior works on goal-conditioned reinforcement learning [1,2,3]. In Appendix F.2, we demonstrate that Cago performs comparably in high-dimensional visual environments, even when using a simple similarity metric such as mean squared error (MSE) between image-based observations. Since Cago guides agents toward low-level demonstration states in a progressive manner, it does not rely on or infer high-level abstract goals—doing so is outside the intended scope of our method.

[1] Planning goals for exploration, ICLR 2023

[2] Learning World Models for Unconstrained Goal Navigation, NeurIPS 2024

[3] Discovering and Achieving Goals via World Models, NeurIPS 2021

3. More sensitivity analysis on how lambda_visit and delta affect performance.

Regarding $\lambda_{visit}$: This parameter controls the threshold for how frequently a state must be visited before it is considered “mastered” by the agent, and thus eligible for sampling more difficult goals. If it is set too small, Cago may prematurely progress to harder goals before acquiring sufficient competence in easier ones. Conversely, if it is set too large, Cago may become overly conservative, spending excessive time on already-mastered states. We have conducted a sensitivity analysis by setting $\lambda_{visit} \in {50, 100, 200}$, and evaluated Cago on three environments: Stick-Push, Disassemble, and AdroitPen. The final success rates after 1M training steps, averaged over 5 random seeds are shown below.

Results show that while very small $\lambda_{visit}$ may slightly influence learning, Cago consistently reaches similar final performance, indicating robustness to this parameter across a broad range.

Regarding $\delta$: This parameter defines the goal sampling window size, i.e., the proportion of expert states from which new goals are sampled during training. A large $\delta$ value may lead to sampling overly distant or inappropriate goals, affecting stability and learning efficiency. In contrast, a very small $\delta$ may overly restrict exploration and hinder curriculum progression. We added a sensitivity experiment comparing the default $\delta$ = 10% with a smaller value $\delta$ = 5% and a larger value $\delta$ = 20%, across the same three environments. The table below shows that while larger values can sometimes slightly reduce performance, Cago remains capable of learning effective policies in all settings. We will include these results to the paper to provide a more thorough discussion of both parameters.

4. Although Cago is compared to demonstration-based baselines, it overlooks comparisons with broader goal-conditioned reinforcement learning methods beyond demonstrations.

Cago introduces a novel paradigm for leveraging demonstrations, with goal-conditioned RL (GCRL) serving only as the underlying framework. Our focus is on demonstration utilization, so we adapt a broad set of strong baselines that differ in how they use demonstrations. To further address the reviewer’s concern, we include PEG [1], a recent competitive model-based GCRL method, alongside the existing GCRL-Dreamer baseline. As shown in the table below, Cago consistently outperforms these GCRL baselines.

Table4: Results of Cago and GCRL baselines.

[1] Hu. et al., Planning goals for exploration, ICLR 2023

5. How Cago's learned policy behaves during the exploration phase, and how Cago succeeds in specific scenarios where baselines fail.

We visualized the exploration phase of Cago for the StickPush environment in Figure 4(d). Each red dot represents the normalized position of a sampled goal within a demonstration trajectory, with 0 indicating the start and 1.0 indicating the final demonstration state. Early in training, the agent predominantly samples goals at lower normalized positions, focusing on subgoals near the beginning of the trajectory that are within its current capabilities. As training advances, goal sampling gradually shifts toward higher normalized positions, indicating the agent’s increasing ability to pursue more challenging goals closer to task completion. By continuously targeting goals just at the boundary of the agent’s current capability, Cago facilitates efficient learning in this long-horizon task. In challenging tasks like Peg-insertion, Cago's curriculum-like progression—guided by internal capability estimation—significantly improves learning efficiency. In contrast, we found the baseline policies, while attempting to imitate the demonstrations, move the manipulator to the final pose even the it does not yet grasp the peg. Similar exploration trends are observed across other benchmarks, and we will include these figures in the revised paper.

I thank the authors for their response and the additional experiments. The additional experiments and clarifications address several of the initial concerns. However, several fundamental concerns remain unaddressed or only partially mitigated.

1. Unrealistic Demonstration Assumptions

While the additional experiments with noisy demonstrations are appreciated, all variants still assume that the demonstrations lead to successful task completion. In real-world settings, demonstrations are often incomplete, ambiguous, or include failed attempts—conditions under which Cago, as currently designed, is not equipped to operate. This limits its robustness in practical deployment scenarios.

2. Scalability to Abstract or Semantic Goal Spaces

Cago critically depends on defining goals as observable states directly sampled from demonstrations, and similarity is computed via MSE in pixel or state space. This approach is:

• Fragile in high-dimensional visual settings, and
• Inapplicable to tasks requiring language conditioning or abstract goal inference.

Therefore, I am inclined to slightly raise my score to reflect the improvements, while still noting that further refinement is needed to reach broader applicability.

We appreciate your insightful feedback and constructive comments!

Q1. Provide a clearer framing, I would suggest committing to the IL narrative.

We would like to clarify that Cago is fundamentally different from traditional IL methods, both in methodology and motivation. IL typically requires direct learning from demonstrations, either by recovering a reward function from demonstrations, or matching expert and agent state-action distributions (e.g. GAIL), or incorporating demonstrations as anchors to mitigate overestimation and instability caused by out-of-distribution actions (e.g. Cal-QL). In contrast, Cago presents a new paradigm to utilize demonstrations: it dynamically tracks the agent's competence along demonstrated trajectories and uses this signal to select intermediate states in the demonstrations that are just beyond the agent's current reach as goals to guide online trajectories collecting. To evaluate this novel perspective, we compared Cago against state-of-the-art baselines that represent diverse strategies for learning from demonstrations, including distribution mathcing (GAIL and PWIL), curriculum learning (JSRL), model-based exploration (MoDem), and offline-to-online fine-tuning (CalQL and RLPD). These comparisons are essential to highlight the effectiveness of Cago as a new paradigm for learning from demonstrations.

We clarify that our baselines can access an absolute sparse reward where +1 is rewarded only on success states and 0 is given otherwise. This reward is, however, not accessible to Cago during training.

Q2. Extend the related work to better cover imitation learning

In the revision, we will expand the Related Work section to include a broader and more up-to-date survey of IL literature and clarify more explicitly how Cago differs from standard IL methods.

Q3. Provide more IL state-of-the-art baseline

We appreciate the suggestion to include more imitation learning baselines for clarity and completeness. To address this, we conducted additional experiments comparing Cago with two suggested state-of-the-art imitation learning methods—SQIL and ValueDice—in addition to our existing IL baselines, GAIL and PWIL, across three environments: Stick-Push, Disassemble, and Adroit-Pen. The table below reports the final success rates averaged over 5 random seeds (each evaluated over 100 test episodes):

Cago consistently outperforms these baselines. We will update the paper to include the additional baselines suggested by the reviewer across all our benchmark tasks.

Q4. Include results on D4RL for which there are results on established baselines.

We would like to clarify that our evaluation already includes tasks from the D4RL benchmark. Specifically, the Adroit environments used in our experiments—Door, Hammer, and Pen—are directly taken from D4RL.

Q5. Provide further discussion on initial state resets, how this impacts real world robot experiments, and what alternatives there are, if any?

Indeed, resetting to specific initial states is a limitation of Cago, but as the reviewer notes, it is already a substantial improvement over arbitrary state resets, which are often infeasible in real-world robotics. Importantly, Cago does not require exact resets to the full demonstration initial states. For instance, in the Peg Insertion task, the peg's initial position can differ from the demonstration, and Cago can learn to reach states within the early portion of the demonstration. However, some elements—like the hole and the box—are non-movable and randomly initialized by the environment. Since the agent cannot manipulate these objects, we have to reset them to match the demonstration configuration to ensure goal reachability. The constraint over other controllable components, such as the robot arm and peg, can be relaxed. This partial reset strategy reduces the burden and makes Cago more practical for real-world settings.

Q6. Further details on the implications of theorem 1 would be helpful. Also, it feels like assumption (2) would mean BC is good enough.

Theorem 1 is intended to formalize that the imagined rollouts generated by the world model are relevant and close to trajectories that could have been sampled in the real environment. This property is important because it ensures that policies trained over imagined rollouts can generalize to the real world. Assumption (2) states that the world model can be accurately learned via supervised learning from trajectories collected by the BC Explore policy. This is a reasonable assumption, and it does not require the BC Explore policy itself to be of high quality.

Q7. $D_{cap}$ is not used to train $\pi_{G}$ in Algorithm 2 and it feels a bit wasteful.

$D_{cap}$ plays a crucial role in training $\pi_{G}$. We learn $\pi_{G}$ based on imagined rollouts from the learned world model $\widehat{\mathcal{M}}$. Each rollout begins at $s_0$, a state randomly sampled from a trajectory $\tau$ in $\mathcal{D}_{cap}$, and is rolled out for $H$ steps using the goal-conditioned policy $\pi^G(a_t | s_t, g)$. The goal state $g$ is selected as a future state $s_H$ from the same trajectory $\tau$. The objective is to train $\pi^G$ to reinforce trajectories that efficiently reach $g$ in the imagined rollouts from $s_0$ under the learned dynamics $\widehat{\mathcal{M}}$.

Q8. Cago applied to Model-free RL.

In early experiments, we attempted to train Cago in a model-free manner by learning the goal-conditioned policy directly from real environment rollouts. But we further find that instead of directly using $D_{cap}$ to train $\pi_G$ in Model-free manner, the world model can offer a richer set of imagined rollouts to train the $\pi^G$, which improves the learning efficiency. In our setting, the simulated trajectories have starting states and goal states randomly sampled from the same trajectory in $D_{cap}$. As a result, the simulated trajectories still resemble segments from $D_{cap}$​, while significantly increasing trajectory diversity and data richness. The MBRL—leveraging simulated trajectories allows us to generate more diverse and abundant training data. This promotes generalization across the environment.

Q9. The use of the similarity metric to count visitation could be a limitation for more complex tasks.

We have evaluated Cago in visual environments (Appendix F.2) and found it performs comparably to the state-based setting. While that experiment used MSE to compute differences between image-based observations, more advanced metrics—such as SSIM (Structural Similarity Index) or FSIM(Feature Similarity Index)—can be incorporated. Importantly, the similarity metric used in our paper serves as a simple instantiation, rather than a core component of the paradigm. Cago can be configured with alternative metrics, such as similarity between state embeddings in a world model's latent space, to reduce input dimensionality, or use other signals like future state predication uncertainty from an ensemble of world models to assess whether the surrounding space has been sufficiently explored.

We thank the reviewer for the constructive feedback. Your observation regarding whether exogenous rewards are required is a valuable perspective. In the revision, we will strengthen the discussion of Cago's connection to standard imitation learning methods from this viewpoint in both the introduction and related work. For evaluation, we will move the comparison results with GAIL and PWIL (currently in Fig. 8) into the main paper and additionally include SQIL, ValueDice, and CSIL as relevant baselines across all tasks. We have already evaluated SQIL and ValueDice during the rebuttal and found that Cago outperforms both. Unfortunately, we were unable to run the official implementation of CSIL due to CUDA version constraints (their code requires CUDA 11, while our system only supports CUDA 12), but we expect to resolve this in the final version.

Notably, the CSIL paper reports only normalized scores (relative to a base policy) for Adroit Door and Hammer, rather than absolute task success rates. Their results suggest that CSIL does not significantly outperform BC. In our experiments, BC achieves only 24% and 8% success rates on Door and Hammer, respectively, using 10 demonstrations—even after careful tuning. In contrast, Cago significantly outperforms the BC results, demonstrating its effectiveness in the challenging low-demonstration-data regime.

We appreciate your insightful feedback and constructive comments!

1. Cago should be compared with baselines using demonstrations to generate a curriculum [1-4].

In our experiments, we included Jump-Start Reinforcement Learning (JSRL, ICML’23), a recent and strong curriculum learning baseline. JSRL pretrains a guide policy on offline data to provide a curriculum of starting states for the RL policy, which significantly simplifies the exploration problem. As the RL policy improves, the effect of the guide-policy is gradually diminished, leading to an RL-only policy that is capable of further autonomous improvement.

Regarding the specific curriculum-based approaches in [1–4]:

[1] constructs a personalized demonstration curriculum by computing task-specific difficulty scores based on the teacher’s and learner’s policies. In contrast, Cago does not require hand-crafted difficulty metrics—instead, it extracts intermediate goals directly from the demonstration, and guides the agent to reach them progressively based on its own learning progress. This avoids the challenge of designing difficulty metrics for complex manipulation tasks, which is non-trivial without expert knowledge. [2] assumes access to a demonstrator policy, either via imitation learning or inverse reinforcement learning. However, our experiments show that standard imitation learning performs poorly on our benchmarks, limiting the effectiveness of such approaches in our domain. [3,4] segment demonstrations and construct a curriculum by resetting episodes to intermediate states along the demonstration trajectory, starting from the end and moving backward as learning progresses. Cago explicitly avoids such state resets, which are often unrealistic in real-world robotic settings without simulator support (Line 83).

To further address this concern, we conducted additional experiments comparing Cago to [2] and [3] on the Fetch-PickPlace and Fetch-Slide environments that they were evaluated on. The tables below report the final success rates after 1M training steps, averaged over 5 random seeds (each evaluated over 100 test episodes):

We will incorporate these comparisons in the revised paper.

2. BC Explorer appears to be a crucial component for Cago to work, but its training details are unclear.

The BC-Explorer is trained using behavior cloning on a limited set of demonstrations $\mathcal{D}_{\text{demo}}$ (Line 3, Algorithm 2). It is a stochastic policy trained to mimic expert actions by maximizing their log-likelihood. This stochasticity—combined with limited demonstrations and controlled training steps—prevents overfitting and ensures a good balance between exploration and imitation.

3. Cago includes multiple trainable modules. How well does each module need to perform to make Cago work?

Cago consists of five main modules: (1) a capability-aware goal sampler, (2) a BC explorer, (3) a goal-conditioned policy, (4) a world model, and (5) a goal predictor. Cago trains the world model and goal-conditioned policy in an end-to-end manner. The world model is updated using real trajectories collected by the goal-conditioned policy, whose goal is set by the capability-aware goal sampler (Algorithm 2, Line 18). Upon goal reaching, a pretrained BC explorer takes over for the remaining time steps in a rollout. Simultaneously, the goal-conditioned policy is trained using imagined rollouts generated by the world model (Line 19). These two components are tightly coupled and co-adapt during training. The goal predictor is trained separately on demonstrations to infer the final goal state from an initial observation. It is used only during evaluation and enables the goal-conditioned policy to generalize to test environments where the goal state is not explicitly provided.

We assess the importance of each component as follows.

(1) To ablate the capability-aware goal sampler, we designed two additional goal sampling strategies:

• Step-based curriculum: This baseline samples goals from demonstrations in proportion to training steps (i.e., current training step / predefined total training steps). However, it does not assess the agent's actual capabilities, and may therefore sample goals that are either too easy or too difficult for the agent at its current learning stage.
• Final-goal sampling (Line 329): This strategy always selects the final observation from a demonstration in the goal phase of our Go-Explore sampling paradigm, ignoring the agent's current goal-reaching capability. The BC Explorer takes control from the goal-conditioned policy halfway through each rollout. The table below reports the results averaged over 5 random seeds:

Cago's goal sampling strategy significantly outperforms the alternatives, highlighting its critical role in the overall effectiveness of our approach.

(2) As for BC-Explorer, we provide detailed ablation studies in Section 4 (Line 328). Specifically, Cago-NoExplorer removes the BC Explorer. We also evaluate Cago-RandomExplorer, which replaces the BC Explorer with a uniformly random policy during the exploration phase of our Go-Explore-style rollout strategy. The results are in Fig. 4. The table below reports the final success rates averaged over 5 random seeds (each evaluated over 100 test episodes):

The results demonstrate that both ablations significantly reduce performance, highlighting the importance of BC Explorer for generating relevant exploratory rollouts to accelerate learning.

(3) For the goal-conditioned policy, we observed that its critic loss decreases over time in correlation with the success rates reported in Fig. 3, underscoring its crucial role in overall performance.

(4) In early development, we attempted a model-free variant of Cago by training a goal-conditioned policy directly from environment rollouts starting at demonstration initial states using actor-critic methods. However, due to the limited number of demonstrations, this approach failed to generalize. We therefore adopted the model-based strategy that leverages a world model to generate diverse imagined rollouts (Line 173), enabling more effective and sample-efficient policy learning.

(5) Empirically, we find that the Goal Predictor reliably infers final goals from held-out demonstrations, enabling effective zero-shot execution in unseen test tasks.

4. Given a small number of demonstrations with complex model structures, it is very likely that the model would overfit to the demonstrations.

Cago mitigates overfitting to a small number of demonstrations by training its goal-conditioned policy $\pi^G$ within a world model. Rather than training $\pi^G$ to reach states solely from demonstrations, Cago instructs $\pi^G$ to learn to reach diverse states sampled from real rollouts in $\mathcal{D}_{\text{cap}}$, which records the trajectories generated under our Go-Explore paradigm with capability-aware goal sampling, in its world model. This promotes generalization across the environment. To evaluate the generalization capability, we tested Cago on 500 unseen initial states generated from random seeds, each differing from those in the demonstrations. We report both the average L2 deviation from demonstration initial states and the success rate across these trials.

The results are consistent with Fig 3.

5. Can reaching a state with a trajectory that is very different from the demonstrations indicate that the agent has learned the ideal skills?

Cago adopts a curriculum-style sampling strategy, where goals are selected progressively from easier to harder states along the demonstration trajectories. This naturally encourages the agent to incrementally reproduce and internalize the demonstrated behaviors, as illustrated in Fig.1 (Right) and Fig.4(d).

6. Cago does not consistently or significantly outperform other methods.

ShelfPlace is the only environment where Cago is outperformed by GAIL. In all other environments, Cago either outperforms the baselines or achieves near 100% success. On ShelfPlace, we observe that GAIL learns a shortcut to achieve the goal, whereas Cago follows the demonstration more faithfully, which results in slower task completion.

Dear reviewer,

Have the authors assuaged your concerns? Would you be able to move to a positive score?

We thank the reviewer for reinforcing this concern. To further strengthen the reliability of our results, we conducted additional experiments using 3 more random seeds on three representative environments: Adroit-Pen, Stick-Push, and Disassemble. This brings the total to 8 seeds per environment. We compared the performance of Cago against the best-performing baseline. The final success rates and standard deviations at 1M steps are summarized below:

These results show that the additional runs yield means similar to those reported with 5 seeds, while further reducing the standard deviation, indicating improved statistical confidence. We hope this extended evaluation more clearly demonstrates the robustness of our method.

In the final version, we will evaluate Cago and all baselines using 10 random seeds to further strengthen the empirical findings. Thank you for your thoughtful comment to help improve our paper!