$\textit{Hyper-GoalNet}$: Goal-Conditioned Manipulation Policy Learning with HyperNetworks

We thank the reviewer nbKf for the insightful feedback and for recognizing the novelty of using hypernetworks in goal-conditioned behavioral cloning. Our responses below, supported by new experiments, aim to address your questions (also thanks for organizing the weaknesses into these structured questions) and resolve the primary concerns listed in the 'Weaknesses' section. Due to space limitations, we focus on the major points here. All other minor edits and clarifications will be incorporated directly into the revised manuscript.

---

Q.5 HyperZero Training Instability, Analysis, and Solutions

A. to Q.5: We appreciate the question about HyperZero's training instability. We address this first to clarify HyperZero’s issues and highlight our method’s architectural advantages.

(a) Diagnosis and Initial Comparison Our initial attempts to stabilize HyperZero with varied learning rates and architectures failed. For the original submission, we avoided specialized initializations to ensure a fair comparison, as our method requires none. We attribute HyperZero’s instability to its architecture, which learns a direct goal-to-parameter mapping that is highly sensitive to initialization. In contrast, our architecture is inherently more robust because it simulates an optimization process, iteratively refining parameters through a series of blocks within a single forward pass.

(b) Enhanced Initialization Schemes for HyperZero Based on your suggestion, we tested two initialization schemes:

• ScalarInit: A learnable scalar to scale the hypernetwork’s output, stabilizing training by controlling initial weight scale.
• Bias-HyperInit[1]: Implemented as recommended in [1] for its compatibility with high-dim conditioning inputs.

(c) Comparative Results New results show that these initializations improve HyperZero, but our method remains superior without requiring such techniques.

Success Rate (%) on Coffee Tasks with Different Initializations

These results confirm the effectiveness of HyperZero’s initialization schemes, yet our Hyper-GoalNet architecture remains superior and more robust.

Revision Plan: We will update Fig. 5 with new training curves and add this analysis to Sec. 4.3.

---

Q.1 Analysis of Baseline Performance

A. to Q.1: We thank the reviewer for this insightful question regarding the baselines. Below, we clarify our approach, provide analytical results, and outline revisions.

(a) HyperZero Analysis & Results

For a fair comparison, we initially used the official HyperZero using the same setup as our method but without special initializations. Following your feedback, we found HyperZero’s training is sensitive to initialization. Applying a specific initialization technique (see Q5 response) significantly improved its success rate. Our method still outperforms, but these results ensure a fairer comparison.

Revision Plan: We will update Tab. 4 with new HyperZero results and add a note detailing the setup.

(b) GCBC and Play-LMP Analysis

• Implementation: Without official code, we used reputable third-party implementations, adapted for our data pipeline with identical core hyperparameters.
• Performance Diagnosis: We verified that the failure stems from design, not implementation errors. We observe that both methods optimize policy via expert action log-probability, leading to overfitting (low training loss, high/stagnant validation loss). This incurs generalization issues to unseen test set variations, especially for high-precision grasping/placing. This finding is consistent with other work (e.g., C-BeT). Our hypernetwork design avoids such overfitting by a structural bias that forces the prediction to follow an optimization process instead of just a memorization of the mapping.

Revision Plan: We will add a paragraph to Sec. 4.2 with these details, technical reasons for performance (likelihood-based vs. distance-based loss), and citing corroborating prior work. We have also added stronger baselines (e.g., Diffusion Policy; see response to Reviewer EGhA), which we believe addresses your concerns.

---

Q.2 Comparison with Hypernetwork-based Meta-RL

A. to Q.2: We thank the reviewer for this question, which helps us clarify our work's novelty relative to hypernetwork-based RL methods [1].

(a) Our Novelty and Clarification

• Paradigm vs. Architecture: Our claim that RL methods "cannot be directly adapted" refers to the training paradigm. The full RL algorithms of [1, 2] are inapplicable in our reward-less BC setting. We only adapted architectures (e.g., HyperZero), not the learning methods themselves. We will clarify this on lines 88-90.
• Architectural Design: Our method, inspired by iterative optimization, is inherently more stable and robust than methods that directly generate "sub-optimal" parameters and rely on initialization fixes like Bias-HyperInit [1].
• Task Conditioning: Our use of high-dim goal images provides superior scalability and generalization over the low-dim one-hot vectors used for discrete tasks in [1], enabling continuous goal specification.

(b) Comparison and Planned Revisions We did not include a comparison due to these fundamental differences(RL vs. BC). However, prompted by your review, we have now benchmarked against a key component of [1]. As shown in our Q5 response, our method significantly outperforms a baseline with Bias-HyperInit[1], confirming our approach's benefits.

Revision Plan: We will:

1. Distinguish the RL vs. BC paradigms in the introduction
2. Expand the Related Work to discuss [1] and [2], highlighting their key differences.
3. Integrate these new results [1] into our main experimental tables.

We believe these changes will clearly establish our work's novelty and contributions.

---

Q.3 Architectural and Training Details

A. to Q.3: We thank the reviewer for these important questions on reproducibility. We will add to the appendix these details.

(a) Policy Network Architecture

• GCBC, Play-LMP, & HyperZero: Details are in our response to Q. 1.
• C-BeT: We adapted its official, state-based implementation for our challenging image-based setting by adding a ResNet-18 encoder. This ensured a fair comparison.
• MimicPlay-M: We used the official implementation within our identical task setup.

(b) Hypernetwork Architecture Our hypernetwork architecture is inspired by [38] and works as follows: First, the goal image is encoded into a goal embedding. This embedding is then fed into a series of hypernetwork blocks. Each block executes a process analogous to a step of Policy Gradient estimation, iteratively optimizing a set of policy parameters as if performing gradient descent. The final, optimized parameters are then applied to a lightweight 3-layer MLP, which serves as the target policy network for predicting actions.

Due to space limitations, we cannot provide a full implementation breakdown. However, we are committed to release our complete codebase upon publication to ensure full reproducibility.

(c) The loss function ℓ in Eq. 5 is the Mean Squared Error loss.

(d) Training Procedure The model is trained end-to-end. The process per step is:

1. The hypernetwork takes the goal image as input and generates the parameters for the target policy.
2. The target policy takes the current observation as input and outputs an action.
3. The final loss is a sum of the classic behavioral cloning loss (MSE) and our proposed latent shaping losses.

Unfreezing strategy: To protect the pretrained R3M vision encoder from noisy gradients early in training, we keep the encoder frozen for the first 20 epochs and then finetune it jointly with the model once the policy has stabilized. Crucially, this exact strategy was applied to all baseline methods for a fair comparison.

(e) Hyperparameter Selection We found our iterative, optimization-inspired method to be highly robust to the choice of hyperparameters. To maintain a fair comparison, we performed a light search and set the initial learning rate for all methods to 4e-5, using the Adam optimizer.

(f) Training Time & Memory Requirements All experiments were conducted on a single NVIDIA RTX 4090 GPU and all methods completed training within one day. While our method requires slightly more resources than HyperZero, this is coupled with a substantial performance gain.

---

Q.4 Verifying Natural Goal Completion Detection

A. to Q.4: While our original submission qualitatively illustrated this concept of automatic detection(Figs. 3, 10, 11), we used environment signals in our main results to ensure a fair comparison with all baselines.

To quantitatively validate this capability, we now compare our autonomous detection (Auto SR) with the environment's ground truth (Env SR) using these metrics:

• Auto SR: Success determined by our method's latent distance threshold.
• Env SR: Success determined by the environment signal.
• Accuracy: Agreement rate between Auto SR and Env SR.
• Recall: Auto SR's ability to identify true successes from Env SR, a crucial metric for not missing completed tasks.

Results for Autonomous Goal Completion Detection (Coffee Tasks)

This strong alignment with the ground-truth signal, shown by an average accuracy of 86.6% and recall of 94.3%, confirms that our latent-distance-based approach is a reliable autonomous completion detector.

---

We thank the reviewer again for these thoughtful questions and hope our clarifications reinforce the value and clarity of our contributions. We would be happy to address any further suggestions or questions you may have.

Dear Reviewer uJn1,

Thank you for your insightful feedback and for recognizing the strengths of our work. We are grateful for your positive comments on the paper's clarity, the novelty of our proposed architecture, and the practical value of our real-world experiments. Your thoughtful assessment is much appreciated. We have carefully considered your questions and concerns, which we address point-by-point in the following.

---

W.1 & Q.1 Necessity of Goal Conditioning

A. to W.1 & Q.1: We thank the reviewer for this insightful question regarding the motivation for goal-conditioning in our chosen tasks. Our primary motivation for using a goal-conditioned architecture is not merely to solve a single task, but to learn a general and versatile policy that can be directed to achieve a wide spectrum of goal states, including critical intermediate steps of a task.

1. Motivation Beyond Ambiguity: Learning a General Skill

The core advantage of our approach lies in its ability to generalize. Instead of learning a reactive policy that simply maps a state to an action for one specific outcome (e.g., "put the lid on the coffeemaker"), our goal-conditioned policy learns a much more fundamental and reusable skill: "given a goal image, manipulate the environment to match that image." This allows the policy to:

• Target any valid goal state, not just the final one from the training data.
• Understand and achieve intermediate goals, demonstrating a deeper comprehension of state progression rather than just overfitting to successful final trajectories.

2. Autonomous Goal Completion Detection

A direct and powerful capability stemming from this architecture is autonomous goal completion detection. Our policy continuously compares the latent representation of the current observation with that of the goal image. When the latent distance falls below a threshold, the policy intrinsically "knows" the task is complete and can terminate. This capacity for self-assessment is impossible for a non-conditioned policy and is a key feature of our method, which we qualitatively illustrated in Figures 3, 10, and 11.

To quantitatively validate this capability, we performed a new analysis comparing our policy's autonomous success detection (Auto SR) with the environment's ground-truth success signal (Env SR). We introduce two key metrics:

• Accuracy: The agreement rate between our method's decision and the ground truth.
• Recall: Our method's ability to correctly identify all true successes. We emphasize recall as it is crucial for a policy not to miss a completed task and continue acting unnecessarily.

The results for the Coffee tasks are presented below:

Table: Quantitative Results for Autonomous Goal Completion Detection

The results show a strong alignment with the ground-truth environment signal, achieving an average accuracy of 86.6% and, most importantly, a high average recall of 94.3%. This confirms that our latent-distance-based mechanism is a reliable method for autonomous completion detection, a critical capability enabled directly by our goal-conditioned design.

Revision Plan: In our revised manuscript, we will:

1. Clarify our motivation in the introduction, explicitly stating that our goal is to learn a generalizable policy capable of achieving a spectrum of goals and performing autonomous completion detection.
2. Add this new quantitative analysis and table to the experimental section to provide strong empirical evidence for this key capability.

We believe this clarification and the new results effectively demonstrate that goal-conditioning provides a crucial advantage—generalization and self-awareness—that goes far beyond simply resolving ambiguity in the initial state.

---

W.2 Venue Fit

A. to W.2:

We thank the reviewer for their thoughtful consideration of our paper's positioning. We agree that our work has strong connections to the robotics community, as the real-world validation is a crucial component of our research.

However, as the reviewer insightfully points out, the primary contributions of our paper are deeply rooted in machine learning topics that are of broad interest to the NeurIPS community. Our work focuses on:

1. Representation Learning for Control: At its core, our paper investigates how to learn effective representations from high-dimensional visual inputs (goal images) to condition policies.

2. A Novel Generative Architecture: Our central contribution is the use of a hypernetwork to generate the parameters of an entire policy network. This positions our work within the broader study of generative models, exploring how one network can produce the weights for another.

We believe that robotics provides a challenging and meaningful testbed for these fundamental ML concepts. Proving the effectiveness of our representation learning and generative architecture on complex, real-world manipulation tasks demonstrates their practical viability and robustness.

Therefore, given this focus on goal-conditioned representation learning and generative hypernetwork architectures, we believe that our paper is well-aligned with the interests of the NeurIPS community and will be of significant value to the community.

---

Q.2 Real Robot Experiment Videos

A. to Q.2: We thank the reviewer for this suggestion. We agree that videos are essential for qualitative assessment and have prepared them to be shared via a project website, which we will link in the camera-ready version.

Due to NeurIPS policy regarding anonymity, we cannot provide the link during the rebuttal period. In the interim, we would direct the reviewer to our Appendix (specifically the "Real Robot Experiment Setting" section), which details our hardware, setup, and procedures. We look forward to sharing the full videos, along with all code and experimental details, upon the paper's acceptance.

Dear Reviewer EGhA,

Thank you for your constructive feedback and for recognizing the strengths of our paper. We are grateful for your positive comments on our core idea being "innovative and well-motivated," our "extensive experiments," and the paper's clarity as "well-written and easy to follow." We have carefully considered your concerns and address your comments and questions in the following.

---

W.1 Single-View Camera Configuration

A. to W.1: We thank the reviewer for this question about camera configurations. Our focus on a single-view setup is a deliberate choice, as we argue it is essential for achieving real-world deployability and generalization. Also, a key to generalization in robotics is the ability to train on massive and diverse datasets. While multi-view systems are information-rich, their setup complexity present significant barriers to this kind of large-scale data acquisition:

1. Barrier to Scale: The need for complex calibration and the high deployment cost make it challenging to gather data from hundreds of different environments, fundamentally limiting data diversity.
2. Barrier to Simplicity: Specifying geometrically consistent goals across multiple views adds significant complexity to the data collection and annotation pipeline, further hindering scalability.

In contrast, a single-view approach is far more scalable. It lowers the barrier for massive data collection (potentially including heterogeneous web data) and enables deployment in novel, un-instrumented environments. We argue that forcing the model to learn from this constrained but realistic input fosters more robust and generalizable representations, paving a more promising path towards agents that can operate "in the wild."

Moreover, the Hyper-GoalNet framework is flexible. Its core mechanism—generating policy parameters from a goal—is agnostic to the vision encoder. Features from multiple cameras could be readily integrated (e.g., via attention) and processed by our latent shaping module. This sensor fusion is a feasible extension but is orthogonal to our primary contribution.

Revision Plan: In the revised manuscript, we will clarify the motivation for our single-view focus, emphasizing its benefits for generalizability and scalability, and discuss multi-view extensions in the future work section.

---

W.2 Additional Experimental Baselines

A. to W.2: We thank the reviewer for this valuable suggestion. To strengthen our evaluation, we have performed new experiments on some tasks against a state-of-the-art Diffusion Policy (DP) and a strong Goal-Conditioned DP baseline. Regarding VQ-BeT, we also worked to include it as a baseline. We endeavored to follow the official implementation, but encountered significant implementation challenges (e.g., the official, (only) image-based model failed to converge, and our efforts to stabilize its VQ-VAE pre-training stage for our high-precision visuomotor data proved non-trivial), preventing us from obtaining stable results within the rebuttal period. We are continuing this effort and aim to include it in the final version.

The new results are summarized below:

These results highlight a crucial trade-off between performance and efficiency. To provide a robust comparison, we benchmarked against a state-of-the-art Diffusion Policy (DP) and an even stronger Goal-Conditioned DP (GC-DP), which leverages the goal image via concatenation. While powerful, both diffusion baselines require a computationally intensive, iterative sampling process at inference time.

In contrast, our method is a single-pass, feed-forward network, making it significantly more efficient. Despite this key efficiency advantage, the results show our method achieves performance comparable to the strong, iterative GC-DP baseline. We attribute this compelling result to our core contribution: the hypernetwork's ability to directly generate high-quality, specialized parameters from the goal, bypassing the need for iterative refinement.

In addition to these policy-level comparisons, we have also added new baselines analyzing the hypernetwork architecture itself, the details of which can be found in our response to Reviewer nbKf's Question 5 (a) and (c).

Revision Plan: We will add this discussion and the new results into our revised manuscript, which we believe substantially strengthens our paper's evaluation.

---

W.3 Real Robot Performance Gap

A. to W.3: We thank the reviewer for this critical question. The significant performance gap on the real robot warrants this deeper explanation. The tasks require high precision, which penalizes policies that are not robust to the noise and observation shifts inherent to the real world. Our analysis points to two distinct failure modes for the baselines:

1. GCBC & Play-LMP: Overfitting from Likelihood-based Training. These methods failed due to their likelihood-based training objective (i.e., maximizing action log_prob), typically under a Gaussian assumption, which is more prone to overfitting than direct regression (e.g., MSE). We observed this overfitting empirically, as their validation loss failed to converge converge to very low values despite low training loss, indicating a poor generalization capability. Similar failure cases also occurred in simulation, where these overfitted policies could only reproduce coarse trajectory motions and consistently failed at the crucial, high-precision steps. This weakness was naturally amplified on the real robot, where noise and observation shifts are unavoidable.
2. C-BeT: Lack of Robustness to Compounding Errors. For C-BeT, the issue is more nuanced. Its powerful Transformer architecture excels at fitting the in-distribution training data. However, it struggles with robustness. In the real world, a small amount of sensor noise can push the robot into a slightly out-of-distribution (OOD) state. A fixed-parameter model like C-BeT makes a small prediction error, which leads to a subsequent state that is even further from the training distribution. This triggers a cascade of compounding errors throughout the trajectory, ultimately leading to task failure.

Why Hyper-GoalNet Succeeds:

Our method is inherently more robust to these challenges precisely because it does not rely on a single, fixed-parameter network. Instead, Hyper-GoalNet dynamically generates the parameters of the policy network on-the-fly, conditioned on the current (and potentially noisy) observation and the specific goal. This mechanism acts as a form of rapid, instance-specific adaptation. The ability to generate and refine high-quality parameters makes the resulting policy far more resilient to observation noise and helps prevent the accumulation of minor errors.

Revision Plan: We will add a detailed discussion of these distinct failure modes and the underlying reasons for our method's robustness to the appendix of our revised manuscript. Thank you for prompting this deeper analysis.

---

W.4 Scaling Ability

A. to W.4: We thank the reviewer for raising this forward-looking question about the scalability of our method. We agree that scaling to a larger number and greater complexity of tasks is a crucial long-term goal for any manipulation policy.

Our current work focuses on establishing the core principle of Hyper-GoalNet: dynamically generating policy parameters from a goal specification (i.e., a goal image) for a lightweight MLP policy. We view this as a foundational proof-of-concept. However, the framework itself is inherently designed for scalability. We believe the key to scaling is not to simply increase the capacity of a single, fixed model, but to enhance the components of our modular system. Specifically, our hypernetwork architecture is compatible with:

1. More advanced guidance: Replacing the single goal image with richer goal specifications, such as videos.
2. More powerful target policies: Generating the parameters for more advanced network architectures beyond a simple MLP, such as a Diffusion Policy.

To validate this potential, we conducted a experiment on scaling a single Hyper-GoalNet architecture across four distinct manipulation tasks. In this setup, we equipped the hypernetwork with a more powerful diffusion-based target policy and used video demonstrations as goal information. The success rates across the subtasks were as follows:

Please note that this was an initial experiment to test the feasibility of scaling. The results are highly promising, demonstrating that when equipped with more advanced components, the HyperNet framework can achieve very strong performance across multiple, diverse tasks. This provides compelling evidence for the scalability of our approach.

Revision Plan: We are very excited by these initial findings and will make scaling a central focus of our future work. We will add a discussion of this to the paper to highlight the future potential of our framework.

---

Q.1 Action Chunking

A. to Q.1: Thank you for this insightful question. We apologize for the omission. In fact, we did employ action chunking in all of our experiments. Our policy predicts a short sequence of future actions at each timestep, and the agent executes this chunk sequentially before the policy is invoked again.

Revision Plan: We omitted this detail for brevity to maintain focus on our core contribution, the Hyper-GoalNet framework itself. We agree this is an important implementation detail and will add it to the experimental setup section in our revised manuscript. Thank you for pointing this out.

Dear Reviewer bvF4,

We would like to sincerely thank the reviewer for their insightful feedback and positive assessment of our work. We are particularly grateful for your recognition of several key strengths, including the novelty of our concept with its detailed theoretical and experimental justifications, the comprehensive experiment setup spanning from benchmarks and ablation studies to real-robot evaluation, and the clarity of the paper as being 'well-written and easy to understand.' We are also encouraged that you highlighted the significant improvements our method achieves over baselines. We have carefully considered your comments and address your questions below.

---

W.1 & Q.1 The need for a more detailed comparison for HyperZero and other baselines.

A. to W.1 & Q.1: Thank you for this critical question. This point touches upon the core motivation for both our work and our experimental design.

First, as you noted, the manipulation tasks we focus on are intentionally challenging. The failure of some baselines on these complex tasks is precisely what motivates the need for a more robust architectural paradigm like our Hyper-GoalNet.

To provide the more detailed comparison you suggested, we have conducted further analysis:

1. Analysis of HyperZero:

In our original experiments, we directly adopted the official architecture of HyperZero for our tasks to ensure a fair comparison. The difference from our method was the hypernetwork architecture itself; all other settings, including the target policy network (a lightweight 3-layer MLP) and hyperparameters, were kept identical. To maintain fairness, we initially avoided any method-specific tricks like special initializations.

However, inspired by your feedback, we revisited the HyperZero ablation with an improved initialization trick. We attribute HyperZero's training instability to an inherent limitation in its design: its approach of directly generating an entire set of network weights makes the training process fragile and highly dependent on tricks like initialization to place the initial weights in a suitable distribution. In contrast, our method is inspired by the process of iterative optimization (akin to gradient descent), which allows for more efficient use of the hypernetwork's capacity and yields better-structured target parameters.

To stabilize HyperZero, we therefore introduced a learnable scalar that acts on the generated weights, effectively placing them in a suitable initial range. We found this technique stabilizes its training process, allowing the loss to converge to a lower value (around 0.14-0.18, though still not as low as our method). This indeed leads to non-zero success rates. The updated results on the coffee task provide the more detailed comparison you were looking for:

These new results offer a more nuanced view, confirming that while HyperZero can be improved,our proposed architecture still significantly outperforms it across all difficulty levels. This reinforces our claim that our specific hypernetwork design and latent space shaping are critical for success.

2. Analysis of GCBC and Play-LMP:

The challenges with these baselines are multi-faceted:

• Implementation: The original authors for these methods did not provide open-source code. We therefore relied on third-party GitHub implementations, making minimal modifications only to align with our experimental setting (e.g., optimizer, learning rate) while preserving the core algorithm. It is noteworthy that in the experiments of a recent paper (e.g. C-BeT), GCBC and Play-LMP also demonstrated near-zero success on similar complex tasks, which corroborates our findings.
• Fundamental Methodological Limitation: After careful inspection, we identified a key technical reason for their failure on our tasks. Both GCBC and Play-LMP model policy learning as a maximum likelihood estimation problem, optimizing the log-probability (log_prob) of actions from a learned Gaussian distribution. We observed that optimizing via probability is more prone to overfitting on the training dataset compared to direct regression with a distance loss (e.g., MSE). This was empirically validated: both methods achieved very low training loss, but their validation loss failed to converge to a low value, indicating poor generalization.
• Observed Failure Mode: In practice, this overfitting prevents them from succeeding in novel test scenes with variations. While they could learn a coarse trajectory, they consistently failed at the fine-grained, precise movements required for successful grasping and placing, which are critical for task completion.

To further strengthen our baseline validation, we have also added comparisons to additional state-of-the-art baselines, as detailed in our response to Reviewer EGhA (W2). The new results are summarized below:

As these results show, our Hyper-GoalNet achieves performance that is highly competitive with, and on some tasks superior to, these powerful diffusion-based models. More importantly, our method accomplishes this in a single inference pass, whereas diffusion-based policies require a much more computationally expensive iterative denoising process. This highlights a significant practical advantage of our approach in terms of efficiency and suitability for real-time applications.

In summary, we believe the failures of the initial baselines are not superficial but stem from fundamental limitations when faced with complex, long-horizon manipulation tasks. The new results for HyperZero provide the granular comparison you requested, and these additional comparisons against diffusion models further validate the effectiveness and efficiency of our proposed architecture. We will add this detailed discussion and the new results to the final version of the paper.