Learning Parameterized Skills from Demonstrations

Thank you for the positive and thoughtful comments. We address your questions below.

Q1: Why are there no comparisons to other variational skill discovery methods like [1], [2]?

A1: We center our comparisons against PRISE, as it is a recent and performant work; we thus utilize the same environments (LIBERO and Metaworld) for fair, standardized evaluations. To the best of our knowledge, there are no available, performant implementations or existing evaluations for [1] and [2] that have been tuned for LIBERO and Metaworld.

Q2: This work is missing some ablations, especially ablating having both discrete and continuous skill latent variables instead of just one or the other. I am curious how important it is to actually have both or could the skill policy be successful parameterized with just a continuous variable. It would also be helpful to ablate the state compression mechanism to see how much compression is necessary to learn meaningful skills.

A2: Thank you for raising this important suggestion. We have run the ablations you have recommended. We present the results here:

1. Only discrete or continuous variables: the following table compares the performance on LIBERO-OOD between DEPS (results from the paper), (ii) only discrete skills (iii) only continuous parameters. The results for (ii) and (iii) are averaged across 3 seeds

   Method	Mean Success	Mean Max Success
   DEPS (results from paper)	0.34 ± 0.08	0.66 ± 0.12
   Discrete only	0.06 ± 0.02	0.16 ± 0.01
   Continuous only	0.15 ± 0.16	0.32 ± 0.30

   DEPS outperforms both settings. We find these results to be consistent with our presented motivation for a two-tiered design in Section 1. Learning only discrete skills can prevent generalization over a smoothly varying distribution of tasks. Similarly, learning purely continuous parameters leads to less structured and interpretable representations.

2. Amount of state compression: To validate the importance of 1D state compression to DEPS’ strong performance, we compare the mean success rates and the mean highest success rates on the LIBERO-OOD test setting for DEPS with 1D state compression (results from the paper) against DEPS with 3D state compression. The results for 3D state compression are averaged across 3 seeds.

   Compression Dimension	Mean Success Rate	Mean Highest Success Rate
   1D (from paper)	0.34 ± 0.08	0.66 ± 0.12
   3D	0.05 ± 0.04	0.17 ± 0.11

   The dramatic performance drop from 1D to 3D compression (0.34 → 0.05 mean success rate) suggests that aggressive compression to a single dimension is crucial to DEPS’ performance.

Q3: How is your method finetuned?

A3: We outline the finetuning procedure in Appendix A5. While we use the same general approach as in the pretraining phase, we no longer utilize the variational network. Instead, we now sample the discrete skill, $k$, and the continuous parameter, $z$, from the autoregressive policies $\pi^K$ and $\pi^Z$. This is done to make the finetuning process align with the process used to sample actions during inference.

Q4: How do you ensure both $k$ and $z$ encode useful and distinct information?

A4: Thank you for the insightful question! Empirically, the first ablation in A2 validates the need for both $k$ and $z$. One of DEPS’ main goals is to ensure that $k$ and $z$ encode useful information. As outlined in Appendix A1, during pretraining, the variational network predicts continuous parameters per discrete skill for each forward pass over a given trajectory (instead of predicting a new continuous parameterization at each time step, which could lead to overfitting). Additionally, to prevent the continuous parameters from overfitting to specific tasks, we introduce the $\mathcal{L}_{\text{norm}}$ regularization term (Appendix A1), which bounds the range of the learned continuous parameters. This limits the amount of information encoded in the discrete skill and continuous parameters, forcing the 1D state to encode meaningful information as an “index” into parameterized skill trajectories. This is validated in Figure 8, where the learned latent 1D states vary monotonically within discrete skills, with discontinuities when the discrete skill switches. The strong performance of DEPS on out-of-distribution downstream tasks (Table 1, LIBERO-OOD) further supports our hypothesis that DEPS learns meaningful abstractions instead of overfitting to $k$ or $z$ values. This is particularly evident in low-data regimes (Table 1, LIBERO-3-shot) and low pretraining settings (Table 2).

We provide several lines of evidence that $k_t$ and $z_t$ do encode skill-relevant and distinct information:

1. Consistent application of discrete skills (Section 5.5 and Figure 7): We find that DEPS discovers semantically meaningful discrete skills corresponding to primitive behaviors such as grasping objects, opening doors, etc. Importantly, we find that the discovered skills are applied consistently across tasks. An example can be seen in Figure 7, where the same ``grasp_object’’ skill is used to pick butter in a cabinet environment (top) and pick a mug in a kitchen environment (bottom). This is not a cherry-picked example; we can release an annotated version of the entire LIBERO dataset across multiple tasks, showing that learned discrete skills consistently hold across distinct tasks and environments.

2. Smooth variation in policy on varying continuous parameters (Section 5.5): We find that for a given discrete skill, slight modifications in the continuous parameter result in smooth variations in the resulting policy. For example, modifying the continuous parameters for a discrete skill used to turn a knob smoothly varies the location at which the knob turning action is executed. Visualizations of this are provided on our website (which was already included at the time of submission; the URL is in section 5.5).

Thank you again for your constructive feedback, which has helped improve the clarity and position of our paper. Please let us know if you have any additional questions.

Thank you for your thoughtful and thorough review. We address your questions below.

Q1: How is K chosen? Please describe any heuristics, cross-validation, or sensitivity analysis; the current text implies it is fixed but gives no guidance.

A1: You raise an important question about the hyperparameter sensitivity of DEPS. In our experiments, we set the maximum number of skills (K) to the same value (10) across all datasets and environments. The value of K is meant to simply be a large number that sets an upper bound on the number of discrete skills that can be learned by DEPS. This is a common technique used in previous skill learning papers (e.g. [1] and [2]). In practice, we find that the number of discrete skills learned by DEPS varies across training datasets, but is below K on average. Because of this, we believe DEPS is robust to higher values of this hyperparameter, and therefore did not prioritize sweeping different values of K.

To validate the robustness of DEPS to different values of K, we present some new results of DEPS on LIBERO-OOD setting K to be twice as large (20). Results for K=20 are averaged across 3 seeds.

As can be seen, setting K=20 in fact improves the performance of DEPS. In general, we find that DEPS is highly resilient to different hyperparameter settings. Due to limited compute availability, we did not do much hyperparameter tuning on DEPS, and mostly used the same hyperparameter settings across datasets/environments.

[1] Learning Options via Compression. Jiang et. al. NeurIPS 2022. [2] Language Guided Skill Learning with Temporal Variational Inference. Fu et al. ICML 2024

Q2: Single-latent ablation – add experiments with (i) only discrete skills and (ii) only continuous parameters to prove the necessity of the two-tier latent design.

A2: Thank you for the suggestion. We have added the new experiments, and the following table compares the performance on LIBERO-OOD between DEPS (results from the paper), (ii) only discrete skills (iii) only continuous parameters. The results for (ii) and (iii) are averaged across 3 seeds

DEPS outperforms both settings. We find these results to be consistent with our presented motivation for a two-tiered design in Section 1. Learning only discrete skills can prevent generalization, while learning purely continuous parameters can lead to less structured and interpretable representations. On the other hand, DEPS learns semantically interpretable discrete skills whose policies vary smoothly with the continuous parameter value. Visualizations can be found at our website (which was already included at the time of submission; the URL is in section 5.5).

Q3: End-to-end baselines – include a diffusion-policy or transformer-BC baseline that maximises πᵃ directly without latent regularisation to test whether πᵃ+{πᴷ,πᶻ} is truly superior.

A3: We would like to clarify that the BC baseline included in all our experiments (e.g., in Tables 1 and 2) is end-to-end. To ensure a fair comparison between DEPS and the BC baseline, we use the same network architecture (LSTM) for both. Therefore, we don't compare diffusion-policy or transformer-BC as the difference in the fundamental network structure makes direct comparison difficult. We focus on DEPS as a general framework for learning parameterized skills from demonstrations; it can be easily extended to diffusion/transformer based policies, and we leave such architecture sweeps as future work.

Q4: Risk of latent shortcuts – with 1-D state input the sub-policy may over-fit to $k$ or $z$ values rather than learning robust motor primitives. Can you visualise latent distributions to rule this out or try less aggressive compression?

A4: (On latent shortcuts) You raise an important point about the potential to overfit to $k$ or $z$ values. One of DEPS’ main goals is to prevent this from occurring. As outlined in Appendix A1, during pretraining, the variational network predicts a fixed continuous parameter per discrete skill for each forward pass over a given trajectory (instead of predicting a new continuous parameterization at each time step, which could lead to overfitting). Furthermore, to prevent the continuous parameters from overfitting to specific tasks, we introduce the $\mathcal{L}_{\text{norm}}$ (Appendix A1) regularization term, which bounds the range of the learned continuous parameters. This limits the amount of information encoded in the discrete skill and continuous parameters, forcing the 1D state to encode meaningful information as an “index” into parameterized skill trajectories. This is validated in Figure 8, where the learned latent 1D states vary monotonically within discrete skills, with discontinuities when the discrete skill switches. The strong performance of DEPS on out-of-distribution downstream tasks (Table 1, LIBERO-OOD) further supports our hypothesis that DEPS learns meaningful abstractions instead of overfitting to $k$ or $z$ values. This is particularly evident in low-data regimes (Table 1, LIBERO-3-shot) and low pretraining settings (Table 2).

(On visualizing latent distributions) In Section 5.5, we provide visualizations showing that discovered discrete skills correspond to primitive behaviors such as grasping objects, opening doors, etc. Importantly, we find that the discovered skills are applied consistently across tasks. An example can be seen in Figure 7, where the same ``grasp_object’’ skill is used to pick butter in a cabinet environment (top) and pick a mug (bottom) in a kitchen environment (bottom). This is not a cherry-picked example; we can release an annotated version of the entire LIBERO dataset across multiple tasks, showing that learned discrete skills consistently hold across distinct tasks and environments. Additionally, we have visualizations that show that for a given discrete skill, there is significant overlap in the continuous parameters chosen for different tasks, ruling out the possibility of overfitting to $z$. This further validates that DEPS learns robust motor primitives. We are happy to update the project website with such visualizations for the reviewers' interest during the discussion period, provided the AC gives approval or clarification on the website update policy; regardless, we will include such visualizations in the final camera-ready.

(On compression aggression) To validate the importance of 1D state compression to DEPS’ strong performance, we compare the mean success rates and the mean highest success rates on the LIBERO-OOD test setting for DEPS with 1D state compression (results from the paper) against DEPS with 3D state compression. The results for 3D state compression are averaged across 3 seeds.

The dramatic performance drop from 1D to 3D compression (0.34 → 0.05 mean success rate) suggests that aggressive compression to a single dimension is crucial to DEPS’ performance.

Q5: Evaluation horizon – report performance after full convergence on the new task. Including both “rapid-adapt” and “fully-finetuned” results would clarify absolute capability and asymptotic ceiling.

A5: Our choice to evaluate after limited gradient steps (500 steps) is intentional and reflects the core motivation of our work: to learn transferable representations that enable rapid adaptation to new tasks. We believe this evaluation protocol serves as a stronger test of generalization ability as it demonstrates that DEPS learns genuinely transferable skills rather than simply learning the downstream tasks from scratch.

To evaluate DEPS’ asymptotic ceiling, we’ve performed additional experiments comparing the performance of DEPS and BC on the LIBERO-OOD evaluation setting with 5x as much finetuning (i.e. 2500 gradient steps). We perform rollouts after every 250 gradient steps of finetuning and report the Mean Success Rate and the Mean Highest Success Rates. The following results are averaged across 3 seeds:

We find that our method is still better than the baseline, and the gap between DEPS and BC actually increases.

We appreciate the constructive feedback, which has helped improve the experimental results in our paper. Please let us know if you have any further questions, and we hope you will consider raising your score in light of our response and additional results.

1. Yes, 20 passes refer to 20 full epochs of pretraining on the LIBERO dataset (80 tasks).
2. The reviewer indeed raises an interesting point. As we partially explained in Appendix G, the success rate discrepancy can be explained by the inherent design of the datasets. The key feature of LIBERO-10 is that it consists largely of in-distribution but longer-horizon tasks, while LIBERO-OOD consists of novel objects and environments that the agent has never seen during pretraining and the horizon is similar compared to the pretrained tasks (10 unseen tasks from LIBERO-90, which is shorter-horizon). The performance of DEPS is not as strong on LIBERO-10 (but still outperforms alternative baselines), compared to LIBERO-OOD. A potential reason for this is our choice to aggressively compress observations to one dimension before passing them to the subpolicy network, which enables generalization, potentially at the cost of perfect reconstruction of long-horizon demonstration trajectories. The observed tradeoff may be acceptable in many practical robotics scenarios, where learned skills from demonstrations typically serve as a starting point for further refinement using RL. In such cases, the initial generalization capability is more valuable than perfect execution on specific, lengthy sequences. We believe that presenting LIBERO-OOD/LIBERO-3-Shot alongside LIBERO-10 evaluation results presents a complete picture of DEPS’ capabilities - highlighting how DEPS can not only handle long-horizon skill chaining but also novel task generalization, all under the same general framework.

We thank you for your continued interest and engagement, and hope we were able to thoroughly answer your questions.

Thank you for the positive evaluation and thorough comments. We seek to address your questions below.

Q1: [paraphrased] The choice of a one-dimensional state compression is arguable

A1: This is a natural question. To answer this question, we ran additional experiments to compare the mean success rates and the mean highest success rates on the LIBERO-OOD test setting for DEPS with 1D state compression (results from the paper) against DEPS with 3D state compression. The results for 3D state compression are averaged across 3 seeds.

The dramatic performance drop from 1D to 3D compression (0.34 → 0.05 mean success rate) suggests that aggressive compression to a single dimension is indeed crucial for DEPS effectiveness, likely due to reduced overfitting to training environments.

In response to the reviewer’s comment that “authors seem to hint at the fact that adopting a higher dimensional state for compression might help…”, we would like to clarify this potential misunderstanding. As can be seen in the empirical result above, in the case of LIBERO-OOD, adopting a higher dimension for state compression does not increase performance. In the Limitations Section, we suggest that training a residual policy [1] which has access to the full observation space and augments the output of the original subpolicy (which will still utilise 1D state compression), may be a promising approach - we leave an evaluation of this to future work.

[1] Silver et. al, Residual Policy Learning. 2018

Q2: As mentioned, it seems that in some cases, the additional complexity of the proposed method doesn't pay off in terms of improved performance, especially in longer horizon tasks. Can you point out which are the cases where, instead, there is a clear benefit and the improved performance clearly justifies the additional complexity?

A2: You raise an important point about the relatively weaker performance of DEPS on LIBERO-10. We would like to stress that the tasks in LIBERO-10 mostly consist of concatenations of tasks in the pretraining dataset, and therefore, there is a minimal distribution shift between the pretraining and finetuning tasks in this setting. This is precisely the scenario where we'd expect less benefit from parameterized skills, as there is no need to generalize to new tasks or environments. We would like to note that despite the in-distribution nature of LIBERO-10, DEPS still performs better than PRISE and BC.

We expect DEPS to perform strongly on long-horizon and out-of-distribution tasks, and leave an evaluation of this to future work.

We believe that there are several scenarios where there is a clear benefit to using DEPS:

1. Out-of-Distribution Generalization (LIBERO-OOD, page 7 Table 1): DEPS shows substantial improvements when generalizing to entirely unseen environments and objects:

• Mean success: 0.34 ± 0.08 vs BC: 0.15 ± 0.04 (127% improvement)

• Mean highest success: 0.66 ± 0.12 vs BC: 0.36 ± 0.08 (83% improvement)

  This represents the core scenario DEPS is designed for - adapting learned behavioral primitives to novel contexts. We believe this scenario is highly relevant for robotics applications, where robots have to deal with limited training datasets and extremely diverse tasks and environments..

2. Low-Data Regimes (LIBERO-3-shot, Table 1): For robotics applications, there often is very little demonstration data available for downstream tasks. When reducing the amount of training data available for downstream tasks, the gap between DEPS and other baselines increases.

• Mean success: 0.26 ± 0.03 vs BC: 0.11 ± 0.05 (136% improvement)
• Mean highest success: 0.49 ± 0.03 vs BC: 0.22 ± 0.08 (123% improvement)

3. Limited Pretraining Compute (Table 2): When reducing the amount of pretraining performed, DEPS significantly outperforms other baselines

• At 5 epochs: DEPS 0.24 ± 0.08 vs BC 0.07 ± 0.03 (243% improvement)

• At 15 epochs: DEPS 0.39 ± 0.05 vs BC 0.12 ± 0.06 (225% improvement)

  This suggests that, in addition to improving downstream generalization to out-of-distribution tasks, DEPS’ learned abstractions improve the efficiency of pretraining (page 8, section 5.4).

We will add a discussion on the specific regimes where DEPS is useful in the revised version of the paper.

Q3: Are discrete skills transferable from tasks to tasks?

A3: Yes! As shown in Figure 7 (page 9), DEPS discovers semantically meaningful skills corresponding to action primitives such as grasping objects, closing doors etc. Importantly, we find that the discovered skills are applied consistently across tasks. An example can be seen in Figure 7 where the same grasp_object skill is used to pick butter in a cabinet environment (top) and pick a mug (bottom) in a kitchen environment (bottom). This is not a cherry-picked example; we can release an annotated version of the entire LIBERO dataset across multiple tasks, showing that learned discrete skills consistently hold across distinct tasks and environments. The transferability of the learned discrete skills is supported by the observation that there is significant overlap in the distribution of continuous parameters used for a given discrete skill across different tasks. It is precisely this non-overfitted continuous parameterization that enables the discrete skills to easily generalize and transfer to unseen tasks.

Q4: Are discrete skills interpretable (eg grasp, lift, ...)?

A4: The learned skills are interpretable and correspond to action primitives such as object grasping/releasing, door closing, knob turning, etc. An example of this is provided in Figure 7. Furthermore, the learned continuous parameters represent smooth modulations in the behaviour of the discrete skill (visualizations are on our website, which was already included at the time of submission; the URL is in section 5.5).

Q5: What are low-level actions $a_t$ (joint positions, velocities, cartesian, ...)? Is one choice of the other preferable in your formulation?

A5: The action space in LIBERO consists of a 6-dimensional continuous-values vector representing OSC_POSE, which consists of the desired position and orientation of the end effector. In addition, there is a binary value representing the gripper state (opened/closed). In line with previous work, we train our policies to output a Mixture of Gaussians representing a desired distribution over actions.

In Metaworld, the action space consists of a 3D vector representing the desired change in the end effector position as well as a binary variable representing the gripper (opened/closed). In line with previous work, we use a deterministic policy that outputs the desired action.

We believe that DEPS is compatible with diverse low-level action types. Our empirical results on LIBERO and Metaworld use their standard respective environment action spaces, as also used in previous works, with minimal differences in the hyperparameters used.

Q6: How do the representation of skills learned by your method compare to those from Prise's action abstractions?

A6: Unfortunately, to the best of our knowledge, PRISE does not provide visualizations of its learned action abstractions.

PRISE learns discrete action abstractions, each of which represents a fixed sequence of “action tokens”. We believe the parameterized skills learned by DEPS, which can be varied smoothly by changing the continuous parameter and do not have a fixed length (unlike in PRISE), present a more flexible and interpretable representation.

Q7: [Figure 7] "We find that general skills corresponding to grasping, moving, and releasing objects are learned and consistently applied across visually diverse tasks." -- are these skills learned every time or are they "transferred" through the learned intermediate level of the architecture?

A7: The learned skills are transferred across the intermediate level of the architecture. This is supported by several aspects of our experiments, including:

1. The same discrete skill is consistently applied in all relevant contexts. For example, discrete skill 9 (“grasp_object”) (as shown in Figure 7) is applied every time an object needs to be grasped, irrespective of the task.

2. Table 2 (page 8) shows that the gap between DEPS and other baselines increases when the number of pretraining epochs is decreased. This supports the argument that the learned skills are being used across tasks, therefore increasing the efficiency of pretraining through cross-task generalization.

Thank you again for your constructive feedback, which has helped improve the clarity of several key aspects of our paper. Please let us know if you have any additional questions.

We thank you for your thorough review and positive comments. We address your concerns below.

Q1: Fig. 2 is confusing. I am not sure what the gray areas represent, and the components could be explained better.

A1: As denoted in Figure 2’s legend, the gray area represents the variables that are observable to each policy/encoder. For example, the variational encoder can see the whole trajectory history and future, while the low-level policy can only see the previous transitions and the skills being selected at the current time step. We will clarify this further in the revision.

Q2: Please include an ablation analysis showing the benefit of compression and tradeoffs with different compression sizes for $s_t$.

A2: We thank the reviewer for the helpful suggestion. We have added a set of new experiments, comparing the mean success rates and the mean highest success rates on the LIBERO-OOD test setting for DEPS with 1D state compression (results from the paper) against DEPS with 3D state compression. The results for 3D state compression are averaged across 3 seeds.

The dramatic performance drop from 1D to 3D compression (0.34 → 0.05 mean success rate) suggests that aggressive compression to a single dimension is crucial for the method's effectiveness.

Q3: A highly compressed st means that $k_t$ and $z_t$ are forced to encode all task-critical information - perhaps even information that isn’t relevant to skill. Could the authors provide more compelling results showing that $k_t$ and $z_t$ encode only skill-relevant information?

A3: We thank the reviewer for the insightful comment; it is indeed important to ensure that $k_t$ and $z_t$ encode skill-relevant information rather than just general task information. One of DEPS’ main goals is to ensure this precise behavior. As outlined in Appendix A1, during pretraining, the variational network predicts continuous parameters per discrete skill for each forward pass over a given trajectory (instead of predicting a new continuous parameterization at each time step, which could lead to overfitting). Additionally, to prevent the continuous parameters from overfitting to specific tasks, we introduce the $\mathcal{L}_{\text{norm}}$ (Appendix A1) regularization term, which bounds the range of the learned continuous parameters. This limits the amount of information encoded in the discrete skill and continuous parameters, forcing the 1D state to encode meaningful information as an “index” into parameterized skill trajectories. This is validated in Figure 8, where the learned latent 1D states vary monotonically within discrete skills, with discontinuities when the discrete skill switches. The strong performance of DEPS on out-of-distribution downstream tasks (Table 1, LIBERO-OOD) further supports our hypothesis that DEPS learns meaningful abstractions instead of overfitting to $k$ or $z$ values. This is particularly evident in low-data regimes (Table 1, LIBERO-3-shot) and low pretraining settings (Table 2).

We provide several lines of evidence that $k_t$ and $z_t$ do encode skill-relevant information:

1. Consistent application of discrete skills (Section 5.5 and Figure 7): We find that DEPS discovers semantically meaningful discrete skills corresponding to primitive behaviors such as grasping objects, opening doors, etc. Importantly, we find that the discovered skills are applied consistently across tasks. An example can be seen in Figure 7, where the same ``grasp_object’’ skill is used to pick butter in a cabinet environment (top) and pick a mug (bottom) in a kitchen environment (bottom). This is not a cherry-picked example; we can release an annotated version of the entire LIBERO dataset across multiple tasks, showing that learned discrete skills consistently hold across distinct tasks and environments.

2. Smooth variation in policy on varying continuous parameters (Section 5.5): We find that for a given discrete skill, slight modifications in the continuous parameter result in smooth variations in the resulting policy. For example, modifying the continuous parameters for a discrete skill used to turn a knob smoothly varies the location at which the knob turning action is executed. Visualizations of this are provided on our website (which was already included at the time of submission; the URL is in Section 5.5).

3. Overlap of skill-specific continuous parameterizations across tasks: We have visualizations that show that for a given discrete skill, there is significant overlap in the continuous parameters chosen for different tasks, which suggests that $z$ only encodes skill-specific, and not task-specific, information. We are happy to update the project website with these visualizations for the reviewers' interest, provided the AC gives approval or clarification on the website update policy; regardless, we will include such visualizations in the final camera-ready.

Q4: Please explain if and why $z_t$ changes at each timestep.

A4: During pretraining, $z_t$ is not changed at every timestep. Instead, $z_t$ can only change when the sampled discrete skill changes. This is essential to ensure that the learned parameterized skills are temporally extended, and helps prevent the model from taking degenerate shortcuts (such as encoding the action at each timestep directly in the continuous parameter) that would prevent the model from learning meaningful abstractions. Details on this can be found in Appendix A1.

On the other hand, during finetuning, we allow the model to choose to change its choice of continuous parameter at each timestep if necessary. Our goal during the finetuning phase is to tune the abstractions learned during pretraining to maximize performance on downstream tasks. By modifying its continuous parameter at each timestep, the robot can “close the planning loop” and therefore more flexibly react to potential stochasticity (or unexpected outcomes) at inference time. Details on this can be found in Appendix A2.

In summary, during pretraining, our goal is to learn reusable abstractions/compressions, and therefore, we do not change $z_t$ at each timestep to encourage compression. On the other hand, during inference, we allow $z_t$ to change at each timestep as our goal is to use learned specialized abstractions to maximize performance on a task.

We appreciate the reviewer’s thoughtful comments and suggestions on strengthening our work, and welcome further opportunities to clarify any additional questions that may remain.

Thank you for the clarifications. I find the result of decreased performance with 3D compression very interesting. I think it would add to the paper if the authors include this result and short discussion about why they see this result. Otherwise, the authors have adequately addressed my concerns and I will keep my score of 5