The assumption of orthogonality for the validity of the coefficients is vital. However, both the authors in [13] and in this paper do not share any analysis on the validity of this assumption. It would be interesting to see how well this holds in practice (computing inner product of obtained basis functions/ODEs should be easy) and how many iterations are required to reach orthogonality in the examples provided.

Please see the response to all reviewers, section 3. Also section A.5 in [13] may be of interest.

The introductory paragraph of section 3 where the link between a new unknown function $f$, its observed trajectory $D$, and ODE basis functions is made, could benefit from more motivation and explanation. It is not immediately clear why reasoning at the derivative level while dealing with integral trajectory data is more advantageous. Perhaps a figure would help as the manuscript is text heavy.

To clarify, in continuous time dynamical systems, the dynamics is inherently a differential equation that is being integrated through time. However, it is typically the case that system observations occur at discrete time intervals, and do not include derivative measurements. Hence, neural ODEs are advantageous because they allow you to learn a model of the true system derivatives while using integral trajectory data.

Explanations of the MuJoCo results are not very clear to me, they seem more like observations than interpretations. Why is the oracle unstable (does it require more data and training given the conditioning on hidden params)? What explains the differences between both experiments: FE + RES does well on half cheetah (better than FE + NODE) and terrible on ant? Where does the lack of inductive bias intervene in this case ?

The apples-to-apples comparison is between FE + Res. and FE + NODE + Res. These two approaches differ only in basis function architecture, and FE + NODE + Res. consistency outperforms. The other relevant comparison is FE + NODE + Res. and the Oracle. These two approaches both benefit from the inductive bias of neural ODEs, and both have information on the hidden parameters. However, the format of the information is different. Oracle gets this information explicitly as an additional input, whereas FE + NODE +Res. implicitly gets this information via the coefficients of the basis functions. We expect that Oracle is performing badly because it is generalizing to an entire space of dynamics with one NODE, which is a very complex function to learn. We believe that Ant is much more difficult and varied than Cheetah, and the relative ease of Cheetah is likely why both Res. methods perform so well on Cheetah. We have improved the discussion in section 4.2 to include these details.

The term zero-shot does indeed refer to the ability of networks to perform in novel circumstances without retraining, and does apply here. However, it can be slightly misleading as adaptability here requires analysis of new data from the new setting. Can the authors elucidate the scales involved in the tradeoff? In other words, when does it become interesting to train 100 base models (on the 100 datasets) to gain a deployment advantage that still requires new data to function, versus fine-tuning one model or retraining?

Two things to note. Rather than training a function for every dataset, we train $k$ basis functions to span all datasets. So, if we have 100 datasets but only want to train 10 basis functions, this is possible without issue. Secondly, it does require a small amount of online data. This data is necessary for any method because you need some information to identify the current dynamics. For example, finetuning a neural ODE based on online data is another approach, but this also requires online data. Indeed, finetuning a model likely requires even more data than our approach, and incurs a significant computational overhead that is going to take significantly longer than our approach. In contrast, we can compute the coefficients of the basis functions in milliseconds, as the inner product calculation is effectively a sample mean, and then we can instantly adapt our predictions based on the data.

Do the authors have any insights on changes in levels of performance whether the new ODE is in the convex hull of the available basis or outside it? (for example in the Van der Pol example, if the basis datasets contain trajectories for values of $\\mu$ between 0.1 and 3, can the system perform for $\\mu=5$?

Examining OOD generalization is an interesting future direction. Please see the response to all reviewers, section 2, for the experiment you described.

A major numerical challenge working with continuous time neural networks is that of integration, with a plethora of solvers and schemes available. The authors touch upon this topic in their limitations sections and the impact of the integration horizon selection on behaviour predictability as well as the compute overhead involved (also in appendix C). It would have been interesting to give readers a better sense about the tradeoffs with numerical comparisons that go beyond verbal description.

We leverage RK4, and there are indeed more accurate integrators available. However, there is inherently a trade off with respect to both training time and execution time. Integrators such as adaptive step size solvers can potentially make 20 or more calls to the neural ODE during a forward pass, while RK4 makes only 4. The increased number of neural ODE forward passes greatly increases memory usage and compute time. We experimented with better integrators, but ultimately found this tradeoff to be unfavorable. We find RK4 to be the right balance of speed and accuracy for this work. We have added an additional section of the appendix discussing the choice of integrator.

The paper presents a promising framework for zero-shot transfer of neural ODEs, but there are several areas for improvement. Firstly, the reliance on a large and diverse dataset for training is a significant limitation. The approach requires extensive data that spans the entire function space of possible dynamics, which may not always be feasible. This dependency on comprehensive datasets should be addressed by exploring data-efficient learning methods or leveraging transfer learning techniques.

Any approach that aims to model a large set of dynamics from online data inherently must learn from a large dataset. This is because the space of functions is infinite dimensional, and this large, diverse dataset is effectively teaching the model which subspace is most important. Therefore, the ability to infer behavior from data necessitates the presence of a large training dataset.

Our approach learns basis functions to span the functions present in the training set. However, it’s important to note that any function in the span of the basis can be perfectly represented, and thus our approach is an efficient transfer learning technique for this domain. However, examining strategies for further improving data efficiency is an interesting future direction. For example, it may be possible to augment the training dataset with additional common functions, such as linear transformations between inputs and outputs or trigonometric functions, as this would increase the diversity of the training dataset without requiring more data.

Secondly, the computational overhead involved in training multiple neural ODEs is substantial. This might hinder the scalability and real-time applicability of the proposed method. The authors could investigate more efficient training algorithms or consider approximations that reduce computational costs without compromising accuracy.

Overhead is indeed an important consideration, and we have two strategies to address this. During the offline training phase, the key consideration is memory usage. This is because neural networks converge faster with larger batch sizes, and so leveraging the largest batch size possible is key. To address this, we run the basis functions sequentially during training. Since only one basis function is being called at a time (both during the forward pass and back-propagation), the memory overhead is the same as a single neural ODE. This does increase the training time by a factor of $k$. However, as neural ODEs alone cannot model multiple dynamics functions, we view this cost as acceptable for the benefits our approach provides.

During the online phase, memory usage is inherently reduced as all memory usage results from online data only. So, memory is not a concern. Instead, the compute speed is the most important factor for real-time control. To address this, we run the basis functions in parallel. As there is no dependence between them, each basis function can run at the same time. Thus, by running the basis functions in parallel, there is no additional compute time overhead. Therefore, this method is equivalent to a vanilla neural ODE with respect to real-time compute speeds.

We have improved the discussion in the appendix, section C, to include this additional information.

Thirdly, the paper does not explicitly enforce the orthogonality of basis functions, relying instead on implicit regularization through the loss function. This might lead to suboptimal representation of the function space, affecting the model's performance.

Please see the Response to all Reviewers, section 3.

Moreover, the experiments lack diversity in evaluating the model's robustness to completely unseen environments or significantly different conditions. Including more varied test scenarios or stress tests could provide a deeper understanding of the model's adaptability and limitations.

Verifying OOD generalization capabilities is an interesting future direction. Please see the response to all reviewers, section 2.

The experiments lack diversity in evaluating the model's robustness to completely unseen environments or significantly different conditions. How confident are you in your model’s adaptability in such scenarios?

Please see the response to all reviewers, section 2. It’s worth noting that the test environments are sampled from the same distribution of environments, but are unseen by the model during training.

The evaluation metrics used in the MuJoCo experiments are primarily focused on prediction accuracy. Have you considered additional metrics that might better capture your model's practical performance? For example, task-specific performance such as scores or cumulative rewards.

The MuJoCo experiments are designed specifically to highlight prediction accuracy. We agree that other metrics might be useful as well, and so we leverage a task-specific efficiency metric in the quadrotor experiments. For more information see section 4.3, especially the last paragraph.

It seems to lack a sufficient variety of baselines for comparison. Have you considered additional zero-shot / few-shot or multitask / meta-learning studies?

We are unaware of any other zero-shot techniques for neural ODEs. It is possible to use a transformer for the same type of data, however transformers suffer notable drawbacks. First, their memory usage is quadratic, and so they are unable to make use of large, online datasets. Second, their forward pass time is quite slow, and so they are not suitable for real time model-predictive control as many forward passes must be made every timestep, e.g. 100s of forward passes in 30 milliseconds. Meta-learning techniques are similar in that they adapt models given instance-specific data. However, these techniques perform additional finetuning or retraining based on this data, which makes them ill-suited for real-time control.

The authors' rebuttal and general response have improved my understanding and addressed many of my concerns. As a result, I have revised my score to 5.

My main criticism of the paper is the technical contribution. The problems this system can solve seem to be constrained to systems limited variation in parameter, where the offline dataset & online system share a high level of similarity. While the authors explained how they apply NODE very clearly, I think using NODE is not a fundamental contribution because you can also have other sequence models trained to predict residue. The idea of basis function has also been explored by many prior literatures.

• We use hidden environmental parameters as an example problem setting because it is frequent in robotics. However, we never make use of the hidden parameters directly. As mentioned in Section 3, this algorithm can be applied to a space of dynamical systems that arises for any reason.

• Generalization to OOD online systems is indeed an interesting direction. Please see section 2 of the response to all reviewers for a discussion.

• We are interested in model-based control. Sequence prediction models, e.g. transformers, are often too computationally complex for model predictive control. SOTA control algorithms such as iCEM MPC need hundreds of forward passes per control-action. Transformers are well known to have quadratic cost with respect to input space, and their forward pass is much slower than neural ODEs, and so they are not suitable for this setting. Meta-learning algorithms can be applied under the same data assumptions, but often finetune or retrain the learned model based on new data. This training period is not amenable to real-time control. Lastly, there is indeed a plethora of analytical basis function methods. However, analytical basis functions often scale extremely poorly to high-dimensional function spaces. Other approaches like SINDy have similar scaling issues, but also require a potentially lengthy sparse coefficient identification procedure which can be too time-consuming for real time control. In contrast, our method enjoys the same mathematical interpretation as these approaches, while scaling to high-dimensional problems due to the use of a relatively small number of neural network basis functions. We have extended the related works to include information on why common sequence models and meta-learning are not applicable to this setting.

The title is misleading - you are doing quick online adaptation (few shot) instead of a zero-shot setting.

The term “zero-shot” means different things to different communities. For example, the LLM community will often say “few-shot prompting” to mean operating a model with a few examples as input, and no finetuning. However, for ODEs, dynamics modeling, and reinforcement learning, “Zero-shot” is the ability to leverage unseen data at execution time to improve model performance without additional training, where the zero comes from zero gradient steps. See “Does Zero-Shot Reinforcement Learning Exist?” (Ahmed Touati, Jérémy Rapin, Yann Ollivier, 2022), “Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodings“ (Kevin Frans, Seohong Park, Pieter Abbeel, Sergey Levine, 2024), “Zero-Shot Reinforcement Learning via Function Encoders” (Tyler Ingebrand, Amy Zhang, Ufuk Topcu, 2024), etc.

A few trajectories of online data is required to adapt to the new system.

For the Mujoco examples, we only leverage 200 data points. For a system running at 30 Hz, this is equivalent to only ~7 seconds of data rather than a full trajectory. This data can then be used to identify the system dynamics immediately at runtime. We have improved the experiment description in section 4.2 to clarify this.

In line 114, the reasoning doesn't provide grounding to the orthogonality question, ..., Are you saying that non orthogonal is okay as soon as they span the space?

Thank you for your comment, we have revised the wording of line 114 for clarity. Please see the response to all reviewers, section 3 for more information on orthonormality.

It would be good to give more intuitive visualizations of estimation error in mujoco Ant. e.g. render the predicted trajectories as video.

Good idea, we will add screenshots to the appendix.

Ablation about the number basis functions used would be helpful to understanding.

We agree that the effect of the number of basis functions is an interesting ablation. Please see the response to all reviewers, section 1a.

In line 165, the authors said "Given data collected online from a single trajectory". I am curious to see ablations how the method improves as the amount of online data grow.

We agree that examining the prediction accuracy as the amount of data increases is an interesting experiment. Please see the response to all reviewers, section 1b.

In figure 3, error goes up as the number of look-ahead steps increases. While this is partially expected due to compounding error, how much of it shall be attributed to limited expressiveness of linear basis. e.g. I can do an experiment where I train a NODE with many many data, not just 200 example points, and roll it out. You will witness how MSE loss increases and gain insights about a upper bound for prediction accuracy that's not due to limited data / limited number of basis.

Please see the response to all reviewers, section 1, which may provide some more intuition on the effect of the example data and number of basis functions on the performance. While using more example data slightly improves performance, a key feature of our approach is that it can operate in low data settings. Additionally, the method is insensitive to the number of basis functions, which implies that the basis functions are more than sufficient to express the dynamics. Lastly, “NODE” and “Oracle” are also neural ODE models, and so they suffer from the same compounding error problems as our approach. Thus, those approaches provide a reference for how much of the error is due to the inherent difficulty of dynamics prediction.

Thank you for your feedback. See below for responses to your suggestions.

To place the results in context with prior work, it would be additionally helpful for the robotics experiments to show comparisons to other methods that enable adaptation (e.g. training a model free method with domain randomized parameters).

Domain randomization allows RL algorithms to find policies which are robust to randomized parameters, however the policy does not adapt to those parameters. Our approach is inherently solving a different problem, in that our controller can adapt to the specific dynamics it is experiencing. This is done by using a small, online dataset to identify the coefficients of the current dynamics function. Then, the basis functions (in combination with the coefficients) are used to model the current system dynamics. A controller using this system model therefore adapts to the current dynamics in an online fashion.

It’s not clear how well the method will scale to more complex dynamics, or when trying to generalize beyond the learned basis functions (e.g., if they are not expressive enough or span the relevant spaces for a dynamical system, or if insufficient data is used to learn the dynamics space).

The ablation in the response to all reviewers, section 1a, shows that the basis functions are sufficiently expressive to span the space of dynamics present in the system. Furthermore, as $L_2$ is a Hilbert space, any function in $L_2$ can be perfectly described as a linear combination of basis functions. In other words, basis functions are sufficiently expressive for the vast majority of problem settings.

Generalization beyond the function space in the dataset is a fascinating question. Please see the response to all reviewers, section 2 for more information.