Two-shot learning of continuous interpolation using a conceptor-aided recurrent autoencoder

We thank you for your feedback, which we would like to address as follows

1. Acknowledging the Relevance of Older References:
   We would like to point out that the age of a reference does not undermine its validity. In fact, older concepts often find renewed significance in contemporary research. Additionally, linear subspace characterization linear subspace control did have recent successes by being effectively combined with backpropagation to address catastrophic forgetting issues, as highlighted in recent studies [1], [2]. Also, the concept of low-dimensional dynamics has recently gained traction in computational neuroscience, serving as a foundation for rich theoretical and experimental work to understand the dynamical aspect of the brain [3]. Our research proposes a novel interpretation of these phenomena by connecting it to the rich field of learning representation and autoencoding in Deep Learning.

2. Differentiating from Jaeger’s Work and Aligning with Current SOTA Research:
   Our approach represents a significant departure from Jaeger’s initial work [4], where linear subspace control was not linked to learning with RNNs but to store memories akin to Hopfield Networks but for time series. Instead, our methodology is more closely aligned with contemporary SOTA research in meta-learning [5], where some meta-parameters (equivalent of the conceptor) are multiplicatively interacting with a base model to improve the learning data-efficiency. These recent studies recognize and utilize the potential of linear subspace control in developing advanced learning models.

3. Emphasizing the Role of RNNs in Deep Learning:
   In the realm of deep learning, it's crucial to note that RNNs stand on par with other architectures like transformers on data-sparse domains. This includes the distillation and compression of locomotion policies into multi-task policy networks [6, 7]. Finding good compression principles is recognized as promising to be more data-efficient. Moreover, various RNN architectures are increasingly challenging transformers in terms of reducing computational costs and offering unbounded context length [7]. Additionally, some RNNs architectures have been shown to be very close to attention-based computations [8].

4. Extending the CARAE Architecture:
   As mentioned in our responses to other reviewers, we have expanded our work in the appendix. Here, we demonstrate that the CARAE (Conceptor-Aided Recurrent AutoEncoder) architecture can be extended to scenarios involving more data, higher-dimensional bottlenecks, and other commonly used architectures like the LSTM. Furthermore, we adapted the CARAE to static Feed Forward Networks (FFWs) like MLPs applied to time series prediction, theoretically allowing for application to transformers.

[1] X. He and H. Jaeger, “Overcoming Catastrophic Interference using Conceptor-Aided Backpropagation,” presented at the International Conference on Learning Representations, Feb. 2018. Accessed: Aug. 21, 2023. [Online]. Available: https://openreview.net/forum?id=B1al7jg0b
[2] L. Duncker, L. Driscoll, K. V. Shenoy, M. Sahani, and D. Sussillo, “Organizing recurrent network dynamics by task-computation to enable continual learning,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2020, pp. 14387–14397. Accessed: Aug. 17, 2023. [Online]. Available: https://proceedings.neurips.cc/paper/2020/hash/a576eafbce762079f7d1f77fca1c5cc2-Abstract.html
[3] “Low-rank RNNs in ten minutes,” neurosopher. Accessed: May 31, 2023. [Online]. Available: https://adrian-valente.github.io/2022/06/01/low-rank-summary.html [4] H. Jaeger, “Controlling Recurrent Neural Networks by Conceptors,” arXiv:1403.3369 [cs], Apr. 2017, Accessed: Dec. 09, 2020. [Online]. Available: http://arxiv.org/abs/1403.3369
[5] N. Zucchet, S. Schug, J. von Oswald, D. Zhao, and J. Sacramento, “A contrastive rule for meta-learning.” arXiv, Oct. 03, 2022. Accessed: Jul. 07, 2023. [Online]. Available: http://arxiv.org/abs/2104.01677
[6] J. Merel et al., “Neural probabilistic motor primitives for humanoid control.” arXiv, Jan. 15, 2019. Accessed: May 05, 2023. [Online]. Available: http://arxiv.org/abs/1811.11711
[7] N. Wagener, A. Kolobov, F. V. Frujeri, R. Loynd, C.-A. Cheng, and M. Hausknecht, “MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control.” arXiv, Jan. 13, 2023. doi: 10.48550/arXiv.2208.07363.
[8] D. Y. Fu, T. Dao, K. K. Saab, A. W. Thomas, A. Rudra, and C. Ré, “Hungry Hungry Hippos: Towards Language Modeling with State Space Models.” arXiv, Apr. 28, 2023. doi: 10.48550/arXiv.2212.14052.
[9] “RNNs strike back,” neurosopher. Accessed: Nov. 18, 2023. [Online]. Available: https://adrian-valente.github.io/2023/10/03/linear-rnns.html

Thank you for your critical feedback. We have made several amendments to our manuscript to address your concerns, particularly regarding the evaluation methodology.

We agree with the need for objective evaluation metrics, and we have devised a quantitative metric in the revised manuscript. In this context, we would not like to use a third pattern to evaluate our method as the interpolated temporal patterns are based on probabilistic interpolation related to the training data. A third example of a parametric family could contain features that are absent or only weakly present in the training data and therefore can not be inferred by the CARAE mechanism. Using such third patterns for evaluating our method could lead to false understanding of what our method aims to achieve. To quantitatively measure the performance of the interpolated temporal patterns we have developed a similarity metric specifically tailored to our task. This metric provides a probabilistic measure and a temporal correlation measure of the generated patterns' fidelity to the training data, addressing the subjective nature of our initial evaluations. Further details on this metric, including its formulation and application, are now included in the appendix and referenced in the manuscript. Finally, we would also like to note that auto-encoders are often judged on their output re-generation with subjective judgments [1]. Motion modeling also [2] because one of their main goals is to develop plausible motion for simulated characters. To provide further clarity on the role of conceptor loss and its regularization term, we included in the appendix an ablation study of the regularization parameters beta_1 and beta_2. This addition aims to elucidate the identifiable effects of conceptor loss on controlled neural networks, also illustrating the applicability of the newly introduced evaluation metrics. Both regularization parameters show different influences for the two evaluated tasks. Thereby, we find that the usage of both is needed to significantly improve the generalization during the interpolation.

In response to your remarks on SPD matrices, we would like to discuss our choice of the linear interpolation. We also discussed alternative metrics like log-euclidean and interpolation methods for SPD matrices, as mentioned in the referenced literature. However despite its simplicity, the linear interpolation approach preserves the SPD matrix properties [3], ensuring that the interpolated conceptors are still valid soft projection matrices. Furthermore, we decided to focus on linear interpolation due to its computational efficiency. Other metrics include the usage of matrix logarithm and exponential that are computationally expensive for the forward and backward-pass and could potentially introduce instabilities in the training. Furthermore, these functions are often not implemented in standard deep learning libraries making their usage more complicated. Nevertheless, as it is computationally more feasible, we tested the usage of the log-euclidean and the shape-and-orientation (SAO) metric during the inference while relying on the linear notion for the training. The log-Euclidean and SAO metrics are developed to tackle known challenges in statistical operations involving semi-definite matrices. One such problem is the "flattening effects" [4]. This refers to the deformation and shrinkage of singular values that occur during linear interpolation. However, in our study, we found that these metrics did not significantly impact the inference process. Whereas this might be related to the particular task at hand we agree and mention in the paper that a more rigorous investigation of the influence of the metric might further improve our results. Due to the limited time, we started this investigation of the metrics for inference only and attached an initial discussion to the appendix as section F.

Minor remarks:

We adapted the references in the revised manuscript making it more clear to which figure/table/equation we refer.

In summary, these revisions and additions address your concerns by enhancing the objective evaluation of our approach and justifying our methodological choices. We believe these changes strengthen the manuscript and look forward to your further feedback.

Thanks for these answers.

I still have some concerns of the lack of evaluation with respect to other possible approaches. It is especially difficult to see the benefit of the proposed quantitative metric without further details. For instance, if to consider work done in [1] how could we position it here. In [1] use of auto-encoder is also proposed as well as a quality metric based on prediction metrics on further patterns. When I was suggestion 3rd pattern this was to go into that direction. To see how the proposed generation model could predict intermediate patterns.

Considering feedback on $\beta$ parameters also provide a mixed feeling since finally it seems quite sensitive on the considered use case as said in response from authors.

So i am rather enclined to keep my initial rating.

[1] Hughet et al, 'Manifold interpolating Optimal-Transport flows for trajectory inference', Neurips 22.

Thank you for your constructive feedback, which has led to significant improvements in our manuscript.

Interpolating trajectories is more difficult than filling out single-time steps

We thank you for your feedback and reference to the “MIOFlow” framework by Huguet et al. However, we would like to note that while both CARAE and MIOFlow leverage auto-encoder concepts to analyze time series, their aim, methodology, and applications diverge significantly.

• Very different task: CARAE focuses on interpolating between trajectories conditioned on a latent variable. MIOFlow only deals with filling held-out time points within a trajectory. They never use their methods on novel interpolated trajectories.
• Evaluation: because they don’t evaluate on interpolation between trajectories, it’s much easier to quantify by holding out a one-time point. We also remark that the referenced paper relies on many qualitative comparisons.
• Different degree of difficulty: We would like to argue that their task is relatively simpler because: Their task does not involve long-term stability issues because they deal with very short trajectories with simple dynamics: max 5-time steps versus around 150-time steps (of dimension around 100) for our work. Also, the video in the supplementary material shows that our pattern generator is very stable through transitions and at various levels. Our approach emphasizes the controlled generation of continuous pattern families from extremely limited data (as such data might be difficult to generate, particularly in RL settings, see other comments). They do not propose a mechanism to morph between trajectories while they are unfolding (as we showed in section 4.3).
• The bottleneck definition: MIOFlow employs a combination of dynamic models, manifold learning, and optimal transport to interpolate single timesteps between static population snapshots. While MIOFlow emphasizes a probabilistic framework to define their bottleneck, CARAE emphasizes the geometry of neural dynamics. MIOFlow's use of Neural ODEs and its emphasis on optimal transport with manifold ground distance distinguishes it from CARAE, which prioritizes the linear interpolation of conceptors and the generation of patterns specifically within a defined probabilistic framework.

Benchmarking with a quantitative measure.

We have now addressed your suggestion for quantitative evaluation using a third pattern to benchmark the performance of our CARAE framework, taking inspiration from the MIOFlow paper that you added as reference.

In our revised approach, detailed in Appendix I, we compared the performance of CARAE against a baseline method of simple linear interpolation between two distinct temporal motion patterns: walking and running. Our goal was to generate an intermediate time series that would ideally represent a jogging motion (the “correct” intermediate between walking and running from the dataset), and then measure the accuracy of these generated patterns against the ground truth jogging data available in our dataset.

We used MSE and JSTCC the as the metric for quantification. The results, as presented in Appendix I, demonstrate that CARAE achieves a lower MSE in comparison to the direct linear interpolation approach when evaluated against the ground truth jogging data. This finding substantiates the efficacy of CARAE in generating more accurate intermediate temporal patterns and also provides a clear, objective measure of its performance. This establishes a tangible benchmark for future research, offering a solid foundation upon which to evaluate, measure, and enhance our proposed framework, task, and solution.

We believe that this additional experiment and its results effectively address your concerns regarding the lack of quantitative evaluation and comparative analysis in our initial submission. The ability of CARAE to outperform a straightforward linear interpolation method in this context highlights its potential as a more nuanced and effective tool for interpolating between temporal motion patterns.

We hope that this new evidence, alongside the other revisions made in response to your comments, strengthens the case for the novelty and utility of our proposed CARAE framework.

We thank you for your valuable feedback on our work. We would like to address your concerns step-by-step in the following:

1. 1. We mention related works of interpolatable representation learning in different areas, including locomotion modeling, robotics, and reinforcement learning because they deal with certain challenges, like sparsity of data while aiming for stable periodic solutions that can be interpolated. We would like to argue that the CARAE framework, although not yet applied to RL learning tasks, shows promising results in the motion modeling using the MoCap data set. Thereby, we address and overcome several of the discussed challenges that yet could only be solved by massive computational effort.

   2. While our current focus is on two-shot learning, we see the potential for expanding CARAE to few-shot learning scenarios. Thereby, the loss function needs to be reformulated to include several conceptors, one per learned temporal pattern. The CARAE framework even enables us to encode into the loss function whether the patterns should be all-to-all interpolated or whether they should line up along a single dimension. We briefly outline potential extensions of our method in the manuscript's conclusion and in appendix E, discuss how CARAE could adapt to and benefit from additional training examples, thereby enhancing its practicality and applicability in a broader range of data-constrained environments.

2. We appreciate your concern about extending CARAE to more commonly used RNN architectures like LSTM and to transformer architectures. As CARAE is an architecture-independent mechanism for manipulating neural dynamics through conceptors, we did not expect any problems with this. Indeed, in response to your suggestion, we have extended CARAE to the commonly used LSTM architecture. Furthermore, we have outlined a version of the CARAE that relies on a feed-forward neural net. Both architectures are able to learn continuous interpolation of temporal patterns as we present in the appendix C. These extensions demonstrate CARAE’s versatility and significantly broaden CARAE's practical applicability.

Minor weaknesses:

1. We thank you for the careful reading and have streamlined the formulation in section 5 to avoid any confusion.

2. In the appendix, we added the introduction of a similarity measure (see revised manuscript) that evaluates the relation of our inferences to probabilistic features and temporal correlations of the training datasets. Based on this metric, we performed an ablation study of the two regularization parameters beta_1 and beta_2. The metric thereby adds a quantitative measure to evaluate the inferences beyond the loss terms we design, allowing for a more systematic investigation of our proposed method.

Regarding further questions:

The CARAE framework effectively addresses all four challenges outlined in Section 2. The incorporation of the conceptor in the autoregressive mode ensures stable dynamics, protecting against exploding dynamics, interferences, and side dynamics. Our explicit training approach within the CARAE loss term thereby specifically tackles fixed-point dynamics, a critical aspect when aiming for interpolation. These improvements are needed to yield stable interpolated temporal patterns when dealing with sparse data as showcased by the two experiments discussed in the manuscript. Additionally to the failing vanilla RNN used in the sine wave interpolation, we added unsuccessful experiments using a RAE on both training sets to the appendix.

In conclusion, we believe that our revisions and additional analyses respond comprehensively to your concerns, thereby significantly strengthening the contribution and clarity of our work. We look forward to further feedback!

I would like to thank the authors for the detailed response. Maybe I'm not the target audience of this work but I'm not fully convinced by the authors' arguments for the practical significance of its unconventional problem setting. The response acknowledges the importance of learning interpolatable representation in many areas including reinforcement learning, robotics, and locomotion modelling. To my best knowledge, none of existing works in these areas that showcased interpolatable representation has a similar problem setting. Can the author give examples of existing works that adopt such a setting or examples of realistic problem settings where learning interpolatable representation from two samples is necessary? In addition, I'm not sure if the results on MoCap data are enough to justify the practical significance of the work. Controllable human motion generation under various settings has been studied in many existing works [1, 2]. Moreover, there's no unique way of interpolating between two human motion trajectories that looks realistic. Therefore I do not see a direct and objective way of evaluating the interpolated trajectories except for indirect and qualitative evaluations. I would encourage the author to spend more efforts on justifying the practical importance the problem setting for a more general audience and design experiment settings to help support that in future revision.

[1] Petrovich, Mathis, Michael J. Black, and Gül Varol. "Action-conditioned 3D human motion synthesis with transformer VAE." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.

[2] Kania, Kacper, Marek Kowalski, and Tomasz Trzciński. "TrajeVAE: Controllable Human Motion Generation from Trajectories." arXiv preprint arXiv:2104.00351 (2021).

Thank you for your valuable feedback. Regarding the interpolation of different features, we have not yet incorporated a way to address different features independently. That means, in terms of your proposed colored digit interpolation where only 2 examples are given, both features (digit pattern and color) are interpolated at the same time guided by interpolating the conceptor during inference. However, and this would be partially related to the answer to reviewer 1, learning these disentangled factors of variation (digit pattern and colors) would require extending CARAE to the case where more than two-time series are present and a bottleneck of more than one dimension (in conceptor space). The multi-dimensional bottleneck is critical to discover the different factor of variations and to allow for feature-specific interpolation. Furthermore, once CARAE is expanded in this manner, it becomes possible to adapt techniques from the auto-encoder literature to promote disentanglement. For instance, one can enforce a probabilistic interpretation of the bottleneck and ensure the independence of its dimensions, as seen in VAE and Beta-VAE methods.

It's worth mentioning that the disentanglement of factors of variation holds significant relevance in the context of motion modeling. This is particularly important because a key challenge in achieving realism in simulating characters is the ability to portray various behaviors with distinct styles, such as depicting characters as drunk, sad, or excited [1].

In the appendix, we introduced a similarity metric (see the updated manuscript) to assess the connection between the intermediary inferences of the CARAE framework and the probabilistic attributes and time-related correlations to the training datasets. Utilizing this metric, we conducted an ablation analysis on the regularization parameters beta_1 and beta_2.

[1] Taylor, Graham & Hinton, Geoffrey. (2009). Factored conditional restricted Boltzmann Machines for modeling motion style. Proceedings of the 26th International Conference On Machine Learning, ICML 2009. 129. 10.1145/1553374.1553505.