Shape analysis for time series

Thank you for your time and valuable comments. Here are the responses to the weaknesses and questions you raised:

• Typos. You have pointed out our typos accurately. We acknowledge these errors and will make the necessary corrections.

• Comparisons to other methods for the interpretability. In the attached PDF of the common author rebuttal, you will find new figures comparing TS-LDDMM with Shape-FPCA [1] on the mouse dataset, as you requested, to assess the interpretability of these methods.

  The take-home message is the following: while Shape-FPCA managed to represent the main phenomena in the data, it missed the subtle behavior of the respiratory cycle after drug injection, which is captured by TS-LDDMM.

• impact of the model. The goal is to encode time series by using the parametrization $\mathbf{\alpha}$ of the diffeomorphisms. In the LDDMM framework, the parametrization of the diffeomorphisms is highly dependent on the chosen RKHS's kernel. The kernel choice we give with TS-LDDMM offers a better representation $\mathbf{\alpha}$ compared to the classic LDDMM kernel, as depicted in Figures 2 and 3 in the paper. Remark that in both cases, TS-LDDMM and LDDMM, we can recover any time series graph (as you said) because of the great flexibility of LDDMM for learning diffeomorphisms, but the final representation $\mathbf{\alpha}$ is not the same depending on the chosen kernel. Although kernel hyperparameters inject prior information into the reference graph, the main source of dependency comes from the dataset.

• Factor Analysis. Our method distinguishes of Factory analysis on two points. First, TS-LDDMM's goal is to find a vector representation $\alpha \in \mathbb{R}^d$ encoding an irregularly sampled time series. The dimension $d$ is not dependent on the size of the represented time series but depends on the size of the time series graph of reference $\mathsf{G}_0$. It is particularly convenient to apply statistical methods to vectors of fixed dimension, such as PCA. In your suggested paper [2], the latent representation is related to each observation in such a way that if the dimension of the representation in $m$ and the size of the time series at hand is $T$, the dimension of the representation of the time series is $mT$. In the suggested paper, their goal is to perform a factor analysis on the gene observations through time, while in our case, we carried out a PCA/factor analysis on the vectors representing the whole time series (not the latent variable related to each observation). Therefore, the two problems are different: We apply a factor analysis (PCA) to the sequence of observations, and they apply a factor analysis to the observations during a sequence.

We greatly appreciate your interest in interpretability and modeling, which will help others be more receptive to our paper. Thank you once again.

[1] Yuexuan Wu, Chao Huang, Anuj Srivastava, Shape-based functional data analysis, 2023

[2] Cai, Jiachen, et al. "Dynamic Factor Analysis with Dependent Gaussian Processes for High-Dimensional Gene Expression Trajectories." arXiv preprint arXiv:2307.02781 (2023).

Thank you for your time and valuable comments. Here are the responses to the weaknesses and questions you raised:

• Regarding the Functional Data Analysis (FDA) literature. We have compared TS-LDMMM to Shape-FPCA [1] in Figure 1 of the attached PDF to the common rebuttal and on the classification task in Appendix J.1 (Table 4). In both cases, TS-LDDMM compared favorably. Shape-FPCA employs the Square Root Velocity Field (SRVF) representation to separate space and time. However, this method is designed for continuous objects and only applies to time series of the same length. This issue can be addressed through interpolation, but this approach is not always reliable in sparse and irregular sampling scenarios. Most FDA papers [2,3,4] that we are aware of cope with the same issue using interpolation or basis function expansion. In a nutshell, FDA methods have to deal with continuous objects, while LDDMM algorithms can keep a discrete-to-discrete analysis. Thanks to your comments, we plan to extend the discussion on FDA approaches in our background section.

• Time and space transformation analysis. This topic is indeed addressed in Remark 3 of Section 4.1. Due to the TS-LDDMM's kernel parametrization, the time transformation is independent of the space transformation, but the reverse is not true. Our representation encodes both space and time by design which is a competitive advantage compared to method separating the time and space representation. Indeed, post-hoc analysis of separated time and space representation are not straightforward. The separated space and time representations correlate, and understanding this correlation is crucial to interpreting the data. Consequently, you must concatenate the space and time representations, but there is no single way to do this because the two representations are not commensurable. This fact might explain why TS-LDDMM compared favorably to Shape-FPCA, as depicted in Figure 1 of the PDF in attachment to the common rebuttal.

• Difference with the metamorphosis framework introduced by 3DMM [5]. The primary difference lies in the pre-processing requirements. The 3DMM framework requires that each mesh be re-parametrized into a consistent form where the number of vertices, triangulation, and the anatomical meaning of each vertex are consistent across all meshes (as stated in the introduction of [6]). In our context, we do not need such pre-processing; the time series graph can have different sizes. However, applying TS-LDDMM to videos to analyze the inter-variability of facial movement is a promising idea.

• Varifold loss complexity. We acknowledge that the Varifold loss might seem quite complicated for solving a time-series analysis problem. However, in our framework, the loss must take as input two sets of possibly different sizes, which is not a common property for losses. During the design of TS-LDDMM, we also tried using the Maximum Mean Discrepancy (MMD), which is a more straightforward loss, but the performance was lower.

We greatly appreciate your interest in connecting our paper to the broader literature. We believe these connections will help others situate our work within ongoing research. Thank you once again.

[1] Yuexuan Wu, Chao Huang, Anuj Srivastava, Shape-based functional data analysis, 2023

[2] John Warmenhoven , Norma Bargary , Dominik Liebl , Andrew Harrison , Mark A. Robinson , Edward Gunning , Giles Hooker PCA of waveforms and functional PCA: A primer for biomechanics, 2020

[3] Han Lin Shang, A survey of functional principal component analysis, 2013

[4] Yu, Q., Lu, X., Marron, J. S. Principal nested spheres for time-warped functional data analysis, 2017

[5] Volker Blanz and Thomas Vetter, Face Recognition Based on Fitting a 3D Morphable Model, 1999

[6] A 3D Morphable Model learnt from 10,000 face, 2016

Thank you for your detailed comments and your time, which will help us improve the quality and clarity of this paper. We appreciate your accurate understanding of the paper. We are pleased to say that we have been able to carry out the experiment you suggested during the rebuttal period.

Here are our answers to your concerns:

• First, we sincerely apologize for the typos and are thankful for your advice. The article has been reviewed and corrected.

• Regarding the novelty. Representing multivariate and irregularly sampled time series of different lengths for analyzing inter-individual variability is a complex problem.

  Our proposed solution might seem simple for someone knowing LDDMM, but the problem we tackled was never appropriately addressed with LDDMM. For instance, in Figure 1, we show that some deformations in the set of deformations considered by a classical LDDMM do not preserve the time series structure. Additionally, Figure 3 shows that the deformations learned on time series with LDDMM do not carry physiological meaning in the Mice experiment.

  The given representation of the kernel has been carefully designed to integrate space and time while keeping time independent of space. Initially, we considered separating space and time, as suggested in Weakness comment 3. However, post-hoc analysis of this representation is not straightforward. The separated space and time representations correlate, and understanding this correlation is crucial to interpreting the data. Consequently, you must concatenate the space and time representations, but there is no single way to do this because the two representations are not commensurable. Therefore, we decided to have a representation that includes both space and time by design.

  Moreover, we compare favorably to the state-of-the-art in both deep learning and Functional Data Analysis (FDA) literature addressing the same problem. In summary, we have capitalized on a gap in the LDDMM literature and shared our results with other communities working on similar topics.

• The experiment you suggested. In the author rebuttal, you will find the experiment you suggested in Weakness comment 3. In brief, TS-LDDMM compares favorably to Shape-FPCA in this example, even though Shape-FPCA already provides impressive results. Moreover, Shape-FPCA requires good interpolation of time series, which is not feasible in scenarios with few or missing samples. Additionally, Shape-FPCA has a very low classification score compared to TS-LDDMM (0.38 for Shape-FPCA against 0.83 for TS-LDDMM). This low performance stems not from the package (which we verified with the owner) but from the method itself. This observation opens an avenue for future improvements to the Shape-FPCA method.

• Mice breathing behavior experiment. To clarify the computation of the PCA, we performed a Kernel-PCA in the RKHS encoding the velocity fields. Each respiratory cycle is represented by its initial velocity field, which is the velocity field that encodes the geodesic from the reference graph to the respiratory cycle at time t=0.

  When describing Figure 3, we stated: "Compared to wt mice, the distribution of colq mice feature along the PC1 axis has a heavy left tail, and the associated deformation (-2$\sigma$) shows an inspiration with two peaks". Indeed, inspirations in two peaks are representative of colq mice as they suffer from a motor control impairment [1]. Figure 4.a shows an extreme real example of colq respiratory cycle; it is the furthest on the left of PC1. The principal component preserves physiological meaning, and it can be used to differentiate colq mice from wt mice.

  Figure 4.b shows the coordinates of individual learned reference graphs in the PCA coordinates associated with the overall learned reference graph. Figure 4.c shows an example of an individual learned reference graph. As you mentioned, some colq mice are close to wt mice. Indeed, colq mice suffer from a genetic mutation that affects them throughout their growth. The impact on motor control is variable, and some colq mice may appear closer to WT mice.

  As you mentioned: "In Figure 5, the change in PC1 after exposure to the irritant is interesting, and the exposure does appear to affect wt mice more than colq mice". This remark is highly relevant. The irritant molecule prohibits the action of a neurotransmitter involved in motor control. Due to their genetic mutation, colq mice have a deficiency in this neurotransmitter, which leads to motor control impairments. When exposed to the irritant molecule, colq mice are already partially accustomed to the neurotransmitter deficiency, whereas WT mice, the control group, highly suffer from the prohibition.

  In light of your remarks, we will improve the soundness of this part.

• Noise sensitivity. TS-LDDMM focuses on fine-shape registration to encompass small sources of variability. It, therefore, has some sensitivity to noise with pros and cons. As depicted in Appendix G, TS-LDDMM performs good registrations for numerous kernel settings, even for noisy data, meaning that we register the exact time-series shape. On the downside, like other shape registration methods, the learning of the reference graph is affected by noise, as illustrated in the second experiment in the attached pdf. However, the regularity of the reference graph can be controlled by penalizing the velocity fields' norm in the loss. Further work on penalization will be conducted to leverage noisy data.

We thank you for your investment in reviewing our paper and the suggested experiments that we conducted, allowing us to demonstrate our method's significance further.

[1] Aurélie Nervo, André-Guilhem Calas, Florian Nachon, and Eric Krejci. Respiratory failure triggered by cholinesterase inhibitors may involve activation of a reflex sensory pathway by acetylcholine spillover. Toxicology, 424:152232, 2019.

Thank you for your time and valuable comments. Here are the responses to the weaknesses and questions you raised:

• As depicted in Table 1 of the PDF and thanks to its minimal architecture, the training time of TS-LDDMM is below that of large neural networks and beyond that of classic statistical tools. Note that the TS-LDDMM computation time and memory usage can be further optimized with the dedicated package named KeOps.

• We deeply apologize for the codes; we thought the deadline was one day after the paper. We send the link to an anonymized Github to the Area Chair in the hope that it will be made available to you soon.

• Though the method is technical, its outcomes are easy for medical professionals to interpret. One possibility consists of visually representing a PCA's principal components as shape deformation (see Figures 3 & 5), enabling the physicians to interpret the principal sources of variability within a population. For example, in the mice experiment and from the deformations shown in Figure 5 we can see that the primary source of variability corresponds to the pause duration between inspiration and expiration. We are also working on creating an online demo with a user-friendly interface to enhance the impact of this paper on medical professionals.

We greatly appreciate your interest in the application and usability of the method. We hope to make this work available soon. Thank you again.

We appreciate the attention you've given to our paper. We are pleased to note that Reviewer HVwJ has raised their rating from 3 to 5, acknowledging that our rebuttal addressed their concerns regarding novelty and real data experiments. Their remaining concern is the clarity of the presentation. We are committed to making all necessary efforts to improve the clarity of this work. Thank you once again for your time and valuable feedback, which help us remain humble and focused on delivering a high-quality paper.