G-AlignNet: Geometry-Driven Quality Alignment for Robust Dynamical Systems Modeling

$**W2: Correctness of Proposition 4**$.

$**Continued Response to Point 2**$:

We appreciate your insightful question regarding the discretization error. We agree that the discretization error, particularly when employing methods such as the optimize-then-discretize approach (e.g., the adjoint method), is critical to understanding the overall approximation error in Neural ODEs. Below, we clarify the training method used in this work and how we accounted for the discretization error in the proof of Proposition 4.

$**Training Method and Discretization Error.**$

In our previous proof, the first and second terms in Equation (48) in the Appendix C.5 were combined into a single $\mathcal{O}$-notation term, which might give the impression that the discretization error was omitted. However, this is not the case—we incorporated the discretization error into the derivation through $\delta(t)$.

In our paper, we employed the Runge-Kutta 4 (RK4) solver for both Neural ODE training and inference. RK4 is a fixed-step explicit solver that balances computational efficiency and accuracy. As a fourth-order method, the total truncation error for RK4 is of order $\mathcal{O}(h^4)$, where $h$ is the step size.

$**How We Account for Discretization Error in Proposition 4.**$ In the proof of Proposition 4, we explicitly account for the truncation error (discretization error) using the term $\delta(t):\mathbb{R}\to\mathbb{R}$ in Equation (48). This term represents a continuous function that bounds the difference between the true ODE dynamics $\dot{\Theta}_x(t)$ and the approximated dynamics $\Omega(t)\bar{\Theta}_x(t)$, as follows: $\\left\\|\\dot{\\Theta}\_x(t) - \\Omega(t)\\bar{\\Theta}\_x(t)\\right\\|\_F \le \delta(t)$.

Given that the step size $h$ typically lies within the range $h \in [10^{-4}, 10^{-2}]$ in practice, and considering the Runge-Kutta 4 (RK4) solver has a truncation error of order $\mathcal{O}(h^4)$, $\delta(t)$ can be treated as a small and approximately constant upper bound. This approach is consistent with the derivation in Theorem III.3 of [2], which imposes a similar condition, $|\mathcal{R}_{\hat{\phi}}(t)| \leq \delta(t)$.

$**Modifications in the revised paper to significantly improve the clarity**$. In the revised manuscript, we have implemented the following improvements to address this issue: (1) We explicitly describe the use of the RK4 solver for the Neural ODE model and emphasize the associated discretization error. (2) We have decomposed the error into two terms in Equation (49) of the revised manuscript, where the second term explicitly accounts for the discretization error. (3) Consequently, we revised Proposition 4 and its proof in the appendix to formally include the discretization error in our results. These revisions highlight that the discretization error is explicitly and rigorously addressed.

$**W3: Incomplete experimental results**$.

$**Response to point 1**$:

In the manuscript, we use RNN as the base model because RNN is suitable for different sequential data processing, but INR focuses more on the continuous domain [40]. We appreciate the author's suggestion to add the version of G-AlignNet with INR. We use G-AlignNetR and G-AlignNetI to denote the case with RNN and INR as base models, respectively. Hence, we give additional results for G-AlignNetI. Please see the results in the modified paper. In general, G-AlignNetI works well in continuous systems like power events, air quality, and spiral datasets and achieves state-of-the-art performance with around $1\%\sim 10\%$ error reduction compared to G-AlignNetR methods. However, for systems with more uncertainty, e.g., the load and PV systems, G-AlignNetI's performance is not competitive. The main reason is that the INR model is less powerful than RNN in capturing historical trends and patterns for predictions. In the modified paper, we present all results for G-AlignNetR and G-AlignNetI in Sections 4.3 and 4.4.

$**Response to point 2**$:

Thank you for your careful reading and feedback. We apologize for the confusion caused by a typographical error in the legend of Fig. 4. The green dotted line represents the results of Neural CDE, not Neural ODE. Neural CDE is indeed highly relevant for extrapolation tasks, as you noted, and it was included in the comparison. Neural ODE, on the other hand, is less suitable for sequential data processing, which aligns with your observation. We will correct the legend to ensure clarity and consistency in the final version.

$**W4: Problem formulation and experimental setting**$.

$**Response**$:

Thank you for your insightful comment. We have provided a detailed response in the Questions section and have significantly improved the paper's clarity. We sincerely appreciate your efforts in highlighting these areas for improvement. Please refer to the revised version of the paper, and let us know if you have any further questions or suggestions.

$**W5: Green points in Fig. 3**$.

$**Response**$:

Thank you for pointing this out. In Fig. 3, the green points in the left part of the Fig. represent the parameters $\Theta_y(t_i)$ corresponding to the low-resolution data points. The green points in the right panel of Fig. 3 represent the low-resolution measurements that are used in the training procedure. High-resolution measurements are not visualized as they are too dense to display effectively.

The results demonstrate that G-AlignNet excels in aligning high-resolution and low-resolution measurements, extracting shared knowledge, and constructing a more accurate dynamic learning model for low-resolution data (as shown in the right panel of Fig. 3). This capability arises from G-AlignNet's ability to achieve precise parameter flow alignment (left panel of Fig. 3), which maximizes the extraction of common knowledge. The success of this alignment is attributed to our geometric representation learning approach, which leverages a well-structured parameter manifold, i.e., the orthogonal group.

We hope this clarifies your concern and appreciate your attention to detail. We make all modifications accordingly in the revised paper.

$**Q1: Problem formulation**$.

$**Response**$:

Thank you for your thoughtful comments. We acknowledge that the problem formulation in the paper lacked clarity, and we have revised the paper significantly to address these concerns. Below are point-by-point responses to your questions for clarity:

$**1. Your first point is correct**$. We define the state as $\boldsymbol{s}=[\boldsymbol{x},\boldsymbol{y}]$, where measurements of the state $\boldsymbol{s}\in\mathbb{R}^{d_x+d_y}$ consist of High-Quality (HQ) measurements of $\boldsymbol{x}\in\mathbb{R}^{d_x}$ and Low-Quality (LQ) measurements $\boldsymbol{y}\in\mathbb{R}^{d_y}$. This setup is common in many real-world systems, such as power [6] and transportation [25], where economic considerations lead to HQ and LQ sensors being deployed in different parts of the system.

$**2. Regarding the ambiguity in lines 151-152**$, we confirm that the correct definition is $\boldsymbol{s}=[\boldsymbol{x},\boldsymbol{y}]$. The text has been revised to consistently adhere to this definition throughout the paper. For example, we modify lines 151-152 as: "Here, $\hat{\boldsymbol{s}}(t_i)$ represents either the true measurements, which are a combination of HQ and LQ data $[\boldsymbol{x}(t_i),\boldsymbol{y}(t_i)]$ ($\forall i\in\mathcal{N}_y$), or a combination of HQ measurements and interpolated LQ data $[\boldsymbol{x}(t_i),\tilde{\boldsymbol{y}}(t_i)]$ ($\forall i\in\mathcal{N}_x\setminus\mathcal{N}_y$)."

$**3. Yes, we assume that some variables are sampled with HQ and others with LQ. **$ This reflects the practical scenario where HQ measurements are only available for a subset of variables due to cost constraints, while LQ measurements are used for the remaining variables. The fixed role of HQ and LQ variables is primarily for the ease of mathematical modeling. In real-world systems, some HQ sensors may experience sensor or communication failures and temporarily function as LQ sensors. However, this dynamic does not affect our HQ-LQ alignment procedure, which is designed to make the most use of all available data regardless of the sensor status.

$**4. We apologize for the confusion**$. $\boldsymbol{y}$ is not a downsampled version of $\boldsymbol{x}$.

$**W2: Correctness of Proposition 4**$.

$**Response to point 1**$:

We appreciate your careful reading for both our proposition 4 and the reference [2] in the manuscript (i.e., [33] in the response reference list). Your misunderstanding may come from the title and final results of [2]. However, we only employ the intermediate results of [2], which are general for all Machine Learning candidate functions to approximate the underlying ODE dynamics, namely, $\hat{\phi}(\cdot)$ in Section II in [2]. To give a clear understanding, we fully explain why we utilize the intermediate theorems in [2] to prove our Proposition 4 and how the utilized theorem applies to the Neural ODE.

$**Why do we employ theorems in [2] to prove the proposition 4**$? The proof for Proposition 4 leverages Theorem III.3 from [2], which provides an error analysis of learning parameter flows using machine-learning models as candidate functions to approximate the ODE dynamics. Importantly, we do not rely on the final theorems of [2] with additional PDE loss in PINN but instead use its intermediate results.

Specifically, Theorem III.3 in [2] analyzes the prediction error $e(t) = \\|\hat{\Theta}\_{x}(t) - \Theta\_{x}(t)\\|$, where $\hat{\Theta}_x(t)$ is the machine-learned estimate of the true trajectory $\Theta_x(t)$. This theorem is broadly applicable to any machine learning method, including Neural ODEs, because it considers the general case of approximating continuous parameter flows governed by ODEs. The general error analysis comes from the classic theory of the so-called perturbed Initial Value Problem [33,34]. In this error analysis, the approximation error of the Neural ODE works as the perturbation to the IVP problem and the impact is upper bounded with moderate smoothness assumptions.

In our manuscript, we model parameters $\Theta_x(t)$ with an ODE $\dot{\Theta}_x(t) = \Theta_x(t) \Omega_x(t)$, which aligns with the settings described in Equations (1) and (8) of [2]. Thus, the error bounds provided in Theorem III.3 are directly relevant to our analysis, irrespective of the specific machine learning approach used to approximate the dynamics.

$**How the theorem shows the result of Neural ODE**$? The error bounds provided in Theorem III.3 from [2] are derived by formulating the prediction error $e(t)$ as the solution of the perturbed IVP for a perturbed ODE. Specifically, as shown in Equation (10) of [2], the evolution of the prediction error can be expressed as a perturbed IVP. By solving this IVP and applying the triangle inequality, Theorem III.3 provides an upper bound for the cumulative error.

In our proof of Proposition 4, we adapt Theorem III.3 to the Neural ODE framework by directly treating the neural network in Neural ODE as the ML candidate function $\hat{\psi}(\cdot)$ in [2]. All the results still hold.

$**W1: Claim of limitations for existing work**$.

$**Continued response**$:

$**Theoretical support**$. To substantiate our claims about the limitations of existing methods, we provide rigorous theoretical support. First, the classical theory of solving Initial Value Problems (IVPs) for ordinary differential equations (ODEs) establishes that cumulative error increases over time due to the accumulation of truncation and round-off errors, with propagation and amplification influenced by the system's dynamics and the numerical method employed [32]. High-resolution measurements mitigate these errors by enabling the IVP to be solved over smaller intervals, starting from each sample point.

Second, in contrast to solving IVPs with known ODEs, Neural ODE methods introduce additional approximation dynamics during training, which can be analyzed through the framework of perturbed IVPs [33,34]. This framework shows that cumulative error persists under such settings. Our Proposition 4 also makes use of this result, which we will elaborate on in response to the next question. In summary, the derived error bounds emphasize the necessity of high-resolution data with small sampling intervals for effective error control.

$**Data imputation to pre-process low-quality data**$. To address this information gap, data imputation techniques are employed to enhance data quality before using Neural ODE-based methods. These techniques leverage prior knowledge, explicit assumptions about the system's behavior, or relevant high-quality data streams to reconstruct the missing information and enhance the learning process. Model-based methods, such as multidimensional interpolation [35] and physical model-based estimations [36], rely on explicit assumptions about system behavior. Optimization-based techniques, including Compressed Sensing [37], matrix completion, and Bayesian methods [38], frame imputation as minimizing a loss function by assuming low-rank or sparsity structures. Signal processing and machine learning models offer data-driven solutions that can adapt to complex patterns [6,39], yet these often overlook domain-specific structures. Despite their utility, many existing approaches are inconsistent with the underlying data structure, as they rely on simplifying assumptions that fail to capture the intrinsic dynamics of complex systems. For example, results in Table 2, Fig. 4 and Fig. 5 in the paper show that even with the best data imputation method (i.e., Cubic Spline in our tests) as pre-processing, Neural ODE-based methods perform much worse than our G-AlignNet.

Our G-AlignNet is a unified model to boost the quality and learn a good dynamic model. G-AlignNet brings significant benefits to real-world systems by using many low-quality (LQ) sensors and limited high-quality (HQ) sensors to enhance dynamic data availability for all LQ sensors and the overall dynamic model estimation.

$**Why can the proposed G-AlignNet work**$? G-AlignNet effectively leverages data geometry by constructing a geometric representation that bridges high-quality (HQ) and low-quality (LQ) data. Additionally, the parameter space is carefully structured to maintain orthogonality through the use of Lie algebra, ensuring a robust foundation for geometric optimization. This geometric optimization aligns HQ and LQ data while preserving orthogonality, which enables globally optimal solutions with rigorous theoretical guarantees.

Theoretical analysis shows that the above process in G-AlignNet has, to the best of the author's knowledge, the fastest convergence rate with respect to the number of LQ samples $|\mathcal{N}_y|$. Specifically, according to past analytical frameworks [15-18], errors are caused by measurement noise and the on-manifold flow approximation errors, presented as the first and the second term in the right-hand-side of Eq. (10) in the paper. Compared to the previous methods, we have the same error bound $\mathcal{O}(\frac{1}{\sqrt{|\mathcal{N}_y|}})$ for the error caused by noise. However, our approximation error is bounded by $\mathcal{O}(\frac{1}{|\mathcal{N}_x|})$, much smaller than the error bound in cutting-edge manifold-based compress sensing, i.e., $\mathcal{O}(\frac{1}{\log{|\mathcal{N}_y|}})$ [17]. The latter result is based on local linearization for the data manifold. Instead, G-AlignNet intelligently combines the high approximation power of ODE flows in Neural ODE and a geometric optimization with global optimality on the well-structured parameter manifold.

$**W1: Claim of limitations for existing work**$.

$**Response**$:

Thank you for your question regarding the limitations of current methods and the motivation behind our approach. It is important to clarify that our work does not address all aspects of Neural ODE learning under data quality issues. Instead, we focus on a well-defined scope with specific motivations related to data quality challenges, highlighting the limitations of existing methods. To provide clarity, we emphasize the following: the study's scope, the motivation for addressing data quality issues, the limitations of existing approaches—supported by theoretical and numerical results as well as references—and the significance of our proposed method in overcoming these challenges. We add the above contents to the modified paper Introduction, Related Work, and Appendix.

$**Study scope**$. Our paper aims to learn a dynamic model for engineering and control systems considering data quality issues, which are persistent and common in these systems. The learned dynamical model is critical for decision-making, Model Predictive Control (MPC), and model-based Reinforcement Learning (MBRL) in complex systems [20-22].

$**Motivations of data quality issues**$. For many realistic systems, such as power grids, healthcare, and transportation networks [23-25], data quality fundamentally determines the extent to which dynamic information can be stored and captured in measurements for learning accurate dynamic models. For example, we focus on data incompleteness, including $(1)$ Low-Resolution (LR) measurements due to LR sensors [6] or downsampling to meet communication constraints [7]. $(2)$ A period of data losses due to communication/sensor failure, external events, etc. [8]. $(3)$ Random data losses (i.e., irregular sampling [9,10]) due to sensor configurations, data corruptions, human errors, etc. [11]. These three types are all tested in our Experiment in the first paragraph of Section 4.3. We present a clear definition in the Introduction and Section 3.1 in the revised paper. Moreover, we give visualizations for categories $(1)\sim(3)$ in Appendix A, Data Quality Definition and Visualization. Notably, data quality issues also include data inaccuracy and inconsistency with respect to the true values. However, in most physical systems, inaccurate and inconsistent measurements will be removed using mature technologies such as bad data detection [12], anomaly detection [13], noise filtering [14], etc. Consequently, these problems are converted to a data incompleteness problem. In general, data incompleteness is the central, common, and long-standing data quality issue for growing physical systems.

$**Limitations of Neural ODE family**$. To learn accurate dynamic models, the family of Neural ODEs is heavily used [26-29] because of their capacity to model the continuous process using ODE solvers. This enables the process and evaluation of measurements sampled at arbitrary times. Thus, some less severe data quality issues, such as a small portion of data losses (i.e., irregularly sampled data [9,10,30]), can be properly tackled. However, this doesn't necessarily mean that all data incompleteness issues in the above categories $(1)\sim(3)$ can be fully addressed.

$**Negative numerical results when learning with significant data losses**$. Significant data losses, such as low-resolution data, inherently lead to insufficient dynamic information, posing a fundamental challenge to uncovering the hidden dynamics for time intervals without samples. Under this condition, directly applying Neural ODE methods can hardly quantify the dynamic transitions within these intervals. This can lead to significant negative results for real-world systems. For example, as shown in Section 4.6, if we only know hourly load data, the voltage may not get stable between every two hourly samples, and the overly high voltage may cause overheating or damage to sensitive equipment, such as electronic devices [31]. Moreover, as shown in Table 2 in Section 4.3, for many test systems, using Neural ODE variants leads to large prediction errors ($>10\\%$ MAPE).

$**Q2: More detailed explanation for experimental setting**$.

$**Response to point 1**$:

Thank you for your valuable feedback. Below, we provide a more detailed explanation of the experimental setup. We add this part to Appendix D.4 in the revised paper. Our experiments were conducted on multiple systems, including the Load dataset, PV dataset, Power event dataset, Air quality dataset, and spiral dataset. The input dimension for each system is $10$, $10$, $6$, $8$, and $2$. Moreover, we split HQ/LQ dimensions to be $2/8$, $2/8$, $1/5$, $2/6$, and $1/1$, respectively.

$**Response to point 2**$:

We don't test the noise issues in our experiments because the impact of noise is quite limited in engineering systems. As noted in answer to your first question, our target data quality issue is data incompleteness, including $(1)$ Low-Resolution (LR) measurements due to LR sensors [6] or downsampling to meet communication constraints [7]. $(2)$ A period of data losses due to communication/sensor failure, external events, etc. [8]. $(3)$ Random data losses (i.e., irregular sampling [9,10]) due to sensor configurations, data corruptions, human errors, etc. [11]. These three types are all tested in our Experiment in the first paragraph of Section 4.3. We present a clear definition the in Introduction and Section 3.1 in the revised paper. Moreover, we give visualizations for categories $(1)\sim(3)$ in Appendix A, Data Quality Definition and Visualization.

The remaining data quality issues can be data inaccuracy and inconsistency issues, which can be caused by noises. In most physical systems, inaccurate and inconsistent measurements will be removed using mature technologies such as bad data detection [12], anomaly detection [13], noise filtering [14], etc. Consequently, these problems are converted to a data incompleteness problem. In general, data incompleteness is the central, common, and long-standing data quality issue for growing physical systems.

$**Q3: Application to Transformer and Mamba**$:

$**Response**$:

Thank you for your insightful suggestion. Extending our on-manifold parameter flow and manifold-based geometric optimization to Transformers [41] and Mamba [42] is indeed an exciting direction, as both are powerful and widely-used models for time-series tasks. However, there are considerations regarding feasibility and computational costs. Below, we elaborate on these points:

$**Feasibility of our hypernetwork structure**$. Our G-AlignNet employs a hypernetwork structure, where a Neural ODE governs the parameter flows for RNNs and INRs. This approach could, in principle, be extended to Transformers and Mamba. Notably, there are studies that explore hypernetwork-controlled Transformers, where hypernetworks facilitate task-specific adaptation in feed-forward layers [43] or value networks [44] within the attention mechanism. As for Mamba, while there is limited discussion of hypernetworks in this context, Mamba shares similarities with RNNs in its hidden state transitions. Thus, it is intuitive to extend our Neural ODE to produce the parameter flow $A(t)$ for the $A$ matrix in Mamba [42] on an orthogonal group. This can effectively capture the dynamic information because $A(t)$ governs the evolution of the hidden state dynamics.

$**Computational complexity**$. While our orthogonal matrix flow framework can theoretically be integrated with any model, computational cost remains a key concern. Transformers, for instance, are computationally intensive due to the attention mechanism, and even efforts to extend attention to continuous time [10] have not fully mitigated this. Conversely, RNNs and INRs are more computationally efficient, requiring significantly less training time. Mamba, in particular, is highly efficient because of its state-space model foundation. This efficiency makes Mamba a promising candidate for integrating our geometric representation in future studies.

Thank you for taking the time and effort to respond to my reviews. Overall, most of my questions have been addressed satisfactorily. However, I believe there is one point where the authors may have misunderstood my concern.

When I referred to the additional discretization error of Neural ODEs, I was specifically discussing the error introduced in the optimize-and-discretize approach, such as the adjoint method. As noted in [1], adjoint methods can introduce gradient errors, which could impact the overall analysis.

Therefore, I think it is important for the authors to clarify the following:

Does GAlign-Net use adjoint methods for gradient computation?

• If yes, this additional error should be included in the analysis.
• If no, further clarification about the gradient computation approach is needed.

Once this point is addressed, I am happy to re-evaluate my score.

[1] Zhuang, Juntang, et al. "Adaptive checkpoint adjoint method for gradient estimation in Neural ODE." International Conference on Machine Learning. PMLR, 2020.

$**Q1: Prediction error quantification**$.

$**Response**$:

While we present extensive experimental results supporting the superiority of G-AlignNet, we acknowledge the difficulty of deriving strict error bounds for LQ predictions. This challenge arises from the nonlinearity of the base RNN or INR model, as well as the need to evaluate whether $\Theta_1$ is adequately trained. We identify this quantification as an important direction for future work and avoid making overclaims in this regard.

$**Q2: Validation of orthogonality**$.

$**Response**$:

Answered in W2.

$**Q3: Definition of$\Theta_0$and$\Theta_1$**$.

$**Response**$:

Answered in W4.

$**W1: Assumption 1 and justification**$.

$**Response**$:

Thank you for your thoughtful comments. We address your concerns in detail below.

$**Empirical evidence of data similarity**$. We agree with you that the observed responses of HQ and LQ variables often exhibit similar dynamical behaviors due to strong spatiotemporal correlations and physical constraints. These similarities are illustrated in Appendix B through the visualization of data similarity.

$**The hypothesis of parameter flow alignment**$. Based on our observations, we hypothesize that a well-designed geometric representation of the data should share the same structure in a simplified latent space, such as the parameter space governing data dynamics. In Assumption 1, the similar shapes of $\Theta_x(t)$ and $\Theta_y(t)$ represent shared knowledge extracted from HQ and LQ data. Our G-AlignNet architecture facilitates this structural alignment through restricted geometric representations and optimization, ensuring effective learning.

$**Model expressiveness despite flow restrictions**$. Even with restrictions on the shape of the parameter flow, our model remains highly expressive and capable of capturing differences between HQ and LQ data. This is due to the following reasons: (1) the flow of $\Theta_x(t)$ and $\Theta_y(t)$ can reside in different regions of the manifold. (2) Eq. (1) allows for distinct static components ($\Theta_0$ and $\Theta_1$) within $\Theta$, such as bias vectors, which are designated to the dynamic learning functions of HQ and LQ data, respectively. In general, our model is highly expressive in representing different HQ and LQ data and capturing the main similarity.

$**Experimental validation of parameter flow alignment**$. In Section 4.2, we demonstrate the effectiveness of our model by comparing G-AlignNet with a flow-based learning model lacking shape alignment. As shown in the right part of Fig. 3, the LQ learned dynamics (green curves) from G-AlignNet better fit the true data. This is attributed to G-AlignNet's ability to perfectly align the shapes of the parameter flows, as illustrated in the left part of Fig. 3 (note: we centralize the flows for better visualization of shape differences).

$**Challenges in theoretical quantification**$. While we present extensive experimental results supporting the superiority of G-AlignNet, we acknowledge the difficulty of deriving strict error bounds for LQ predictions. This challenge arises from the nonlinearity of the base RNN or INR model, as well as the need to evaluate whether $\Theta_1$ is adequately trained. We identify this quantification as an important direction for future work and avoid making overclaims in this regard.

$**W2: Validation of orthogonality**$.

$**Response**$:

Thanks a lot for your insightful suggestion. To verify the orthogonality between $\Theta_x(t)$ and $\Theta_y(t)$, we use the definition of matrix orthogonality and write a sub-program, shown in Section D.2 in the Appendix. Specifically, the program evaluates if a matrix $Q$ is orthogonal by computing the error $||Q^{\top}Q-I||_F$ and checking if the error is smaller than a tolerance value. Hence, we utilize the program to check each iteration of the training procedure for $Q$ in Eq. (2) in the manuscript. The results show that in each iteration, the error is around $10^{-8}\sim 10^{-7}$. Consequently, numerically we show that the orthogonal relationship is maintained.

$**W3: Result bold**$.

$**Response**$:

We appreciate your attention to detail. We rechecked the result and find that the correct value for MAPE for RNN is $10.58 \pm 1.05$, worse than our G-AlignNet. We have carefully refined the table and verified all entries to ensure that the bolding correctly highlights the best-performing model in each scenario.

$**W4: Definition of$\Theta_1$and$\Theta_2$**$.

$**Response**$:

Thank you for pointing this out. $\Theta_0$ and $\Theta_1$ are distinct static components within the overall parameter set $\Theta$. These components, such as bias vectors, are specifically associated with the dynamic learning functions of HQ and LQ data, respectively. For example, for HQ and LQ RNNs, we have different bias terms in Eq. (5) in the manuscript. To clarify this further, we have now added a detailed explanation below Eq. (1) in the paper. We also proofread the complete paper to make sure all notations are properly defined. We appreciate your feedback and encourage you to refer to the updated manuscript for improved definitions.

$**Q6: Limitations of Assumption 1**$.

$**Response**$:

$**Impact of Noise on Assumption 1.**$ We admit that the data property described in Assumption 1 can be affected by noise. When there are significant random factors such as sensor noise, Assumption 1 may not hold since the data similarity is reduced. In Section 3.3, we quantify the error caused by a type of noise, which demonstrates the certain robustness of our G-AlignNet. However, for more complicated noise, we need more investigations. In addition, noise can be reduced by employing more precise sensors or noise filtering techniques in engineering systems [14]. We add the limitation to Appendix B of the revised paper.

$**Assumption 1 holds when noise is limited.**$ When the noise is limited, Assumption 1 holds for a nonlinear system because it only states the data correlations and similarity in response to disturbances between HQ and LQ data. Then, the high data correlations between HQ and LQ data can lead to parameter flow with the same shape but different locations on a manifold, where the shape captures similar patterns between HQ and LQ data. As shown in Appendix B, Visualization of Data Similarity, highly nonlinear and uncertain engineering systems still have strong data correlations and similarities. We add the above clarification to the description of Assumption 1 in the revised paper.

$**Q7: Justification for Assumption 1**$.

$**Response**$:

$**Assumption 1 holds in many nonlinear and uncertain engineering systems.**$ Our target is engineering and control systems with nonlinearity and uncertainty. For these systems, Assumption 1 states that the high data correlations between HQ and LQ data can lead to parameter flow with the same shape but different locations on a manifold, where the shape captures similar patterns between HQ and LQ data.

We have the following justifications for the validity of Assumption 1: (1) The data similarity stems from the spatial-temporal correlations and physical correlations for the system, which significantly exist in engineering systems. For examples, the visualization in Appendix B shows that highly nonlinear and uncertain engineering systems still have strong data correlations and similarity. (2) Under a probabilistic setting, HQ and LQ variables within a local region in the system can have high similarity due to spatial-temporal and physical correlations in both mean and variance. Then, $\Theta_x(t)$ and $\Theta_y(t)$ in our learning framework, as long as being well-trained to extract patterns of this similarity, can maintain the same shape. (3) With external forces the HQ/LQ measurements in systems usually still contain high spatial-temporal and physical correlations. For instance, when an event happens to power systems, system states (i.e., nodal voltage) have similar behaviors because of network constraints [19]. The left Figure in Appendix B illustrates the voltage fluctuations after an event. The PV systems or residential loads, affected by weather such as wind movements and temperature, have similar data patterns within a local region, shown in the middle and the right Figure in Appendix B.

$**Scenarios that Assumption 1 might break down.**$ When there are significant random factors such as sensor noise, Assumption 1 may not hold since the data similarity is reduced. In Section 3.3, we quantify the error caused by a type of noise, which demonstrates the certain robustness of our G-AlignNet. However, for more complicated noise, we need more investigations. In addition, noise can be reduced by employing more precise sensors or noise-filtering techniques in engineering systems [14]. We add the justification to Appendix B of the revised paper.

$**Q8: Assumption for external noise**$.

$**Response**$:

Assumption 1 relies on data similarity. As explained in previous answers, for nonlinear systems with significant random sensor noise, data similarity in Assumption 1 may not hold. When engineering systems have external forcing terms, the HQ/LQ measurements in systems usually still contain high spatial-temporal and physical correlations. For instance, when an event happens to power systems, system states (i.e., nodal voltage) have similar behaviors because of network constraints [19]. The left Figure in Appendix B illustrates the voltage fluctuations after an event. The PV systems or residential loads, affected by weather such as wind movements and temperature, have similar data patterns within a local region, shown in the middle and the right Figure in Appendix B. This is because, in a local region, the external environments are almost the same. In general, our result is very beneficial since, with our methods, we only need to guarantee each local region can contain a small number of HQ sensors and can boost the quality of all LQ sensors in the region. We add the justification to Appendix B of the revised paper.

$**Q2: Clarification to resolution, sampling rate, etc.**$.

$**Response**$:

As explained in previous questions, we tackle $**data incompleteness**$, including LR data, a period of data losses due to communication failure, and random data losses. Data noise belongs to the $**data inaccuracy**$ issue, and there is another $**data inconsistency**$ issue. In most physical systems, inaccurate and inconsistent measurements will be removed using mature technologies such as bad data detection [12], anomaly detection [13], noise filtering [14], etc. Consequently, these problems are converted to a data incompleteness problem. In general, data incompleteness is the central, common, and long-standing data quality issue for growing physical systems. We clarify these statements in the Introduction of the revised paper.

$**Q3: Consistent terminology **$.

$**Response**$:

In the revised paper, we utilize High/Low-Quality (HQ/LQ) data to denote overall quality differences and emphasize that we focus on data completeness quality. In Experiment, we utilize High/Low-Resolution (HR/LR) data to denote the quality difference associated with the different sampling resolutions (category (1) in previous answers). We utilize missing intervals to denote the low quality associated with the absence of the interval data (category (2) in previous answers). We utilize irregularly sampled data (or data with random drops) to denote the low quality associated with random data losses. We note that these two terms are frequently and almost equivalently used in [9,10]. We clarify these terminologies throughout the revised paper.

$**Q4: Relation to experimental setup**$.

$**Response**$:

In our Experiment, the number of dropped data points is equal to $|\mathcal{N}_x|-|\mathcal{N}_y|$. Hence, given fixed HQ data, if the data drop rate is higher, $|\mathcal{N}_x|-|\mathcal{N}_y|$ is higher, the LQ data amount $|\mathcal{N}_y|$ is lower, the data quality is worse, and the obtained MSE/MAPE error should be generally larger. In Section 3.3, we quantify that our method's error bound is around $\mathcal{O}(\frac{1}{\sqrt{|\mathcal{N}_y|}} + \frac{1}{|\mathcal{N}_x|})$, which is the lowest among other data interpolation methods. In Fig. 4, we plot the error with respect to the LQ data coverage rate ($\frac{|\mathcal{N}_y|}{|\mathcal{N}_x|}$, proportional to $|\mathcal{N}_y|$ given fixed $|\mathcal{N}_x|$ ), which approximately aligns with the theoretical results. We add the above explanations to the Experiment in the revised paper.

$**Q5: Aspect of data quality to Assumption 1**$.

$**Response**$:

As we explained in the previous answer, we focus on data incompleteness, the severe, common, and persistent issue, in engineering and control systems. We agree with you that within our study scope (i.e., data incompleteness), Assumption 1 holds. For systems with high uncertainty, we can still give some justifications for the validity of Assumption 1: (1) $\Theta_x(t)$ and $\Theta_y(t)$ in Assumption 1 can be naturally extended to a probabilistic setting, as pointed out in the first paragraph of Section 3.1. These parameters can represent neural network weights to approximate both the mean and the variance. Hence, G-AlignNet has the capacity to capture probabilistic dynamics. (2) Under a probabilistic setting, HQ and LQ variables within a local region in the system can have high similarity due to spatial-temporal and physical correlations in both mean and variance. Then, $\Theta_x(t)$ and $\Theta_y(t)$ in our learning framework, as long as being well-trained to extract patterns of this similarity, can maintain the same shape. We treat the investigation of the probabilistic setting in future work. Finally, to make readers easily understand Assumption 1, we visualize the realistic datasets to demonstrate the data similarity. The visualization is introduced in Appendix B, Visualization of Data Similarity in the revised paper. We add the above clarification to Appendix B in the revised paper.

$**W1: Data quality definition**$.

$**Continued response**$:

$**Clarifications of your proposed details**$. First, we modify line 130 by specifying our target data incompleteness with a clear definition and three categories. The definition and visualization are also presented in Appendix A to make readers easily understand the target problems. Then, we show that all issues indicate the differences in data amount. Mathematically, $\mathcal{N}_y\subset \mathcal{N}_x$ and $|\mathcal{N}_y|\ll |\mathcal{N}_x|$. Second, in line 273, the measurement noise and approximation error are used as two sources to analyze the final error bound. In addition to data incompleteness, the analytical result suggests that our method is robust to linear measurement noise, similar to Compressed Sensing. Third, in line 364, we use Low-Resolution (LR) and High-Resolution (HR) (i.e., category $(1)$ in the previous statement) as an example to visualize parameter flow, which reveals that our G-AlignNet can achieve perfect shape match. The general results for categories $(1)\sim (3)$ are presented in Section 4.3. Finally, in line 385, we present three categories, which are achieved by dropping data for low resolutions, dropping consecutive intervals, and dropping data randomly. Our results in Tables 1 and 2 show that in most datasets for the three quality issues, G-AlignNet has the best performance.

$**W2: Clarification of Assumption 1**$.

$**Response**$:

Thanks for your comments. We provide the following thorough explanations for the validity, limitations, and justifications of Assumption 1. Related contents are included in the description below Assumption 1 and Appendix B in the modified paper.

$**Assumption 1 validity under data incompleteness and noise impact**$. First, as we explained in the previous answer, we focus on data incompleteness, the severe, common, and persistent issue in engineering and control systems. We agree with you that within this study scope (i.e., data incompleteness), Assumption 1 holds.

$**Limitations of Assumption 1**$. Second, we admit that the data property described in Assumption 1 can be affected by noise. When there are significant random factors such as sensor noise, Assumption 1 may not hold since the data similarity is reduced. In Section 3.3, we quantify the error caused by a type of noise, which demonstrates the certain robustness of our G-AlignNet. However, for more complicated noise, we need more investigations. In addition, noise can be reduced by employing more precise sensors or noise filtering techniques in engineering systems [14].

$**Justifications for Assumption 1 under high data uncertainty**$. Third, we give some justifications for the validity of Assumption 1 under high data uncertainty: (1) $\Theta_x(t)$ and $\Theta_y(t)$ in Assumption 1 can be naturally extended to a probabilistic setting, as pointed out in the first paragraph of Section 3.1. These parameters can represent neural network weights to approximate both the mean and the variance. Hence, G-AlignNet has the capacity to capture probabilistic dynamics. (2) Under a probabilistic setting, HQ and LQ variables within a local region in the system can have high similarity due to spatial-temporal and physical correlations in both mean and variance. Then, $\Theta_x(t)$ and $\Theta_y(t)$ in our learning framework, as long as being well-trained to extract patterns of this similarity, can maintain the same shape. In general, when the noise is limited, Assumption 1 holds for a nonlinear system because it only states the data correlations and similarity in response to disturbances between HQ and LQ data. Then, the high data correlations between HQ and LQ data can lead to parameter flow with the same shape but different locations on a manifold, where the shape captures similar patterns between HQ and LQ data. As shown in Appendix B, Visualization of Data Similarity, highly nonlinear and uncertain engineering systems still have strong data correlations and similarities.

$**Q1: Definition of data quality**$.

$**Response**$:

In general, our G-AlignNet mainly tackles $**data incompleteness**$, in control and engineering systems. Data incompleteness refers to the absence of values in the dataset, including $(1)$ Low-Resolution (LR) measurements, $(2)$ a period of data losses, and $(3)$ random data losses (i.e., irregular sampling [9,10]). Mathematically, $\mathcal{N}_y\subset \mathcal{N}_x$ and $|\mathcal{N}_y|\ll |\mathcal{N}_x|$. In the revised paper, we add the definition to the Introduction and mathematical explanations to Section 3.1. Finally, we give visualizations for categories $(1)\sim(3)$ in Appendix A, Data Quality Definition and Visualization. The definition is added to the Introduction in the revised paper.

$**W1: Data quality definition**$.

$**Response**$:

We really appreciate your concern about the scope of the problem we can tackle. Hence, we follow your suggestion to give a clear definition of our target data-quality issues. Then, we emphasize why our G-AlignNet is capable of tackling these problems and is the state-of-the-art method. Finally, we make clarifications in the revised paper for your detailed comments.

$**Definitions for data quality issues**$. In general, our G-AlignNet aims to tackle the most severe and persistent issue, $**data incompleteness**$, in control and engineering systems. Data incompleteness refers to the absence of values in the dataset. More specifically, the incompleteness can be categorized into $(1)$ Low-Resolution (LR) measurements due to LR sensors [6] or downsampling to meet communication constraints [7]. $(2)$ A period of data losses due to communication/sensor failure, external events, etc. [8]. $(3)$ Random data losses (i.e., irregular sampling [9,10]) due to sensor configurations, data corruptions, human errors, etc. [11]. These three types are $**all tested in our Experiment**$ in the first paragraph of Section 4.3. We present a clear definition in the Introduction and Section 3.1 in the revised paper. Moreover, we $**give visualizations for categories$(1)\sim(3)$**$ in Appendix A, Data Quality Definition and Visualization.

Notably, data quality issues also include $**data inaccuracy and inconsistency**$ with respect to the true values. However, in most physical systems, inaccurate and inconsistent measurements will be removed using mature technologies such as bad data detection [12], anomaly detection [13], noise filtering [14], etc. Consequently, these problems are converted to a data incompleteness problem. In general, data incompleteness is the central, common, and long-standing data quality issue for growing physical systems.

$**Why is G-AlignNet state-of-the-art to tackle all data incompleteness problems in categories$(1)\sim(3)$**$? G-AlignNet can tackle all data incompleteness problems using a unified framework. This is because training G-AlignNet essentially learns the geometric matrix flows $\Theta_x(t)$ and $\Theta_y(t)$, which are further utilized to solve an optimal quality-alignment problem in Eq. (2) in the manuscript. The optimization demands the evaluation of the matrix flow at Low-Quality (LQ) observable times $\\{t_i\\}\_{i\\in\\mathcal{N}\_y}$. Luckily, the matrix flow is continuous and can be evaluated at arbitrary times with the help of ODE solvers, no matter how $\\{t_i\\}_{i\in\mathcal{N}_y}$ behaves (such as irregular interval, random drop, etc.) in categories $(1)\sim(3)$.

Thus, all parameter matrices evaluated at $\\{t_i\\}_{i\in\mathcal{N}_y}$ can be inputted to the optimization. Then, the optimization outputs optimal transformation matrix $Q^{\ast}$ that converts $\Theta_x(t_i)$ at HQ observable time $\\{t_i\\}\_{i\in\mathcal{N}\_x}$ to generate high-quality values for $\Theta_y(t)$. Namely, the output is $\tilde{\Theta}\_y(t\_i)$ at HQ time $\\{t_i\\}\_{i\in\mathcal{N}\_x}$ that can be guaranteed to have good approximations with a small error bound.

Moreover, theoretical analysis shows that the above process in G-AlignNet has, to the best of the author's knowledge, the fastest convergence rate with respect to the number of LQ samples $|\mathcal{N}_y|$. Specifically, according to past analytical frameworks [15-18], errors are caused by measurement noise and the on-manifold flow approximation errors, presented as the first and the second term in the right-hand-side of Eq. (10) in the paper. Compared to the previous methods, we have the same error-bound $\mathcal{O}(\frac{1}{\sqrt{|\mathcal{N}_y|}})$ for the error caused by noise. However, our approximation error is bounded by $\mathcal{O}(\frac{1}{|\mathcal{N}_x|})$, much smaller than the error bound in cutting-edge manifold-based compress sensing, i.e., $\mathcal{O}(\frac{1}{\log{|\mathcal{N}_y|}})$ [17]. The latter result is based on local linearization for the data manifold. Instead, G-AlignNet intelligently combines the high approximation power of ODE flows in Neural ODE and a geometric optimization with global optimality on the well-structured parameter manifold (i.e., orthogonal group). In particular, the structure is, by design, embedded into our proposed representation learning using Lie algebra.

$**Response:**$

We greatly appreciate your thoughtful comments and your acknowledgment that the work is sound, along with the score increase. Below, we address your concerns regarding the study scope, representative experiments, and the expansion of Figure 4. While the paper revision deadline has passed, we will make every effort to incorporate these updates into the final version if the paper has the chance to be accepted.

$**Narrow the Scope of the Paper**$.

In our last revision, we narrowed the scope by comprehensively modifying $**the Introduction**$, $**Assumption 1**$, $**Section 3.3 (Theoretical Analysis)**$, $**the Experiment**$, $**Conclusion and Future Work Section**$, and $**Appendix B**$.

In this revision, we emphasize the scope in the statement of the proposed method. Hence, the study scope is $**complete and clear for the whole paper**$. Specifically,

(1) After introduction Optimization (2), we add the following sentence. "Optimization (2) emphasizes the parameter alignment to represent HQ and LQ measurements with limited noise."

(2) To describe Figure 1, I state that " Figure 1 illustrates how HQ and LQ parameters are aligned to capture the measurement correlations with limited random noise."

(3) In Proposition 2, I mentioned that "Suppose Assumption (1) holds".

(4) After Corollary 1 that states the global optimal solution for the alignment, we add "The alignment captures the HQ and LQ data correlations for weakly nonlinear systems and limited noise. In particular, in Proposition 3, we demonstrate the methods' robustness to linear Gaussian noise. The impact of other nonlinear noise needs further investigations."

$**Representativeness of Table 1 and Table 2**$.

Our experiments in Tables 1 and 2 are representative for several reasons.

(1) $**Dataset Diversity**$. The datasets span a variety of systems influenced by human consumption behaviors (load data), weather patterns (PV/air quality data), and events (event synchrophasor data). Moreover, a continuous ODE system (Spiral data) is also considered.

(2) $**Different data incompleteness scenarios are considered**$, including low resolutions, missing data, and irregularly sampled data.

(3) $**Baseline models are comprehensive**$. In general, discrete sequence models (RNN), continuous models (ODE-RNN, Neural CDEm MFN), and parameter flow-based models (Neural ODE + RNN/Neural ODE + MFN) are comprehensively utilized.

(4) $**Tasks are complete**$. We consider both interpolation and extrapolation tasks for time series.

$**Expansion of Fig. 4**$.

We follow your suggestion to conduct additional tests to expand experiments in Fig. 4. Specifically, we consider the additional LQ data coverage rate: $30\\%, 40\\%, 50\\%$. These tests are sufficient to demonstrate sensitivity with respect to the coverage rates. In general, $**the results are consistent with the results we already present in Figure 4**$. We present the MAPE (%) for additional tests as follows.

$**For interpolation**$:

LQ data rate 30%, 40%, 50%

G-AlignNetR $**7.28\\%**$, $**4.55\\%**$, $**3.13\\%**$

G-AlignNetI 8.62%, 7.36%, 5.65%

Linear Spline 10.29%, 8.65%, 6.91%

Cubic Spline 10.13%, 8.91%, 6.44%

CS 11.24%, 9.38%, 7.13%

DCS 18.45%, 16.42%, 14.78%

Semi-NN 12.45%, 10.35%, 8.12%

MFN 16.24%, 13.13%, 10.24%

$**For Extrapolation**$:

LQ data rate 30%, 40%, 50%

G-AlignNetR $**8.95\\%**$, $**8.25\\%**$, $**7.69\\%**$

G-AlignNetI 11.51%, 11.12%, 10.76%

MFN 14.92%, 14.83%, 13.59%

RNN 12.01%, 11.76%, 11.52%

ODE-RNN 12.16%, 11.91%, 11.69%

Neural CDE 12.05%, 11.79%, 11.53%

Neural ODE+RNN 12.59%, 12.27%, 11.96%

Neural ODE+MFN 13.27%, 12.50%, 12.03%

These results demonstrate the strong performance improvements for our proposed models due to our $**innovative geometric representation learning embedded with geometric optimization for assured quality alignment**$.

We sincerely thank you for your feedback. All necessary updates will be incorporated into the camera-ready version if the paper is accepted, and we hope this revision fully addresses your concerns.