Orthogonal Deep Neural Networks (ODNN): Uncovering Hidden Physics in Partially Observable Systems

$**S1. Concerns of Dataset.**$

$**Response:**$

We thank the reviewer for detailed feedback. Below, we address the concerns regarding the connection to physical laws, clarity of problem settings, input-output relationships, and the validity of function decomposition.

$**Connection to Physical Laws**:$ We acknowledge that some examples, such as signal source separation and watermark removal, may not directly involve traditional physical laws. These examples were chosen to demonstrate the flexibility and generality of ODNN, showcasing its ability to disentangle components with contrasting properties in diverse settings. While these applications do not explicitly model physical systems, they align with the broader theme of separating known and unknown components, which is common in physics-guided learning.

Regarding the robotics dataset, the task focuses on regression for reward functions rather than explicit system modeling. However, the reward function itself is derived from the underlying dynamics of the robotic system, which inherently involves physical principles. For instance, the motion constraints of the robot are governed by kinematic and dynamic equations. We agree that this connection could be better articulated, and we will revise the manuscript to clarify the relationship to physical laws in these examples.

$**Clarity of Problem Settings**:$ We appreciate the feedback about the clarity of the problem settings in Sections 5–7. To address this, we will revise these sections to:

• Clearly define the physical and non-physical components in each problem.

• Explicitly state the inputs ($x$) and outputs ($F(x)$) for each experiment.

• Provide context on how the decomposed function $F(x) = f(x) + g(x)$ relates to the system being modeled.

For example, in the power system task, the inputs are partial measurements of system states, and the outputs are estimated line parameters. This involves separating the known physics (power flow equations) from unknown disturbances. We will add similar clarifications for all experiments to ensure the relationships are explicit.

$**W1. Concern of Orthogonal Decomposition.**$

$**Response:**$

We thank the reviewer for the feedback regarding the applicability of orthogonal decomposition for physical systems.

$**Applicability of Orthogonal Decomposition:**$ We acknowledge that orthogonal decomposition is not universally applicable and depends on the nature of the problem. In this work, we focus on systems where the physical ($f(x)$) and non-physical ($g(x)$) components can be additively separated and exhibit properties that allow for explicit or implicit orthogonality. Specifically:

• Appropriateness for Physics-Guided Problems: Many physical systems exhibit separable behavior due to the superposition of distinct phenomena. For instance, in power systems (Sec 4.4), $f(x)$ represents the known power flow equations, while $g(x)$ captures disturbances or unknown residuals. In these cases, orthogonal decomposition provides a principled way to disentangle the two components.

• Acknowledging Limitations: We agree that not all problems in physics or other domains align with the assumptions of our framework. To address this, we will revise the manuscript to explicitly state the conditions under which orthogonal decomposition is applicable, as well as the scope and limitations of our method.

$**Consistency Between Method and Claims:**$ Our primary claim is that orthogonal decomposition can disentangle physics-informed and data-driven components under certain conditions (e.g., additive separability and orthogonality). To ensure consistency:

• Refining Claims: We refine our claims to emphasize the specific scenarios where our method applies, rather than generalizing its applicability to all physics-related problems.

• Clearer Explanations: We will clarify in the manuscript how orthogonal decomposition is leveraged in each experiment, linking the method to the assumptions and properties of the system being modeled.

$**Q3. Inner Product Calculation.**$

$**Response:**$

We thank the reviewer for pointing out this issue and for highlighting the computational implications of the inner product evaluations.

$**Correction of Notation:**$ We agree with the reviewer that the notation $\langle f(x), g(x) \rangle$ in Equation (3) is improper. To eliminate confusion, we will revise the notation to correctly reflect inner products in function space. Specifically: $h_{\text{ODNN}}(x) = h_{\text{DNN}}(x) - \frac{\langle h_{\text{DNN}}, f \rangle}{\langle f, f \rangle} f(x),$ where $\langle f, g \rangle$ explicitly denotes the inner product of functions $f$ and $g$ over their domain.

$**Evaluation of Inner Products in Practice:**$

• Evaluation of $\langle f, f \rangle$: Since $f$ is a known physics basis function with an analytical form, $\langle f, f \rangle$ is computed using symbolic or numerical integration over its domain. This approach ensures high precision and avoids the need for pointwise evaluations.

• Evaluation of $\langle h_{\text{DNN}}, f \rangle$: The function $h_{\text{DNN}}$ is not available in closed form but is represented by data points generated during training. Thus, $\langle h_{\text{DNN}}, f \rangle$ is approximated by discretizing the integral: $\langle h_{\text{DNN}}, f \rangle \approx \sum_{i=1}^N h_{\text{DNN}}(x_i) f(x_i) \Delta x,$ where $x_i$ are the discretization points, $N$ is the number of data points, and $\Delta x$ is the spacing between points.

$**Experimental observations on computational costs:**$ The number of discretization points, $N$, corresponds to the number of available data points. In our experiments, $N$ is above 1,000, providing sufficient resolution for accurate approximations. In our experiments, these optimizations have successfully kept the computational costs manageable:

• For synthetic datasets in Sec 4.1, the computation of inner products required less than 20% of the total training time.

• For larger datasets (e.g., Sec 4.4, power systems), batching and efficient sampling ensured that inner product evaluations scaled linearly with data size.

$**Q4. More Baseline Comparison.**$

$**Response:**$

We thank the reviewer for emphasizing the importance of benchmarking against specialized prior works in addition to general-purpose methods. To address this concern, we want to clarify the following:

$**Rationale for Comparison with General-Purpose Methods:**$ We included comparisons with general-purpose methods like SINDy and PINNs because these approaches are widely applicable across diverse physics-guided problems. They represent standard baselines for evaluating hybrid frameworks that combine physical and data-driven modeling. Specifically:

• SINDy (Sparse Identification of Nonlinear Dynamics): Offers a data-driven approach for discovering governing equations, making it a relevant baseline for tasks involving physics inference.

• PINNs (Physics-Informed Neural Networks): Leverages known physical laws in neural network training, aligning with our goal of integrating physics and data-driven modeling.

While these methods are general-purpose, they provide a valuable point of reference for assessing the versatility and baseline performance of our approach.

$**Comparisons with Specialized Methods:**$

We agree with the reviewer that each problem setting has specialized prior works, and comparing against such methods would better demonstrate the practical value of our approach. To address this, we incorporated three additional baseline methods that represent recent SOTA of learning hidden physics, as suggested by Reviewer 2:

• [1] Physics-informed learning of governing equations from scarce data.

• [2] Bayesian spline learning for equation discovery of nonlinear dynamics with quantified uncertainty.

• [3] Symbolic physics learner: Discovering governing equations via monte carlo tree search.

These baselines include a physics-informed learning of governing PDE from scarce data $**PINN-SR**$ [1], a Bayesian spline learner $**BSL**$ [2] for equation discovery with quantified uncertainty, and a symbolic physics learner $**SPL**$ [3] leveraging Monte Carlo Tree Search to discover governing equations.

The comparison is summarized into Figure 7 of the revised manuscript. We observe that, in most datasets, the MAPE achieved by the ODNN approach is smaller compared to the baseline methods. This improvement can likely be attributed to the distinct focus of ODNN: while the baseline methods aim to learn hidden physics by modeling the entire system as a single equation or differential equation, ODNN is specifically designed for systems that exhibit the $f$ + $g$ structure. By leveraging the distinction between $f$ and $g$, ODNN enables a more accurate recovery of the physics parameters, ensuring robustness and precision.

$**Q1. Question of Widely Accepted Methods .**$

$**Response:**$

We thank the reviewer for the comment. Below, we address the concerns regarding the connection to physics, the comparison with widely accepted methods like principal component analysis (PCA) and independent component analysis (ICA), and the need to better articulate the contributions in the introduction.

$**Connection to Physics**:$ We acknowledge that in some applications, such as signal source separation and watermark removal, the direct connection to traditional physical laws may appear less explicit. However, these examples were included to demonstrate the general applicability of the proposed framework to systems where components exhibit contrasting properties, even outside classical physics contexts. In applications like power grid system identification and reinforcement learning reward functions, the connection to physics is stronger:

• In the power grid task, the decomposition leverages known physical principles such as power flow equations and orthogonality constraints to estimate system parameters under partial observability.

• In the robotics reward task, the underlying system dynamics involve kinematic and dynamic equations, tying the regression task to the system's physical behavior.

$**Comparison with PCA and ICA**:$ We agree that methods like PCA and ICA are widely used for problems involving component separation. However, the proposed method differs in key ways:

• Domain Knowledge Integration: PCA and ICA are unsupervised methods that rely on statistical properties like variance or independence but do not explicitly incorporate domain knowledge. ODNN integrates known physical constraints (e.g., orthogonality and structural regularization) into the decomposition process, providing a principled framework for disentangling components based on their functional properties.

• Flexibility with Qualitative Properties: Unlike PCA or ICA, which require linear or independent components, ODNN can handle components with qualitative differences, such as convexity, monotonicity, or periodicity. This makes it more versatile for systems where such properties are critical.

• Theoretical Guarantees: ODNN provides theoretical guarantees for accurate disentanglement of components under specific conditions (e.g., Theorem 1), which are not offered by PCA or ICA.

$**Q2. Question of Recovering Physics.**$

$**Response:**$

$**On the Linearity Assumption:**$ We agree that expressing solutions as sums of functions may introduce a form of linearity assumption. However, our approach does not inherently assume that the underlying physical dynamics are linear. Instead:

• The decomposition $\sum_j \theta_j f_j + g(x)$ serves as a modeling framework where $f_j$ represents known physical components, and $g(x)$ accounts for unknown or unmodeled dynamics. While the combination of these components is additive, the functions $f_j$ and $g(x)$ themselves can represent highly nonlinear behaviors.

• For example, $f_j$ may correspond to solutions of nonlinear differential equations or other complex physical processes, and $g(x)$ captures disturbances or discrepancies that cannot be explained by the known physics.

$**On Capturing Physical Laws:**$ The goal of the proposed approach is to disentangle known physical dynamics from disturbances, rather than to derive the entirety of physical laws from data. In this context:

• The term ``physics" refers specifically to the known components of the system that can be described by the basis functions $f_j$. These components are typically derived from domain knowledge or theoretical models (e.g., solutions to partial differential equations, conservation laws).

• The decomposition allows us to isolate these known components, enabling better interpretability and robustness when combined with data-driven models to approximate unknown dynamics.

$**On the Applicability to Nonlinear Systems:**$ We acknowledge that real-world systems are often highly nonlinear and complex. While the proposed decomposition is useful for isolating additive components, it does not preclude the representation of nonlinear interactions within $f_j$ or $g(x)$. For example:

• $f_j$ can represent basis functions derived from nonlinear dynamics, such as Chebyshev polynomials or Fourier series, which are commonly used to approximate nonlinear solutions.

• $g(x)$ can capture residual nonlinearities or stochastic effects that are orthogonal to the known physical components, ensuring flexibility in modeling complex systems.

Thank you for your detailed response.

While I agree with parts of your responses, I am struggling to understand the positioning of this research due to its claims of extensive contributions and wide-ranging applications.

In this paper, two methods are proposed:

• Explicit orthogonality
• Implicit orthogonality (e.g., convexity versus non-convexity, periodicity versus non-periodicity, symmetry versus asymmetry, and monotonicity versus non-monotonicity).

Additionally, the title includes the phrase:

• Uncovering Hidden Physics

To me, these three aspects seem disconnected.

I understand that Explicit orthogonality is effective for applications such as watermark removal and source separation. However, in such cases, $f$ is hardly described as "physics."

While the idea of Implicit orthogonality is insightful, I do not believe these concepts are necessarily orthogonal. Reviewer d9Ly has pointed this out as well. Given the significantly different contexts in which these methods are applicable, it would be better to present them in separate papers.

As far as I understand, in (Astolfi et al., 2021), the disturbances are modeled as convex functions to facilitate control, not due to any underlying physical principle. Similarly, the assumptions that physical phenomena are non-periodic while disturbances are periodic, or that physical phenomena are asymmetric while disturbances are symmetric, are plausible in specific cases but appear challenging to generalize. The problem setting in Section 4.6, for instance, seems highly unrealistic. In this sense, I find it difficult to conclude that the proposed method leverages "physics."

If you aim to use physical governing equations for $f$, as Reviewer GzHN has pointed out, the term "hybrid model" already exists in the literature, which could potentially undermine the novelty of the work.

As all the other reviewers have similarly noted, the proposed method is based on strong assumptions, making it difficult to assess its utility and generality without verification through experiments in practical settings (to be clear, I am not requesting additional experiments but merely explaining the reason behind my low rating).

Therefore, I will not change my score.

Thank you for addressing my comments, but I remain unconvinced by some of the responses provided. Specifically:

1. Regarding W1-1: While I understand the authors' reasoning, my concern is not about whether the two-stage method would outperform ODNN—this is clear to me. Instead, my point is about the necessity of including comparative experiments to substantiate this claim. Demonstrating this through empirical results would strengthen the paper and make it more accessible to readers who might question the theoretical argument alone.

2. The authors claim that the f(x)g(x) and f(g(x)) by simple preprocessing. However, this is currently presented without empirical evidence. I believe it is essential to include a demonstration of this claim. As these experiments should be straightforward to implement, including such examples would greatly enhance the clarity and impact of the paper.

In summary, "talk is cheap"—clear, well-executed experiments can effectively address these concerns. Once these are included, I would be happy to re-evaluate my score and assessment.

$**Q2. Clarification of Figure 1.**$

$**Response:**$

We thank the reviewer for raising this question.

$**On the properties of physical models:**$ We agree with the reviewer that physical laws can exhibit diverse properties other than those listed in Figure 1. In our manuscript, Figure 1 does not want to imply that the physical component $f$ always corresponds to non-convexity, non-periodicity, asymmetry, or non-monotonicity. There are many physical models that do not obey such properties, and we acknowledge that our approach does not aim to address every possible case.

$**Purpose of Figure 1:**$ The primary purpose of Figure 1 is to provide the following guideline for users to evaluate whether our approach is applicable in a given system. (1) Explicit Orthogonality: The first step is to determine whether $f$ and $g$ are approximately orthogonal. If this is the case, our approach for enforcing explicit orthogonality can be applied. (2) Implicit Orthogonality: If $f$ and $g$ are not explicitly orthogonal, the next step is to check whether they possess contrasting properties (e.g., convexity vs. non-convexity, periodicity vs. non-periodicity). In such cases, our approach for implicit orthogonality can be utilized to disentangle $f$ and $g$.

$**Q3. Clarify of explicit/implicit orthogonality**$

$**Response:**$

We thank the reviewer for their careful reading and for raising this important point.

$**Applicability of explicit and implicit orthogonality:**$ The reviewer’s interpretation is correct: in each experiment, we do not apply explicit orthogonality and implicit orthogonality simultaneously. Instead, explicit orthogonality is applied when ($f$ and $g$) are approximately orthogonal in the function space, allowing for direct disentanglement through the inner product. Implicit orthogonality is applied when $f$ and $g$ are not explicitly orthogonal but exhibit contrasting properties (e.g., convexity vs. non-convexity, periodicity vs. non-periodicity) that can be leveraged to disentangle the components.

$**On the assumption of "Unknown" non-physical components:**$ We agree with the reviewer’s observation that our approach relies on some prior knowledge about the nature of the non-physical components. For example, implementing implicit orthogonality requires knowledge of contrasting properties such as convexity or periodicity. In light of this, we acknowledge that the term "unknown" may not fully capture the assumptions made in our framework. To address this limitation, we propose refining our terminology from "unknown" to "unstructured." This better reflects the fact that while the exact form of the non-physical components may not be known, certain properties (e.g., periodicity, convexity) are assumed based on prior knowledge or domain-specific insights.

$**W3. Other implicit orthogonality cases.**$

$**Response:**$

Thank you for this insightful comment. The primary goal of this work is to establish a generalizable framework for implicit orthogonality that disentangles known and unknown system components based on their distinct properties. While we demonstrate ODNN with commonly encountered properties (e.g., convexity, periodicity, symmetry, monotonicity), this serves as a proof-of-concept to showcase the feasibility and effectiveness of the method. The flexibility of ODNN allows for extensions to other forms of orthogonality, which we acknowledge as an important direction for future work.

We agree with the reviewer that there are additional potential forms of implicit orthogonality, such as time-invariant vs. time-variant systems and energy-conserving vs. energy-dissipating dynamics. For example, recurrent neural networks (RNNs) or time-encoding networks could be employed to address time-variant systems, while static architectures may suffice for time-invariant components. Similarly, Hamiltonian neural networks (HNNs) could model energy-conserving dynamics, and dissipative neural networks could handle energy loss. These align with ODNN's philosophy and could be implemented with suitable architectural modifications or constraints.

The reviewer raises an important point about the reliance on existing off-the-shelf architectures. While ICNNs, periodic NNs, and other current architectures provide a strong foundation, ODNN is not restricted to these tools. Instead, ODNN offers a flexible framework that can adapt to or inspire the design of new architectures tailored to specific forms of implicit orthogonality. For example, energy-based constraints could be incorporated into latent space representations to extend Hamiltonian or dissipative dynamics beyond existing implementations.

We acknowledge that this work does not exhaustively cover all possible forms of implicit orthogonality. Future work will focus on extending ODNN to address more complex and domain-specific orthogonalities, developing novel network architectures, and establishing systematic guidelines for identifying and implementing new forms of implicit orthogonality. These extensions will ensure the broader applicability and continued development of ODNN for a wider range of systems.

$**Q1. Evaluation of inner product.**$

$**Response:**$

We thank the reviewer for this insightful question and the opportunity to clarify the numerical integration method used in our implementation.

$**Evaluation of$\langle f, f \rangle$:**$ Since $f$ is a known physics basis function with an analytical form, we compute $\langle f, f \rangle$ using direct analytical integration over its domain whenever possible. This ensures high precision and eliminates numerical integration errors for this term.

$**Evaluation of$\langle h_{\text{DNN**}, f \rangle:} The function $h_{\text{DNN}}$ is not expressed in closed form but is represented by a set of data points generated during training. Therefore, we calculate $\langle h_{\text{DNN}}, f \rangle$ by discretizing the integral into a Riemann summation: $\langle h_{\text{DNN}}, f \rangle \approx \sum_{i=1}^N h_{\text{DNN}}(x_i) f(x_i) \Delta x,$ where $x_i$ are the discretization points, $N$ is the number of data points, and $\Delta x$ represents the spacing between these points. We use evenly spaced $x_i$, which corresponds to the midpoint rule, which is known for its robustness and accuracy in general settings.

$**Addressing Oscillatory Integrals:**$ We acknowledge the reviewer’s concern about potential inaccuracies introduced by the numerical integration method, particularly for oscillatory integrals. In our framework: (1) For smooth and non-oscillatory $f$, the midpoint rule or Riemann summation used in our implementation provides sufficient accuracy, especially given the large number of discretization points ($N > 1,000$ in most experiments). (2) For particularly challenging cases (such as in Sec 4.2), we implement adaptive quadrature schemes, which are well-suited for oscillatory integrals. These methods adjust the integration weights and points dynamically to minimize error.

$**W1-2. Concern about$f+g$assumption.**$

$**Response:**$

We appreciate the insightful comment regarding the assumption of additive separability $F(x) = f(x) + g(x)$ in our formulation and its implications for the use of orthogonality. Below, we address this concern and discuss how our method can be extended or adapted to address other forms of functional combinations, such as $F(x) = f(g(x))$ or $F(x) = f(x)g(x)$.

The additive separability assumption $F(x) = f(x) + g(x)$ is indeed central to our current framework. While we acknowledge that this assumption may not cover all systems, it aligns with a fundamental principle observed in many natural and engineered systems: the tendency to exhibit contrasting or complementary properties that can be modeled additively.

$**Supporting Evidence for Additive Opposites in Real-World Systems**:$ When observing real-world phenomena, additive separability emerges as a natural approximation in systems where known physics and unknown physical effects coexist but do not directly transform or modify each other. The independence of influence in such systems allows for modeling through additive decomposition.

$**Extensions to Address Non-Additive Cases**:$ For $F(x) = f(x)g(x)$, a logarithmic transformation (e.g., $\log(F(x)) = \log(f(x)) + \log(g(x))$) could approximate the problem in an additive form under certain conditions. For $F(x) = f(g(x))$, one potential approach is to leverage a two-stage decomposition, where $g(x)$ is first approximated as an independent latent variable, and $f(x)$ is learned conditionally.

$**Functional Orthogonality**:$ In cases of non-additive interactions, we propose exploring functional orthogonality, where the goal is to disentangle terms based on their functional forms rather than their additive contributions. For example, using neural architectures tailored to approximate hierarchical (e.g., $f(g(x))$) or multiplicative structures.

$**Future Directions:**$ While the current work focuses on additive separability, we acknowledge the importance of extending our framework to handle non-additive interactions. This is a promising avenue for future research, and we are actively exploring techniques to generalize the orthogonality principle to broader settings.

$**W2. Concern about the unknown$g$**$

$**Response:**$

Thank you for raising this important point. We acknowledge that the implementation of implicit orthogonality in ODNN relies on general prior knowledge about the properties of the unknown components (e.g., convexity, periodicity). However, we argue that this reliance is practical and aligned with common practices in physics-guided machine learning. Rather than requiring precise knowledge of the unknown components, ODNN leverages minimal and qualitative characteristics, which are often inferred from domain expertise or exploratory analysis.

For example, in fluid mechanics, turbulent forces are commonly convex; in biological systems, environmental effects exhibit periodicity; and in power systems, disturbances like coupling effects are often monotonic. These broad characteristics of the unknown components are sufficient for guiding the selection of neural network architectures, without requiring explicit parameterization or precise functional forms. This approach strikes a balance between generality and applicability, allowing ODNN to be effective in a variety of real-world scenarios.

It is also worth noting that leveraging prior knowledge is a standard practice in physics-guided machine learning. Hybrid models typically rely on first-principle equations, and physics-informed neural networks (PINNs) enforce physical constraints during training. Similarly, ODNN uses qualitative priors about the unknown components to design architectures that ensure accurate disentanglement. Unlike traditional hybrid models, ODNN provides theoretical guarantees for this disentanglement while avoiding overfitting or requiring extensive hyperparameter tuning.

To address the reviewer’s concern about the claim of ``unknown'' components, we propose revising the manuscript to clarify that the method does not assume the unknown components are entirely unobservable. Instead, we will acknowledge that their general properties are inferred from prior knowledge or data exploration. For example, we could revise the text as follows: “ODNN disentangles known physical components from non-physical disturbances, leveraging general knowledge about the latter’s qualitative properties (e.g., convexity, periodicity). This approach balances prior knowledge with flexibility, making it applicable to a broad range of systems.”

$**W1. Novelty regarding ODNN.**$

$**Response:**$

Thank you for your insightful comment. While we agree that ODNN shares some similarities with hybrid neural networks that integrate physical models and data-driven components, our approach differs significantly in both objectives and methodology. Below, we clarify the novel aspects of our work:

$**Guarantees for Accurate Identification of Physical Parameters**.$ The primary distinction of ODNN lies in its ability to guarantee the accurate identification of physical parameters, which is not achieved by existing hybrid approaches. This capability is rooted in our focus on a specific subclass of physical systems where the mapping function can be modeled as a combination of two components with opposite functional properties.

• As demonstrated in $**Proposition 1 and Figure 2**$, standard hybrid approaches often fail to disentangle physical and non-physical components accurately. Overfitting occurs when the neural network approximates both $f(x)$ and $g(x)$, thereby undermining the recovery of the physical parameter $a$ in front of $f(x)$.

• In contrast, ODNN ensures that $f(x)$ and $g(x)$ are disentangled correctly by leveraging structural orthogonality. This disentanglement enables provably accurate recovery of $a$, as established in $**Theorem 1**$.

• The ability to recover $a$, which represents a key physical parameter, enhances the $**generalizability**$ of the learned model by grounding predictions in interpretable physical laws.

$**Novel Structural Regularization Design**.$ Another fundamental difference between ODNN and traditional hybrid methods lies in the design of regularization.

• Traditional Hybrid Methods: These approaches often rely on $**loss-based regularization**$, where additional terms are introduced into the loss function to penalize deviations from physical laws. This design requires careful hyperparameter tuning to balance the primary loss and the regularization terms. Such tuning can be challenging and may not guarantee convergence to physically meaningful solutions.

• ODNN’s Structural Regularization: Our method introduces a $**structural regularization framework**$ that modifies the architecture of the neural network itself, rather than the loss function. This approach enforces constraints directly through the design of the network, ensuring that the learned components align with the known properties of $f(x)$ and $g(x)$. By avoiding loss-based regularization, ODNN eliminates the need for hyperparameter tuning and ensures that the disentanglement process is robust and principled.

$**W1-1. Alternative training strategy.**$

$**Response:**$

The reviewer has raised an excellent point that allows us to highlight the contribution of ODNN more clearly. Admittedly, there are papers that adopt a two-stage analysis, where the first step is to fit the known physics and the second is to predict the residuals. However, the key challenge lies in the accuracy of identifying the physics parameters in the first step.

For example, in a system described by $y = a f(x) + g(x)$, minimizing $\min_a \sum (y - a f(x))^2$ does not guarantee finding the true $a$. This is because $g(x)$, representing unknown physics, may introduce significant contributions that obscure $f(x)$. In such cases, the neural network used for residual prediction often absorbs portions of $f(x)$, leading to overfitting and inaccurate identification of the physical parameter $a$.

In this manuscript, ODNN addresses this challenge by enforcing structural regularization to disentangle $f(x)$ and $g(x)$, ensuring accurate recovery of the parameters $\theta_j^\ast$. As shown in Proposition 1 and Figure 2, the reviewer’s suggested approach aligns with one of our baselines (``Regular DNN"), where simultaneous optimization of $\theta_j$ and neural network weights suffers from overfitting. Our results demonstrate that ODNN outperforms this approach by avoiding such pitfalls, providing robust parameter identification and improved generalization.

$**Q3. More Baseline Methods.**$

$**Response:**$

We thank the reviewer for pointing out this important concern and for suggesting additional recent algorithms for comparison. To address this, we have expanded our experiments to include three more state-of-the-art (SOTA) baseline methods specifically designed for learning hidden physics:

• $**PINN-SR:**$ A physics-informed learning method that incorporates sparse regression to discover governing partial differential equations (PDEs) from scarce data [1].

• $**BSL:**$ A Bayesian spline learner that performs equation discovery while quantifying uncertainty [2].

• $**SPL:**$ A symbolic physics learner leveraging Monte Carlo Tree Search to identify governing equations [3].

The comparison results are presented in Figure 7 of the revised manuscript, where we evaluate the performance of ODNN against these baselines across multiple datasets. The results demonstrate that, in most datasets, ODNN achieves a lower mean absolute percentage error (MAPE) compared to the SOTA methods. This improvement can likely be attributed to the unique focus of ODNN, which specifically targets systems exhibiting the $f+g$ structure. Unlike the baseline methods, which model the entire system as a single equation or differential equation, ODNN leverages the distinction between $f$ (known physics) and $g$ (disturbances or residuals). This capability enables ODNN to disentangle the underlying physics more accurately and achieve robust parameter recovery, even in the presence of noise or complex system dynamics.

We hope these additional experiments provide a more comprehensive evaluation and demonstrate the distinct advantages of ODNN in recovering hidden physics, addressing the reviewer's concern about outdated baseline comparisons. Thank you for this valuable suggestion.

$**Q1. Multiple Physics Parameters.**$

$**Response:**$

We thank the reviewer for highlighting this important point. To address the concern about scaling to systems with multiple physics functions and real-world complexity, we have added a new experiment focusing on a real-industry problem in power systems: the Optimal Power Flow (OPF) control problem. This example demonstrates how ODNN can handle multiple physics functions $f$, scaling to more complex systems.

In this experiment, the objective is to optimize the trade-off between maintaining voltage stability and minimizing active power generation costs. The formulation involves multiple physics functions $f$, where each $f$ corresponds to a power flow measurement, and their summation determines the power injection at each bus. Specifically: $P_i = \sum_j V_i V_j \left(G_{ij} \cos d\theta + B_{ij} \sin d\theta \right),$ where $G_{ij}$ and $B_{ij}$ are the unknown conductance and susceptance between buses $i$ and $j$, and $d\theta$ is the phase angle difference.

Using the Pecan Street dataset (https://www.kaggle.com/datasets/cici118/damped-harmonic-oscillator), we validated the ability of ODNN to recover these parameters in a realistic setting with multiple interconnected nodes. Figure 5 illustrates the learning trajectory of the parameters $G_{ij}$. The results show that $G_{ij}$ converges to the true conductance values for connected buses and to zero for non-connected pairs, reflecting the true network topology. A similar level of accuracy is achieved for $B_{ij}$.

This experiment demonstrates that ODNN can scale to systems involving multiple physics functions by leveraging the distinction between convex and nonconvex components, as well as separable contributions from interconnected nodes. System operators can use the learned parameters $G$ and $B$ to restore the network topology and recover hidden physics, enhancing operational insights and system monitoring. The success of ODNN in this experiment provides strong evidence of its applicability to real-world complex systems beyond this specific example.

$**Q2. Concern About Assumption.**$

$**Response:**$

We appreciate the reviewer’s insightful comment regarding the challenges of separating physics and disturbances when their behaviors overlap, such as disturbances being proportional to state variables or following similar trends. Below, we address this concern and clarify how ODNN addresses these cases.

$**Assumptions and Practical Scope**$: We acknowledge that the assumption of a clear boundary between physics and disturbances is a limitation in real-world scenarios. ODNN relies on the presence of distinct properties (e.g., periodicity, convexity, or monotonicity) to disentangle these components effectively. While this assumption holds for many practical systems, we recognize that cases where disturbances are proportional to or aligned with physical variables present a more challenging boundary.

$**Handling Overlapping Trends**$: For scenarios where disturbances mimic the trend of physical components, ODNN’s structural regularization framework can still provide meaningful separation by leveraging qualitative differences. For example:

• If the disturbances exhibit proportional relationships but are monotonic while the physical components are non-monotonic, ODNN can exploit this orthogonality.

• Conversely, if disturbances are proportional but non-convex, while physics are convex, ODNN leverages this structural contrast to disentangle the two.

In cases where no such distinguishing properties exist, the current framework would require modifications or domain-specific knowledge to infer additional constraints.

$**Potential Extensions to Address Overlap**:$ To extend ODNN to systems with overlapping properties between physics and disturbances:

• We could incorporate additional constraints, such as temporal correlations, statistical independence, or domain-driven priors, to enhance separation.

• A potential approach is to adapt the network to focus on latent representations, where disentanglement might be more feasible even when the raw trends overlap.

• Collaborative training with auxiliary datasets, where disturbances and physics are known separately, could help improve the robustness of the separation.

We aim to explore these extensions in future work, broadening ODNN’s applicability to systems with less distinct boundaries. While the current framework addresses a significant subclass of systems with clear separability, we agree that addressing scenarios with ambiguous boundaries is a critical direction for advancing the methodology.

$**W3. More Experiments Are Needed.**$

$**Response:**$

We appreciate the reviewer’s observation regarding the scope of experimental cases. Our intention was to focus on carefully controlled scenarios to illustrate the theoretical guarantees and practical performance of ODNN. The synthetic and simplified datasets allow us to validate ODNN's ability to disentangle known physics from unknown disturbances under well-defined conditions, which is crucial for demonstrating the feasibility and robustness of our approach. Meanwhile, we have included two new experiments in the revised manuscript, involving real-world, real-industry systems, demonstrating the applicability of ODNN to complex, realistic scenarios.

$**Reconstructing Oscillator Dynamics with Observed Data:**$ We consider a real-life temporal dynamic system, the damped harmonic oscillator, initialized with an unknown velocity. This dataset, publicly available from https://www.kaggle.com/datasets/cici118/damped-harmonic-oscillator, simulates the motion of the oscillator governed by the equation $m x^{''}(t) + c x^{'}(t) + k x(t) = 0$. The displacement $x(t)$ is modeled as a combination of known and unknown components, where $A \cos(\omega t)$ (related to the initial displacement) is regarded as $f$, and $B \sin(\omega t)$ (linked to the unknown initial velocity) is treated as $g$. By leveraging implicit orthogonality, ODNN effectively disentangles these components, recovering the unknown parameter $B$ with high accuracy. As shown in Figure 5, ODNN accurately reconstructs the dynamics of the oscillator, with the learned $B$ converging to its true value. This demonstrates ODNN's ability to handle complex, real-world dynamic systems with underlying physical laws.

$**Balancing Voltage Stability and Power Flow Optimization:**$ We also tackle a real-industry problem in power systems: the Optimal Power Flow (OPF) control problem. This experiment uses the Pecan Street dataset (https://dataport.pecanstreet.org), which provides real-world power flow data from residential and commercial buildings. The objective involves multiple physics functions $f$, representing the active power flow equations at each bus, and incorporates both convex (voltage stability) and nonconvex (active power generation cost) components. ODNN successfully learns the unknown parameters $G_{ij}$ and $B_{ij}$ (conductance and susceptance between buses), recovering the true network topology. This validates ODNN’s scalability to systems involving multiple interconnected physics functions and its applicability to critical operational tasks in power systems.

These new experiments illustrate ODNN's ability to scale to real-world, complex systems in different domains, addressing the concern that previous experiments were limited to toy problems. By handling systems governed by physical laws and involving realistic dynamics, ODNN demonstrates robustness and flexibility in solving practical challenges across diverse applications.

$**W1. Too Much Detail in Appendix.**$

$**Response:**$

We thank the reviewer for pointing out this issue, and we agree that certain experiment details were overly deferred to the appendix, which may have hindered clarity. To address this, we have moved key details from the appendix to the main paper, particularly in Sections 4.4 to 4.6. Specifically, we now explicitly define the physics-based component $f$ and the disturbance or residual $g$ for each experiment in these sections. By including these definitions in the main text, readers can directly understand the setup and the role of ODNN in disentangling $f$ and $g$ without needing to reference the appendix.

We hope these changes significantly improve the clarity and accessibility of the experimental details, ensuring that reviewers and readers can follow the methodology and results more easily. Thank you for bringing this to our attention.

$**W2. Concern about Assumption 1.**$

$**Response:**$

$**Acknowledgment of Assumption 1's Limitations:**$ We acknowledge that in many real-world environments, the separation of physics and disturbances cannot always be neatly characterized by properties such as periodicity vs. non-periodicity or monotonicity vs. non-monotonicity. Below, we clarify the scope of our assumptions and discuss their practical applicability.

$**Scope and Practicality of Assumption 1:**$ Assumption 1 is designed to target a specific subclass of systems where the known physical components and disturbances exhibit distinguishable properties. While this does not cover all real-world scenarios, it applies to many systems where qualitative differences between the components are well-understood. For instance, in power systems, known physics such as power flow equations are often non-periodic, while disturbances caused by environmental factors may exhibit periodic behavior (e.g., daily load cycles). Similarly, in biological systems, monotonic growth models can be distinguished from periodic environmental effects.

The purpose of Assumption 1 is to provide a theoretical framework for disentangling physics and disturbances under idealized conditions. While this assumption may not always hold exactly in real-world environments, it serves as a starting point for designing methods that can approximate this separation in practical settings. For example:

• In some engineering systems, disturbances may exhibit approximate periodicity or convexity, which can be exploited for separation even if the properties are not strictly orthogonal to the physical dynamics.

• In other cases, domain-specific knowledge can be incorporated to define more sophisticated separation criteria beyond simple contrasting properties.

$**Practical Relevance of the Current Work:**$ Despite its limitations, the current work provides a valuable framework for systems where contrasting properties (e.g., convexity vs. non-convexity, monotonicity vs. non-monotonicity) are observed. These properties are common in engineering, physics, and biological systems, making the framework broadly applicable even if not universal. By targeting such systems, ODNN demonstrates how leveraging structural regularization can achieve accurate disentanglement and parameter identification in practical contexts.

$**W1. Concerns regarding Assumption 1.**$

$**Response:**$

We acknowledge that the implementation of implicit orthogonality in ODNN relies on general prior knowledge about the properties of the unknown components (e.g., convexity, periodicity). However, we argue that this reliance is practical and aligned with common practices in physics-guided machine learning. Rather than requiring precise knowledge of the unknown components, ODNN leverages minimal and qualitative characteristics, which are often inferred from domain expertise or exploratory analysis.

For example, in fluid mechanics, turbulent forces are commonly convex, in biological systems, environmental effects exhibit periodicity, and in power systems, disturbances like coupling effects are often monotonic. These broad characteristics of the unknown components are sufficient for guiding the selection of neural network architectures, without requiring explicit parameterization or precise functional forms. The followings are examples. Implicit Orthogonality Case (Contrasting Properties): If $f$ and $g$ possess contrasting properties (e.g., non-convexity vs. convexity), Assumption 1 can be satisfied by ensuring that all $f_i$ share the same property. For example, in the power system dataset: $f_i(v, \delta) = v_i v_k \left(G_{ik} \cos{\delta_{ik}} + B_{ik} \sin{\delta_{ik}}\right),$ where $v_k$ is the voltage magnitude, $\delta_{ik}$ is the voltage angle difference between nodes $i$ and $k$, and $i \in \mathcal{N}(k)$ denotes the connected node indices. In this expression, each $f_i$ is non-convex, which contrasts with the convex nature of $g$. Therefore, using an input convex neural network (ICNN) for $g$ ensures that the output satisfies Assumption 1, as the network is constrained to only output convex functions. This approach demonstrates that satisfying Assumption 1 in such cases is both practical and achievable.

It is also worth noting that leveraging prior knowledge is a standard practice in physics-guided machine learning. Hybrid models typically rely on first-principle equations, and physics-informed neural networks (PINNs) enforce physical constraints during training. Similarly, ODNN uses qualitative priors about the unknown components to design architectures that ensure accurate disentanglement. Unlike traditional hybrid models, ODNN provides theoretical guarantees for this disentanglement while avoiding overfitting or requiring extensive hyperparameter tuning.

To address the reviewer’s concern about the claim of “unknown” components, we propose revising the manuscript to clarify that the method does not assume the unknown components are entirely unobservable. Instead, we will acknowledge that their general properties are inferred from prior knowledge or data exploration. For example, we could revise the text as follows: “ODNN disentangles known physical components from non-physical disturbances, leveraging general knowledge about the latter’s qualitative properties (e.g., convexity, periodicity). This approach balances prior knowledge with flexibility, making it applicable to a broad range of systems.”

$**W2. Theorem 1 Description.**$

$**Response:**$

We thank the reviewer for highlighting this important issue. We agree that the original statement of Theorem 1 could be misleading by implying guarantees about the optimization dynamics or the training process, which are beyond the scope of our theoretical results. To address this, we have revised the statement of Theorem 1 to explicitly focus on the properties of the minimizers of the loss function in Equation (2), avoiding any implication about the behavior of the training process.

Specifically, the revised statement of Theorem 1 is as follows: "Under the conditions of Assumption 1, any minimizer of the loss function in Equation (2) corresponds to the correct physical parameter $\theta_j$."

$**Q3. What are$f$in general?**$

$**Response:**$

We thank the reviewer for this important question and the opportunity to clarify the role of $f_i$, the choice of basis functions across different tasks, and the unicity of $\theta_j$ coefficients.

In general, $f_i$ refers to the physics basis function in the system, which is assumed to be known due to the underlying physical laws of the task. The choice of $f_i$ depends on the specific task since the governing physical laws vary across tasks. The specific expressions for $f_i$ in each task are as follows:

$**Synthetic Dataset (Sec 4.1)**$:

• For the 1D case, we choose $f(x) = \sin(4\pi x)$.
• For the 2D case, we extend this to $f(x, y) = \sin(4\pi x) \sin(4\pi y)$.

$**Audio Dataset (Sec 4.2)**$:

• The known noise in the audio is modeled as $f(t) = 0.3\sin(440\pi t) + 0.3\sin(660\pi t) + 0.4\sin(880\pi t)$, representing harmonic components of the noise.

$**Reinforcement Learning Dataset (Sec 4.3)**$:

• Here, $f$ is the true reward for upward movement, expressed as $f = \frac{z}{\Delta t}$, where $z$ denotes the post-action $z$-coordinate, and $\Delta t$ is the frame time.

$**Power System Dataset (Sec 4.4)**$:

• For each node $i$, $f_k$ represents the power flow equation: $f_k(v, \delta) = v_i v_k \left(G_{ik} \cos{\delta_{ik}} + B_{ik} \sin{\delta_{ik}}\right),$ where $v_i$ is the voltage magnitude, $\delta_{ik}$ is the voltage angle difference between nodes $i$ and $k$, and $k \in \mathcal{N}(i)$ denotes the connected node indexes.

$**Heat Transfer Dataset (Sec 4.5)**$:

• $f_i$ corresponds to the known heat source: $f_i(x, y, t) = \frac{Q_i}{4 \pi \alpha t} \exp\left( -\frac{(x - x_i)^2 + (y - y_i)^2}{4 \alpha t} \right),$ where $Q_i$ represents the strength of each heat source, $(x_i, y_i)$ are the coordinates of the heat sources, and $\alpha$ is the thermal diffusivity.

$**Driven Pendulum Dataset (Sec 4.7)**$:

• The physics basis is $f(t) = \frac{f_0}{2\omega} t\sin(\omega t)$, where $\omega$ is the oscillator frequency, and $f_0$ is the unknown driving force.

$**Biology Growth Dataset (Sec 4.7)**$: The known exponential growth is modeled as: $f(t) = \frac{S}{S+K_S}\exp\left(\frac{S}{K_S}t\right),$ where $K_S$ is the half-saturation constant, and $S$ is the substrate concentration.

Due to space constraints, we have detailed some of the task-specific physics expressions in the corresponding appendix sections.

$**Unicity of$\theta_j$Coefficients:**$ For real-world systems, $\theta_j$ coefficients are unique and carry specific physical meanings. For instance, in the Power System Dataset (Sec 4.4), the $\theta_j$ coefficients correspond to $G_{ik}$ and $B_{ik}$, the physical parameters of line $ik$. These values are unique to the IEEE 18-node system and are listed in Figure 14. For synthetic datasets, $\theta_j$ does not have a real-world physical meaning. In these cases, we simulate various values of $\theta$ and evaluate the robustness of the method by calculating the averaged estimation error, as shown in Figure 9.

$**Q4. Signal and disturbance occupy distinct frequency bands.**$

$**Response:**$

We thank the reviewer for raising this important question. This hypothesis is based on common practices in signal processing and physical systems modeling. Many real-world systems exhibit this property due to physical constraints or engineering designs. For example:

• In audio processing, signals of interest (e.g., speech or music) typically occupy lower to mid-frequency bands, while noise (e.g., electrical hum or high-frequency hiss) is often confined to higher frequency bands [Oppenheim1999].
• In power systems, harmonic disturbances caused by nonlinear loads are well-known to occupy higher frequency bands compared to the fundamental frequency of the system [Farhangi2009].
• In wireless communication, signals are often designed to occupy specific frequency bands to minimize interference, leaving disturbances or noise in other bands [Proakis2001].
• These distinct frequency distributions make frequency-based separation a widely applicable and practical assumption in many fields.

$**References:**$

[Oppenheim1999] Oppenheim A V. Discrete-time signal processing[M]. Pearson Education India, 1999.

[Farhangi2009] Farhangi H. The path of the smart grid[J]. IEEE power and energy magazine, 2009, 8(1): 18-28.

[Proakis2001] Proakis J G. Digital signal processing: principles algorithms and applications[M]. Pearson Education India, 2001.

$**W7. Relationship between implicit/explicit orthogonality.**$

$**Response:**$

We thank the reviewer for this comment.

• Implicit orthogonality refers to the contrasting properties of $f$ and $g$ (e.g., convexity vs. non-convexity, periodicity vs. non-periodicity) that naturally reduce their overlap in the function space. This is a heuristic or design principle used to model $f$ and $g$ as distinct components.
• Explicit orthogonality is formally defined by the inner product: $\langle f, g \rangle = \int_\Omega f(x) g(x) \, dx,$ where a near-zero value of $\langle f, g \rangle$ quantitatively confirms orthogonality in the $L_2$ sense.
• Their relationship: we provide these two orthogonality cases to cover many real-life scenarios where $f$ can be disentangled from $g$. One can first check if $f$ is approximately orthogonal to $g$. If so, our approach for explicit orthogonality can be applied. If not, one can further check if $f$ and $g$ has contrasting properties (e.g., convexity vs. non-convexity, periodicity vs. non-periodicity) and utilize our approach for implicit orthogonality.

$**Value of the inner product:**$ in practice, the inner product $\langle f, g \rangle$ is used to measure the degree of explicit orthogonality. Below, we provide observations from our experiments:

• Synthetic dataset (Sec 4.1): Since $f$ and $g$ are deliberately constructed to be orthogonal, the inner product satisfies $|\langle f, g \rangle| < 10^{-3}$, confirming near-complete orthogonality.
• Audio dataset (Sec 4.2): The noise and audio signal exhibit slight overlap in frequency bands, resulting in a slightly larger inner product, $\langle f, g \rangle \approx 0.02$. Despite this, the method performs well, demonstrating robustness to approximate orthogonality.
• Other experiments (implicit orthogonality cases): For tasks where implicit orthogonality is applied (e.g., Sec 4.4, power systems), we do not expect $\langle f, g \rangle$ to be near zero. Instead, the contrasting properties of $f$ and $g$ (e.g., periodic vs. non-periodic behavior) ensure sufficient separability for effective performance.

$**W8. Clarity of of inner product**$

$**Response:**$

We thank the reviewer for pointing out this issue. To eliminate confusion, we will replace all instances of $\langle f(x), g(x)\rangle$ in Equation (3) and related discussions with $\langle f, g \rangle$, where $\langle f, g \rangle$ explicitly denotes the inner product of functions $f$ and $g$ over the appropriate domain. For instance, Equation (3) will be changed to $h_{\text{ODNN}}(x) = h_{\text{DNN}}(x) - \frac{\langle h_{\text{DNN}}, f \rangle}{\langle f, f \rangle} f(x)$.

$**Q1-Q2. Calculation of of inner product**$

$**Response:**$

We thank the reviewer for their detailed reading and for raising this important practical question. Below, we clarify how the inner products are evaluated in our implementation:

• Evaluation of $\langle f, f \rangle$: since $f$ is a known physics basis function, $\langle f, f \rangle$ is computed using Riemann integration over the domain of $f$. That is, we rely on the analytical form of $f$ to evaluate the integral directly.
• Evaluation of $\langle h_{\text{DNN}}, f \rangle$: the function $h_{\text{DNN}}$ is not expressed in closed form but is represented by a set of data points generated during training. Therefore, $\langle h_{\text{DNN}}, f \rangle$ is calculated by discretizing the inner product integral into a Riemann summation: $\langle h_{\text{DNN}}, f \rangle \approx \sum_{i=1}^N h_{\text{DNN}}(x_i) f(x_i) \Delta x,$ where $x_i$ are the discretization points, $N$ is the number of data points, and $\Delta x$ represents the spacing between these points.
• Number of discretization points: the number of discretization points $N$ corresponds to the number of data points in our experiments. In practice, $N$ is above 1,000 in most of our experiments, ensuring sufficient resolution for accurate evaluation of the inner product.
• Inner product calculation of multivariate inputs. For multivariate inputs, such as $f(x, y):\mathbb{R}^2 \to \mathbb{R}$ in our synthetic image dataset in Sec 4.1, the inner product is extended naturally by integrating over the multidimensional domain. Specifically, the inner product is defined as: $\langle f, f \rangle = \int_\Omega f(x, y) g(x, y) \, dA,$ where $\Omega \subseteq \mathbb{R}^2$ is the domain of interest, and $dA$ is the infinitesimal area element. For $\langle h_{\text{DNN}}, f \rangle$, this integral is approximated numerically using a Riemann summation. We discretize the domain $\Omega$ into a grid of $N_x \times N_y$ points and compute: $\langle h_{\text{DNN}}, f \rangle \approx \sum_{i=1}^{N_x} \sum_{j=1}^{N_y} f(x_i, y_j) h_{\text{DNN}}(x_i, y_j) \Delta x \Delta y,$ where $(x_i, y_j)$ are the discretized grid points, and $\Delta x$ and $\Delta y$ are the grid spacings along the $x$- and $y$-axes, respectively.

$**W5. Concerns regarding Implicit orthogonality**$

$**Response:**$

We thank the reviewer for raising this insightful question. Our initial intent was to use the term "orthogonality" in a broader sense to unify the concepts of "explicit orthogonality" and "contrasting properties" under a single framework. By doing so, we aimed to provide a cohesive approach that allows users to evaluate whether our method is applicable to their specific system:

• Explicit orthogonality: The first step is to determine whether $f$ and $g$ are approximately orthogonal. If this is the case, our approach for enforcing explicit orthogonality can be applied.
• Contrasting property (implicit orthogonality): If $f$ and $g$ are not explicitly orthogonal, the next step is to check whether they possess contrasting properties (e.g., convexity vs. non-convexity, periodicity vs. non-periodicity). In such cases, we refer to it as implicit orthogonality.

We acknowledge that this broader usage of "orthogonality" might lead to confusion, as the term has a strict mathematical definition related to inner products and orthogonal complements. To address this, we will revise as follows:

• We will use the term "contrasting properties" to describe relationships such as convexity vs. non-convexity or periodicity vs. non-periodicity.
• The term "orthogonality" will be reserved exclusively for contexts involving inner products and projections, ensuring consistency with its precise mathematical definition.

$**W6. Closed functional spaces.**$

$**Response:**$

We thank the reviewer for raising this insightful concern. Below, we address the issue regarding the lack of topological closure of the properties mentioned and clarify the role and relevance of Assumption 1 in our theoretical framework.

We agree with the reviewer that properties such as symmetry, periodicity, monotonicity, and convexity do not define closed spaces in the topological sense. As the reviewer points out, it is indeed possible to approximate a function possessing these properties by a sequence of functions that do not satisfy them. This is a well-known limitation of these properties when viewed through the lens of functional analysis.

Despite the lack of topological closure, Assumption 1 is not intended to rely on the strict topological separation of functions based on these properties. Instead, it serves as an idealized abstraction for cases where $f$ and $g$ are sufficiently distinct in their behavior, either due to orthogonality or contrasting properties. The goal of Assumption 1 is to provide a theoretical framework for disentanglement, rather than to claim universal applicability across all function spaces. For instance:

• In practical applications, functions with periodic vs. non-periodic or convex vs. non-convex behavior are often distinguishable based on their impact on system dynamics. These distinctions are sufficient for the success of our method, even if the associated function spaces are not topologically closed.
• The framework relies on the ability to constrain the neural network $h_{\text{ODNN}}$ to output functions that align with $g$ while remaining orthogonal (explicitly or implicitly) to the known physical basis $f$. This practical separability is sufficient to implement the framework, as demonstrated in our experiments.

While Assumption 1 does not apply to all function spaces, it remains relevant in scenarios where $f$ and $g$ are distinguishable based on domain knowledge or practical constraints. For example:

• In the context of signal processing, periodic disturbances can often be separated from non-periodic physical signals using frequency-domain techniques.
• In optimization problems, convex residual terms are often distinguishable from non-convex physical dynamics due to their differing properties in gradient-based methods.
• These practical scenarios demonstrate that Assumption 1, though idealized, captures the essence of separability in many real-world systems.