Self-supervised contrastive learning performs non-linear system identification

MORE REALISTIC EXPERIMENTS

As stated by the authors in the limitations of this work, the focus of this paper is only on simulated data. While I understand their point of view, and I also agree that the theoretical contribution/simulated experiments are also valuable by themselves, I fear that the impact of this work in this current state will be more limited than it could be if there was a better demonstration of real world applicability.

You could for example try to apply your method to some of the datasets used in "Discovering State Variables Hidden in Experimental Data" (https://www.cs.columbia.edu/~bchen/neural-state-variables/).

If the above is too challenging to achieve, you should at least try to discuss more in detail what each of you theoretical assumptions means in practice, and what you expect to happen if they are not met in real-world experiments. For example, the fact that $p(u_t)$ is a normal distribution seems quite strict in many applications.

BASELINE

The baseline you use in your experiment seems quite weak, as it does not even use a dynamics model. Have you tried other approaches, for example models doing next-token prediction tasks?

CLARITY

There are several missing definitions/clarifications in the paper that make it a bit harder to follow:

1. N in (3) is not defined
2. "Supp figure" in line 133 is unclear
3. Not sure what "(…)" in line 145 means
4. The name $\nabla$-SLDS is never formally defined
5. In table 1 you have a column called "theory" with different options. What do these option represent exactly?
6. The abbreviation "GT", which I assume stands for ground truth is used in many places but never defined
7. What is $\pi$ in equation (8)?
8. The vMF abbreviation is never defined
9. The DynCL results from Table 1 should be discussed more in depth.

• Notations are not clear. For example, in Eq (3), the meanings of $x, x', x''$ should be mentioned in advance.
• In theorem (1) and its proof, the assumption is not aligned with Equation (1-2), where all noise disappears. Further discussion is required.
• The difference in theorem part should be compared with previous works on CL more detailed, since it is a work focusing on theorem. Making the difference more clear will make it more readable.
• In experiment parts
  • Baselines like TCL should be compared
  • By identifiability, some metrics like MCC should be compared even though they are not component-wise identifiable.
• Lots of typos:
  • line 133 (supp figure), line 145 (...), seems unfinished part
  • line 250: tailor -> talyor
  • line 637: Theorem 2 -> Theorem A1
  • footnote of page 13: broken reference
  • line 659, 665: missing reference

In my opinion, the paper leaves quite a few critical questions unanswered, and in general suffers from a lack of polish. In its current state, I cannot recommend the paper for acceptance. The main weaknesses in my eyes are the following:

The paper claims to perform latent nonlinear system identification. This is a key desideratum in various fields such as reinforcement learning and continuous control, and thus has a rich history and literature. However, the assumptions in this paper--and notably how these inductive biases propagate to the algorithm design--severely restrict the applicability of the method without further evidence. Notably, a design assumption in this paper is that the observer function (i.e."mixing function") is invertible. This is a very strong assumption in the context of non-linear system identification, where even the foundational theory of linear system identification does not presume: in the Linear-Quadratic Gaussian (LQG) model, where the underlying state evolves linearly $x_{t+1} = Ax_t + Bu_t + w_t$, and observations are a linear function of state $y_t = Cx_t + v_t$ (ignoring the control input term for simplicity), the classical set-up has $d_y < d_x$, such that the observations are per-timestep a low-dimensional measurement of the underlying state. This immediately rules out the mixing function $g(x) = Cx$ being invertible, and this is precisely the motivation for notions such as observability/detectability. Partial observability presents the key challenge in non-linear sysID or reinforcement learning. In particular, it is well-known in controls and RL that ignoring partial observability and imposing a Markovian model (which this paper does implicitly by enforcing the state estimate as a function solely of the current observation) can lead to very undesirable outcomes. In the contrastive learning literature, partial observability is usually not a central issue, often because it is irrelevant for the motivating application (e.g. in computer vision), but one must address this problem for time-series data. In fact, the cited Time-Contrastive Learning method (Hyvarinen and Tomioka, 2016), despite making the same assumption in theory, actually propose a method that is more amenable to partial observability, since they predict categorical labels to chunks of observed data.

Regarding the polish of the paper, there are various typos and lacking definitions that make the paper hard to parse at times. The minor ones that I have caught are listed below. A particularly confusing point is the role of the control input $u_t$. The paper presents the control input as entering the latent dynamics directly. However, it is typically the case that the control input enters the state through a (possibly state-dependent) actuation matrix $\mathbf B(x_t) u_t$. In any case, how the control input enters the dynamics should be dependent on the parameterization of the dynamics, e.g. the affine ambiguity in $\mathbf L$ in the paper, which is not reflected in the authors' method as far as I can tell. Furthermore, it is unclear if the control input is available to the learner (which is usually the case in sysID), or if it is playing the role of stochastic noise, which eq (9) seems to suggest compared to eq (1). In either case, what role is the control input playing here: in the authors' set-up, there is no need to learn the actuation matrix, and the experiments involve learning a low-noise, nearly deterministic Lorenz system, which rules out some persistency of excitation effect (Tsiamis and Pappas, 2019).

Minor comments/typos:

Figure 1: x -> $x$

Page 3: "linear identifiability (...)", missing eqref?

Theorem 1: "bijective dynamics model $\mathbf f$", should probably mathematically define what that means.

Theorem 1: $\lambda$ is not defined in the main paper, only in the appendix.

Corollary 1: "$\hat{\mathbf f} :=1$", seems to be bad notation.

Beginning of Sec 4: "non-lineary" -> "non-linearly"

Equation (7): where is $g_k$ defined? Possible hash collision with mixing function notation.

Table 1: should probably introduce acronym "LDS" = Linear Dynamical System somewhere

Table 1: What does LDS$\downarrow$ mean?

Table 1: What do $\mathbf L$, $\mathbf L'$, $(\checkmark)$, $\mathbf I$ in the theory column indicate?

Between (11) and (12): "tailor" -> "Taylor"

Before Sec 5: "matices" -> "matrices"

Implementation paragraph: possibly missing number of A100 cards?

Eq (23): where is $p_u$ defined?

Eq (25): what does $p_{D}(y)$ denote precisely?

After Eq (50): "which is probably still fine because $\exp(-\|\mathbf L \cdot \|^2)$ is a valid kernel function (?)". This probably needs to be formalized/reworded.

References:

Anastasios Tsiamis and George J. Pappas, "Finite Sample Analysis of Stochastic System Identification", 2019.

1. The contributions (comparison with previous work), theoretical techniques, and novelty are very lightly discussed. I would appreciate a detailed discussion comparing this work's conditions with those in recent literature on temporal causal representation learning (e.g., [1] and its follow-up work). This would aid the contextualization of this work and make the contribution more transparent.

2. This work strikes me as theoretically oriented. There is a lack of discussion on the theoretical conditions, limitations, and implications. Just to name a few:

(a) the theorem requires the length for each time series to be infinite, which appears to be quite a strong assumption and not necessitated in recent work (e.g., [1], admittedly I am not an expert and would appreciate correction if I have misunderstood the statements). Why is this necessary, and would it be possible to show results for finite-length series?

(b) How does the control distribution influence the identification results? what conditions should it meet (does it have to be a time-independent Gaussian distribution)?

(c) Line 175: what is the significance of $A$ -- is it previously defined? Could you comment on lines 175-177?

3. Some minor issues:

(a) Line 145: what is (...)?

(b) Line 165: shouldn't $L^{\top} L$ be a matrix?

[1] Temporally Disentangled Representation Learning. Yao et al. NeurIPS 2022.

The paper needs refinement, with minor typos and inadequately captioned figures. While studying the identifiability of time-series contrastive learning might be novel, all the technical tools require carefully controlled assumptions and specific behavior of Jacobians under contrastive loss minimization, which typically do not hold in practice. Nonetheless, such assumptions are common in the literature on the identifiability of dynamical systems from observed time series.