Inter-Environmental World Modeling for Continuous and Compositional Dynamics

Regarding the Action space $\mathcal{A}$

This section is meant to serve as our response to Q2 (Underdefined Action Space). To resolve the confusion, we elaborate our the defintiion and the role of $A$ in WLA with more formality.

We intentionally left the action space $\mathcal{A}$ as abstract as possible, because it is meant to represent the set of "controller inputs" of an unknown interface. At the minimum, let us assume that $\mathcal{A}$ is a topological space. $\mathcal{A}$ may depend on $\mathcal{E}$, so that we shall more precisely write $A_E$ (please forgive us in writing this as $A_E$ in this response, because there is a problem with Markdown.) With this in mind, let us elaborate on where $A_{E}$ stands in the WLA framework.

Our general assumption is that the time series in the world model can be controlled with continuous time series of action inputs. More precisely, let $I=[0,T]$ be the time span and $C(I, A_E ) \subset \lbrace a | I \to A_E \rbrace$ be the space of the control sequences. In an application, we are literally expecting each element of $C(I, A_E )$ to be the sequence of inputs from the controller interface. We assume that there exists a controller interface map

$Ctrl_E : C(I, A_E) \times C(I, \mathcal{X}) \times I \to T_{id}G$

where $T_{id}G$ is the tangent space of $G$ at $id$, or the Lie Algebra of $G$. Because $Ctrl_E$ is "a controller" whose effect depends only on the past, we assume that $Ctrl_E(a, x, t)$ depends only on $a[0, t) = \lbrace a(s) | s \in [0, t) \rbrace$ and $x[0, s) = \lbrace x(s) | s \in [0, t) \rbrace$. Therefore let us write $Ctrl_E(a, x, t) := Ctrl_E(a[0, s), x[0, s))$.

It is for brevity we wrote $Ctrl_E : A_E \times \mathcal{X} \to T_E$ in the manuscript to convey the general idea that the controller input is mapped to transitions, but we are sorry if this caused confusion. In the revised Appendix section, we will add more formal explanations as we presented here.

In the training of $Ctrl_E$, we are assuming a "labeled" sequential dataset of small size $D_{label} = \lbrace (a_i, x^i) \rbrace_i \subset C(I, A_E \times \mathcal{X})$ such that, with $(\Phi, \Psi)$ and $M$ (the learned representation of $G$) in place,

$x^i(t) = \Psi \left( M \left( \exp \left( \int_0^t Ctrl_E(a_i[0, s), x^i[0, s) ) ds \right) \right) \Phi (x^i(0)) \right).$ The purpose of Controller Interface Problem is to learn $Ctrl_E$ from $D_{label}$, as described in the manuscript. Note that, in this expression, we intend $\exp \left( \int_t^{t+\delta} Ctrl_E(a_i[0, s) , x^i[0,s)) ds \right)$ to represent $g^i_{t, \delta}$ in the first part of our response, so that $g^i_{t, \delta}$ depends on $\lbrace Ctrl_E(a_i[0, s), x^i[0,s) | s \in [t, t+\delta) \rbrace.$

We are sorry that we forwent the details of the mathematical setups. Our formal mathematical settings are mostly built on the theory of NFT (Koyama et al., 2024), except that we use continuous and nonstationary time series for training the model. We provide more details in our response below. In the revision, we will prepare an Appendix section with more rigorous notations and mathematical setups.

Please see the Part III for the summary of our responses to each component of the raised concerns.

Regarding the assumption underlying WLA

This section is meant to serve as our response to Q1 (Ambiguity in Transition Operator $g_{t, \delta}$) and Q3 (Non-Compositional Transition Operators). To resolve the confusion, we will elaborate the assumption underlying WLA below with more formality.

Let $D = \lbrace x^i: [0, T] \to \mathcal{X} \rbrace_i$ be the video dataset to be used to train the encoder $\Phi$ and the decoder $\Psi$ in our WLA framework, where $i$ is the sample index. This dataset contains the videos sampled from multiple environments so $\mathcal{X}$ is a superset of observation space of all environments. The underlying assumption of WLA is that, for every $i$, $t >0$ and $\delta>0$, the transition $x^i(t) \to x^i(t+\delta)$ is realized by some common group action $G \times \mathcal{X} \to \mathcal{X}$ of a some Lie Group $G$.
To be more precise, our standing assumption is as follows:

Assumption: For our dataset $D= \lbrace x^i: [0, T] \to \mathcal{X} \rbrace_i$ consisting of movie samples from multiple environments, there exists some Lie group $G$ that acts on $\mathcal{X}$ through a group action $\alpha : G \times \mathcal{X} \to \mathcal{X}$ such that, for every $x^i \in D$, $t >0$ and $\delta>0$ there is some $g^i_{t , \delta} \in G$ such that $\alpha ( g^i_{t , \delta}, x^i(t) ) = x^i(t+\delta)$.

For brevity, we notationally identified $g$ with the map $\alpha_g: x \mapsto \alpha(g, x)$ ( that is, $g^i_{t , \delta}: \mathcal{X} \to \mathcal{X}$ with $g^i_{t , \delta}(x^i(t)) = x^i(t+\delta)$. ) We are sorry that the confusion might have been incurred by our typo of "$\to$" in place of "$\mapsto$". We shall more correctly write $g^i_{t , \delta}: \mathcal{X} \to \mathcal{X}$. We will fix this typo in the revision. With this assumption, we are therefore assuming that the family of transitions $T_\mathcal{E}$ can be identified with a subset of $G$ for every $\mathcal{E}$.

We introduce this assumption of the Lie group structure as our inductive bias because our desiderata of compositionality and continuity matches (almost exactly, except for the invertibility) with the definition of Lie group as a topological set with smooth manifold structure and rules of compositions. Note that, by leaving the group action and the details of $G$ unknown, we leave much freedom in our construction of the world model. We do note however that we assume $G$ to be abelian in order to make the latent transitions computationally simple.

According to the theory and results presented in NFT, when there is some group $G$ acting on $\mathcal{X}$ with a set of reasonable regularity conditions, there exists an autoencoder $(\Phi, \Psi)$ with latent vector space $V$ and a homomorphic map $M: G \to Aut_{\mathcal{R}}(V)$ such that $\Phi ( M(g) \Psi (x)) = g(x)$. Mitchel et al. (2024) and Miyato et al. (2023) had experimentally shown that such $(\Phi, \Psi)$ and $M$ can be learned from a time series dataset, and our work scales their philosophy to the nonstationary, continuous dynamical system on large observation space. In our manuscript, this $M: G \to Aut_{\mathcal{R}}(V)$ derived from $\Phi$ and $\Psi$ is written as $F_{\Phi, \Psi}: G \to Aut_{\mathcal{R}}(V)$. As for the expression around eq (3) in concern, the continuity part of our statement more precisely reads as follows:

Continuity: When each continuous transition $x^i(t) \to x^i(t + \delta)$ is realized by a group element $g^i_{t , \delta}$ so that $\Phi ( M(g_{t, \delta}) \Psi (x^i(t))) = x^i(t + \delta)$, then $\lim_{\delta \to 0} M(g_{t, \delta}) = \lim_{\delta \to 0} F_{\Phi, \Psi}(g_{t, \delta}) \to I$.

In the revision we will strike out ambiguous expression like "when the set of transitions form a Lie group". We chose the modeling of the transitions like (Koyama et al, Miyato et al and Mitchel et al) because (1) if $\Phi$ and $\Psi$ are continuous and invertible, the compositionality and continuity of $M$ directly lift to the those of $G$'s action on $\mathcal{X}$, and because (2) $M(g)$ for each $g$ can be represented by sparse (block diagonal) matrix that is computation-friendly (assuming that finite-dimensional representation space with sufficiently large dimension can reproduce the observations). That is, we use $F_{\Phi, \Psi}$ as the environment-agnostic encryptor of the underlying transition operators.

Q3 (Noncompositionality of $g_{t, \delta}$)

Due to the triviality of ( $g_{t, \delta}$ ), composing transitions over time is not feasible, so why does it form a Lie group?

Once again, our assumption is that all transitions $\mathcal{T}_E$ in the video dataset are realized by some common group action of some Lie group, but $\mathcal{T}_E$ is not necessarily the Lie group itself. We agree that our expression of "transitions forming a Lie group" can be misleading, so we will strike such an expression. Our intension regarding the composability is referring to the composability of transition operators (action of group elements), and they can indeed be composed because we are assuming them to be a member of a Lie group.

It is both theoretically and experimentally verified in the past works (Koyama et al, Mitchel et al, Miyato et al) that the representations of hidden group actions can be found through the autoencoder in which the the transisions are modeled linearly in latent space, and it has been verified that the learned group actions are indeed composable. The learning of $\Phi, \Psi$ in WLA scales their works to non-stationary video dataset of high complexity.

The paper introduces Lie groups without a clear justification for their relevance to the specific environments. Additional reasoning would strengthen its argument for using Lie groups to model compositional and continuous dynamics.

We would like to emphasize that our goal is not to learn an interactive model for a specific environment. Rather, we aim to learn a universal model that can be generalized across multiple environments with continuous dynamics. As we stated above, we assume that all transitions in all environments are realized by a common Lie group. We use the Lie group because our goal is (1) to uncover a family of environment-agnostic transition operators that are composable and differentiable with respect to time, and (2) to use the family to learn an interface that can be used to control various environments.

Lie group is almost a forced choice in modeling such a family of transition operators because a topological set that is both smooth (differentiable) and is closed under a consistent rule of composition is the almost definition of Lie group (a group is a set with identity that is closed under composition and inversion). We would also like to add that because the representations of Lie group can be expressed as direct sum of matrices, and this benefits us computationally in composing the transition operators both at the time of training and inference (especially when we use abelian Lie group. )

We make sure that we make revisions according to our answers above. That being said, our specific responses to each components of the three Questions are as follows.

Q1 (Ambiguity in Transition Operator)

( $g_{t, \delta}$ ) maps specific observations ( $x(t)$ ) to ( $x(t+\delta)$ ), making it appear as a trivial one point to one point mapping. This definition does not capture the desired dynamic evolution. It does not make any sense in the form written in the paper.

We are sorry for our typographical use of $\to$ in place of $\mapsto$. We assume that all transitions in all environments are the dataset are caused by actions of Lie group $G$, and we meant $g_{t, \delta, i}: \mathcal{X} \to \mathcal{X}$ to indicate a group action of 'an' element $g_{t, \delta, i}$ such that $g_{t, \delta, i}(x^i(t)) :=g_{t, \delta, i} \cdot x^i(t) = x^i(t+ \delta)$.

The paper acknowledges that ( $g_{t, \delta}$ ) depends on individual trajectories but simplifies it as a generic operator, risking confusion.

Indeed, at the time of the training, $g_{t, \delta} \in G$ should be more correctly labeled as $g_{t, \delta, i}$ to represent a group that is assumed to realize $g_{t, \delta, i}(x^i(t))= x^i(t+ \delta)$. It was our wish to omit $i$ when we want to discuss a generic random trajectory $x$. In the revision, we make sure to use $i$ when the label is important.

This omission detracts from the model’s mathematical precision, especially for multi-environment dynamics.

Because our underlying assumption is that there is some "common" $G$ that realizes $\mathcal{T}_\mathcal{E}$ for all $\mathcal{E}$, we believe that the absence of $i$ in the notation is orthogonal to the discussion of multi-environment dynamics.

Q2 (Underdefined Action Space)

The action space ( $A$ ) and action ( $a(t, \delta)$ ) lack clarity on whether actions are fixed or variable over intervals. This ambiguity in structure reduces the framework's comprehensibility in continuous control scenarios.

We are sorry that the informal notation of $a(t, \delta)$ might have caused confusion. More formally, we assume that $A$ is some topological space, and our underlying assumption in CIP is that there exists some ground-truth map that maps a time series $a : I \to A$ to a time series on $G$ (e.g $g : I \to G$).

Because we assume $a \in C(I, A)$, where $C(I, A)$ is the set of all continuous map from the time span $I \subset R_+$ to $A$, our $a(t)$ is assumed to change with time $t$. However, because the implementation itself must be conducted discretely, $a(t)$ is assumed to be fixed over $[t_i, t_{i+1}]$ with $t_{i+1}- t_i = \delta$ and $t_i = t$, and we originally meant $a(t, \delta)$ to informally represent $\lbrace a(s) | s \in [t, t+\delta) \rbrace = \lbrace a(s) | s \in [t_i, t_{i+1}) \rbrace$ that takes the same value $a(t)$ over the interval $[t, t+\delta]$. We admit that this can be a confusing notation, so we will not use the notation of $a(t, \delta)$ in the revision and instead add more comments to our description regarding discretization (Below eq (5)).

Thank you for reconsidering your score and for your continued engagement with our work. We understand that you still have some concerns, particularly regarding our assumption of reversibility of actions. We would like to address them.

For example, the authors assume that these actions are always reversible; in most cases, this is not true.

We appreciate your feedback and would like to clarify our assumptions in this regard. In our framework, it is important to make a distinction between the following two types of meaning for the word “action”, dealt separately in the training of $(\Phi, \Psi)$ and in CIP:

1. The group action $g$ and the transition of data $x$ modeled by $g$.
2. The external action signal $a$.

Since it was unclear to us which one your question refers to, we will address both.

1. Reversibility of the Group Action $g$ and Data Transitions

It is true that the group action $g$ in our model is invertible (reversible), because in group theory, every element has an inverse.Moreover, we consider deterministic processes, where the forward transition from $x(t)$ to $x(t+\delta)$ is one-to-one. This means the inverse mapping from $x(t+\delta)$ back to $x(t)$ is always well-defined [1].

While this assumption in our modeling may seem restrictive, note that “every video can be played backward” in real world applications. 　For example, in the real physical world, we cannot directly “undo” the operation of throwing a water balloon against the wall. However, we can surely “imagine” the undoing process of gathering the scattered water drops and repairing the balloon by enacting imaginary forces. In modeling, our group action of $G$ contains such an “imaginary” operators as well, but this is not to be confused with those triggered by the external action signal we describe below.

2. Reversibility of the External Action Signal $a$

When we solve the Controller Interface Problem (CIP), we create a mapping from the external action signal $a$ and observation $x$ to $g$ via the Controller Ctlr. For example, suppose we are playing a Mario game. If Mario falls from a cliff higher than he can jump, no matter what button operations we perform (unless we restart the stage), we cannot return to the state before he fell off the cliff.

Let us consider the set of operators that the controller can output when the player is in state $x$, denoted as $\mathcal{T}[x] \subset G$. When $x$ is the state of Mario at the bottom of the cliff, the example above illustrates the fact that $\mathcal{T}[x]$ does not necessarily include elements that can reverse the action of "falling off the cliff." Fortunately, such situations are within the scope of our controller's design. This is because our controller is designed to take the history of $a$ and $x$ as input, so $\mathcal{T}[x]$ depends not only on $x$ but also on the history of $x$.

As an extreme case, if we have $\mathcal{T}[x] = G$, the model would simulate operations as if it is in a debug mode. This is not what we usually want to do, however, because we aim to obtain a controllable simulator that is faithful to the actual dynamics. We can avoid this scenario by property training the controller Ctlr.

---

In summary, we would like to emphasize that the reversibility of $a$ and the reversibility of $g$ are distinct considerations. The transition $g$ results from both $a$ and $x$, and our assumption of reversibility in $g$ does not constrain our theory. Moreover, we would like to point out that the reversible assumption of G doesn’t hurt the model’s performance. Our experiments show WLA outperforms Genie numerically in both 2D and 3D environments.

If you have additional concerns about the reversibility of actions or other aspects of our work, we would be grateful if you could specify them. We are committed to addressing all your concerns thoroughly.

[1] https://en.wikipedia.org/wiki/Time_reversibility

Thank you for your positive feedback and thoughtful review. We appreciate your insights and address your concerns and questions below.

Response to Weaknesses

W1. Limited Experiments

My main concern is the limited amount of comparability to previous works and approaches. The model is only benchmarked on two datasets, and one is mainly used as a (successful) sanity check.

To address this concern, we have conducted an additional experiment on a large video dataset of robot actions in real-world environments (the 1X World Model dataset). This dataset contains over 100 hours of recordings of android-type robots manipulating objects in various settings, including narrow hallways, spacious workbenches, and tabletops with multiple objects under different lighting conditions. We compared our method against Genie using various metrics. While our method slightly underperforms Genie in terms of per-frame fidelity (PSNR), it significantly outperforms Genie on the temporally local fidelity ($\Delta t$ PSNR) and the video-specific metric (FVD).

To further support these quantitative measurements, we will include video demonstrations of our world model control in the supplementary material. In these videos, comparing our method against ground truth and Genie, we observe that our method predicts much more contextually realistic responses to the control sequences.

For more details, please refer to our general response G1.

---

Response to Questions

Q1. Was the encoder and decoder trained from scratch?

Yes, both the encoder and decoder were trained from scratch without using any pretrained models from other datasets.

Q2. Did you do multiple runs with different random initializations, or are the results from just one experimental run?

Due to limited computational resources, we report results from single runs.

Thank you for your constructive comments and feedback. We address each of your concerns and questions below.

---

Response to Weaknesses

W1. Presentation

In places the presentation can be a bit unclear. This is mostly in the description of the method. I would have benefitted from a diagram of the steps required to train WLA and then use that to solve the CIP. The current Figure 1 may not be necessary; the inter-environment aspect of WLA is probably the most straightforward step in the process.

Thank you for highlighting this issue. We will revise our figures to include a detailed diagram that illustrates the training steps of WLA and how it is utilized to solve the CIP. We will also consider modifying or removing Figure 1 to improve the clarity and focus of our presentation.

W2. Related Work

Although not direct alternatives to the latent space presented here, there is some previous work in structured latent spaces that it may be interesting to compare against in the related work section.

We appreciate you bringing these relevant works to our attention. We will include discussions of "Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images" (Watter et al.) and "Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems" (Goyal et al.) in the related work section.

In particular, the SCOFF model from "Object Files and Schemata" shares similarity with our approach because our approach too introduce concepts analogous to schemata and object files. In our work, the irreducible components of the Lie group action (such as abstract rotation and scaling) function similarly to schemata, and we use slot attention mechanisms akin to object files. Our key difference lies in equipping these schemata with structures of continuity and compositionality, built upon a Lie group structure shared across multiple environments. This structure acts as an environment-agnostic dictionary of dynamics.

Regarding "Embed to Control" (E2C), while it also advocates controlling dynamical systems in latent space, our work fundamentally differs from E2C in that we combine algebraic latent modeling with frameworks like VPT and Genie to separate the causal part of the model from the non-causal part. WLA learns the latent structure in the non-causal part without control sequences, capturing environment-agnostic symmetric structures of the observation space that is independent of causal mechanisms. This separation allows us to model environment-dependent causal dynamics and control inputs distinctly. E2C's ideas are complementary to ours, and applying our scheme to more application-specific tasks is an avenue for future work.

W3. Typo

Not a weakness, but Line 111 contains a reference to OCCAM, which I presume is a previous name for WLA?

Thank you for catching this typo. You are correct; "OCCAM" should be "WLA." We will correct this in the revision.

Dear Reviewer,

Now that the authors have posted their rebuttal, please take a moment and check whether your concerns were addressed. At your earliest convenience, please post a response and update your review, at a minimum acknowledging that you have read your rebuttal.

Thank you, --Your AC

Thank you for your thoughtful review and valuable comments. We have addressed your concerns below.

Response to Weaknesses

W1. No runtime analysis

We appreciate your suggestion to include a runtime analysis. While recent studies in our field, such as Genie and VPT, do not provide detailed runtime analyses due to the architecture-agnostic nature of their frameworks, we recognize the importance of this information.

We provide below the computational complexity of our framework when using a Transformer with single-head attention. Let us define:

• $T$: Number of video frames
• $N$: Number of slots
• $D$: Embedding dimension
• $P$: Number of image patches

The computational complexities of each module in our framework are as follows:

In the revision, we will include a section in the Appendix detailing these parameters.

The two major sources of computational cost in training our encoder are:

1. Cross-attention between slots and patches: $O(TNPD)$
2. Feed-forward network: $O(TND^2)$

The decoder computes self-attention for the patches of every slot, requiring $O(TNP^2 D)$. For time evolution, we compute the element-wise products for scaling and rotation, utilizing a RoPE-like computation trick since the matrix $M$ is diagonal. The controller $Ctlr_{\text{adapt}}$ is implemented as a spatio-temporal transformer, consisting of a stack of self-attention layers dedicated separately to time and slots.

W2. No discussion of hyperparameter selection

Thank you for highlighting the importance of hyperparameter selection. Please refer to our general response G2.

W3. No significance tests and no error bars

We acknowledge the importance of significance tests and error bars for enhancing transparency. However, in the domain of world models, there is no standard protocol for reporting statistical significance, and prominent studies like Genie do not provide such reports. Datasets like Phyre and ProcGen offer only two data splits (train/test), limiting the ability to perform significance tests.

Training world models multiple times to compute statistical significance is also very computationally demanding. Nonetheless, we understand the value of such evaluations and have conducted an additional large-scale experiment on videos of robot actions in real-world environments. Please refer to our general response G1 for details on this experiment.

W4. No scaling laws

We agree that analyzing scaling laws can provide valuable insights. However, scaling laws are typically used to support claims about foundational models whose generalization abilities improve monotonically with increased data and model size, potentially learning abilities not directly accessible in the dataset.

Our current work does not aim to present a foundational model but rather to introduce an algebraically inspired world-model framework of compositional and continuous actions that rivals existing schemes like Genie.

Moreover, analyzing scaling laws requires computational power and datasets of immense scale, which far exceeds our computational budget. We believe that such an extensive study would warrant a separate research project.

W5. No quantitative results on Phyre

We used Phyre primarily for qualitative demonstrations, as it does not provide labeled action-control sequences required for the quantitative validation of CIP against other methods. Therefore, we did not include numerical results.

To enhance the credibility of our framework, we have conducted an additional large-scale experiment on videos of robot actions in real-world environments. Please refer to our general response G1 for more information.

Thank you very much for your thoughtful feedback and for reconsidering the evaluation.

Regarding your question about the Lie group structure in our framework:

The fact that g depends on a specific trajectory x(t) prevents g from being a proper global transition operator. The paper is a bit fuzzy about how the Lie group structure embeds into the proposed framework. So do we have a Lie group per trajectory? Or a global Lie group?

We apologize for any confusion in the original manuscript. We assume there is a common (global) Lie group $G$ that acts on the observation space $\mathcal{X}$, and that it can realize any transition in the observed time series across all environments through group action.

To formalize this, consider a dataset of trajectories $\lbrace x_i(t) \mid t \in I \rbrace_i$ from some environment $\mathcal{E}$. Then for each $i$, we assume that there exists a set of Lie group elements $\lbrace g_{i, t, \delta} \mid t \in I \rbrace \subset G$ that can realize the transitions of $x_i$ through the group action, as follows:

$$x_i(t+\delta) = g_{i, t, \delta} \cdot x_i(t), \quad \text{so that} \quad x_i(t) = g_{i, 0, t} \cdot x_i(0).$$

Here, we emphasize that $g \in G$ acts on the observation space $\mathcal{X}$ via a group action, mapping any state $x$ to $g \cdot x$. We assume that this holds for all trajectories $x_i$ in all environments including $\mathcal{E}$, so the Lie group $G$ is common to all trajectories and environments in our framework.

As we notified all reviewers, we also made sure to clarify this point in the revised section 3 and in Appendix.

Additional remarks:

We would also like to reiterate our response to Reviewer em9c regarding our use of Lie group action. While the elements of $G$ are invertible (reversible), we would like to emphasize that our framework can model non-reversible action controls as well because the group elements $g(t)$ are time-dependent, and because our controller interface takes into account both the history of action signals $a[0, t)$ and states $x[0, t)$ to output an operator $g(t)$.

For example, we can train our controller so that, at a certain state in a certain environment at a given time $t$, no action input can cause the controller to output the inverse $g(t-1)^{-1}$ of the previous transition operator $g(t-1)$. This situation essentially demonstrates an example of an "action input that cannot be undone," and it is fully within the scope of our theoretical framework and the design of our controller.

We note that our model without the controller component (CIP) can be thought of as a "debug room" where any action can be reversed. However, having such reversibility in the latent space does not impede our ability to model irreversible phenomena. As shown in the video included with our submission, we have demonstrated that we can model phenomena involving diverse types of transitions.

Addressing any additional concerns:

They address some (yet not all) of my concerns.

We appreciate your acknowledgment that some of your concerns have been addressed. If there are any remaining issues or questions, we are fully committed to resolving them. We would greatly appreciate it if you could specify any additional points you'd like us to clarify.

Thank you once again for your valuable feedback, which has helped us improve the clarity and quality of our work.

To address your concern about the significance (statistical test / error bar), we conducted additional experiments by re-running our tests with five different random seeds. We calculated the mean and standard deviation for each metric. As you can see from the standard deviations, the variations are quite small, indicating the stability of our results. These experiments were conducted under the same settings as those in Table 2 of our paper.

We hope these additional results address your concern. Please let us know if you have any further questions or suggestions.