Evolving Minds: Logic-Informed Inference from Temporal Action Patterns

We sincerely thank Reviewer BKJo for the insightful analysis and recognition of our work! We hope our responses listed below can address your concerns.

$\star$ Examples Revealing Newly Uncovered Rules: We have reported a subset of temporal logic rules that have been identified as having real-world significance based on real-world datasets, and due to the page limitation, we put the results in the Appendix D.4, Table 17. These rules are learned by our model which are not provided as prior knowledge and easily overlooked. For the reviewer’s convenience, below we analyze one of the learned rules as a concrete example, explaining both its formal interpretation and real-world significance:

$PickUp \leftarrow WantToPickUp\ \wedge\ MoveTo\ \wedge\ Open, (WantToPickU\ before\ MoveTo), (MoveTo\ before\ Open), (Open\ before\ PickUp)$

This rule is learned from the Hand-Me-That dataset, capturing a sequential intention-action pattern: the user first develops an intention to pick up an object ($WantToPickUp$), then moves toward a destination ($MoveTo$), and opens the container ($Open$), ultimately leading to the target action ($PickUp$).

The practical value lies in using this learned rule as guidance for mental state inference. When our model detects the user's WantToPickUp intention, the AI agent can proactively assist -- for instance, by opening the container in anticipation of the user's need. This inference and prediction capability can enhance human-agent interaction efficiency.

$\star$ Discussion on Limited/Few Mental States: In real-world settings, mental states are typically high-level and inherently sparse within specific contexts. Our model operates under a closed-world assumption, focusing on constrained datasets tailored for specific tasks. As a result, the mental states in our experiments are confined to a predefined domain-relevant set—justifying our consideration of only a limited number of mental states for real-world applications. Encouragingly, our model has the potential to adapt to scenarios with unknown or numerous mental states. Inspired by [1], which uses vision-language models for predicate invention, we can also leverage LLMs to generate diverse mental states, circumventing the limitations of pre-defined states while producing arbitrarily numerous variations, before proceeding to rule learning and inference.

[1] Liang, Y., Kumar, N., Tang, H., Weller, A., Tenenbaum, J. B., Silver, T., ... & Ellis, K. Visualpredicator: Learning abstract world models with neuro-symbolic predicates for robot planning. ICLR 2025.

$\star$ Training Time & Computing Infrastructure: We have reported the training time for both synthetic datasets and real-world datasets as well as the computing infrastructure in our previous submission, and due to the page limitation, we put the results in the Appendix. Below we provide the section indexes for reviewer's convenience. The training time for varying sample sizes and the number of ground truth rules have been shown in Appendix D.1, for both synthetic datasets (Figure 5) and real-world datasets (Figure 6). Additionally, we have compared the impact of different hyper-parameter choices on training time and model performance in Appendix E.1. Details of the computational resources and computing infrastructure used in our experiments were provided in Appendix E.2.

$\star$ Open-Source: Yes, we will release the codes for the final version.

$\star$ Correction for Typos: Thanks for pointing it out! We have updated the legend in Figure 3 and made corresponding revisions to the final manuscript.

Thank you for your positive feedback!

Regarding your question:

"One question regarding autonomous rule learning for data-rich domains: in Table 17, the elements of temporal logic rules, such as PickUp and WantToPickUp, are still predefined, right?"

Yes, in our current framework, predicates (i.e., "elements of temporal logic rules" as shown in Appendix D.4, Table 17) are drawn from a predefined candidate pool. This reflects a closed-world assumption, where all possible actions and mental states are specified in advance.

However, our approach can be extended to an open-world setting by incorporating predicate invention mechanisms. Inspired by prior work [1] on using vision-language models for predicate generation, we can leverage LLMs to handle novel situations—first prompting them to generate diverse candidate actions and mental states, and then integrating these into the rule-learning and inference process.

[1] Liang, Y., Kumar, N., Tang, H., Weller, A., Tenenbaum, J. B., Silver, T., ... & Ellis, K. Visualpredicator: Learning abstract world models with neuro-symbolic predicates for robot planning. ICLR 2025.

We thank Reviewer w7nu for the detailed analysis and insightful comments, which benefit us to further improve our paper! To address your concerns, we have prepared a detailed point-by-point response below.

$\star$ Scalability: Your comments are very valuable! Yes. We acknowledge that our original statement was imprecise. A more accurate statement would be: "Our model demonstrates potential for scaling to large datasets."

Compared with related methods, regarding our rule learning module, existing related approaches like TELLER [1] and CLNN [2] require only 600-2,400 training synthetic sequences. While event prediction models with latent states like AVAE [3] use up to 2,000 sequences. In comparison, our reported experiments utilize relatively large sample sizes.

Moreover, our fine-grained model's computational complexity stems from three key factors:

(1) The inherently combinatorial nature of rule learning.

(2) Temporal discretization requiring inference at each tiny time grid for accurate continuous-time mental event inference.

(3) Event density per sequence. In our scalability experiments, the most complex synthetic dataset comprises 5,000 sequences, with an average of 47.60 action events per sequence—totaling 238,000 events. This represents a relatively large-scale dataset.

Therefore, scalability assessment must consider not just sample size, but also rule learning complexity, temporal resolution, and event density -- making our 5,000-sample experiments relatively large-scale under these demanding conditions.

Considering the above key factors influencing scalability, we have added new experiments using Syn Data-2 and larger sample size to further assess the potential of our model. Please find Table 1 in

https://anonymous.4open.science/r/paper9060-F13F

These new experimental results show that the model performance exhibits asymptotic stabilization with increasing dataset size, and our model still has potential to perform well in 10K+ sequences scale dataset within satisfactory training time cost.

[1] Li, S., Feng, M., Wang, L., Essofi, A., Cao, Y., Yan, J., & Song, L. Explaining point processes by learning interpretable temporal logic rules. ICLR 2021.

[2] Yan, R., Wen, Y., Bhattacharjya, D., Luss, R., Ma, T., Fokoue, A., and Julius, A. A. Weighted clock logic point process. ICLR 2023.

[3] Mehrasa, N., Jyothi, A. A., Durand, T., He, J., Sigal, L., and Mori, G. A variational auto-encoder model for stochastic point processes. CVPR 2019.

$\star$ Illustrative Examples: Thanks for your advice! We will incorporate additional illustrative examples in subsequent revisions to enhance comprehension. Following your suggestion regarding the distinction between action and mental events, we added example as:

“Consider a person who intends to start exercising and later maintains regular strength training at the gym. This behavioral sequence involves: (1) a mental event (the unobservable intention to exercise) and (2) action events (the observable gym workouts). Crucially, mental events are internal and unobservable, while actions are external and directly observable—constituting what we actually perceive.”

$\star$ Unfamiliar Situation: Our rule-generation algorithm is inherently capable of learning logic rules from scratch, even in scenarios with limited prior knowledge, making it highly adaptable to unfamiliar situations. In practice, when encountering such scenarios, our system operates in two modes: If pre-trained on similar datasets, it directly transfers the well-trained model and takes the learned rules as priors to the new data; Otherwise, it autonomously learns new rules without requiring predefined logic rules, with the built-in backtracking mechanism further improving accuracy.

We have assessed our method under two distinct unfamiliar conditions: transfer adaptation (for domain-related cases) and fully unfamiliar application (for entirely unseen datasets), with comparative results presented in Table 2 in

https://anonymous.4open.science/r/paper9060-F13F

These new experimental results confirm our model maintains robust performance in fully unseen scenarios. When pre-trained on similar datasets, the learned rules boost both training efficiency and final model performance.

$\star$ Correction for Typos: Thanks for your careful review! All typos have been corrected and will be updated into the final version.