Thank you for your helpful and constructive reviews. We respond to your concerns and questions below.

Claims And Evidence

The term "robustness" refers to the internal consistency of our task design. The GridToM dataset is generated via a fully automated pipeline with systematically controlled scenarios.

Our human experiments are intended to confirm that the auto-generated scenarios and reasoning chains are intuitive and interpretable to humans—thus indirectly validating the method's reliability.

To avoid ambiguity, we will revise "robustness" to "consistency" for clarity.

In L331, our intention was to reference the psychological "unexpected transfer task", where false beliefs arise from unobserved changes. The weaker TB performance may be due to the model overgeneralizing from false-belief patterns and ignoring key multimodal cues.

We will revise the terminology to avoid ambiguity.

Methods And Evaluation Criteria

To our knowledge, we are the first to enhance MLLMs' ToM abilities through targeted attention direction detection and intervention. Our method is not highly sensitive to the choice of hyperparameters. The large range of the hyperparameter Alpha was selected to test the model's limits (interference beyond this range could lead to the failure of all responses).

Our method improves VLMs' ToM performance through interpretable interventions based on internal representations and input perturbations, not hyperparameter tuning. It requires no training or architecture changes, making it generalizable to other reasoning tasks.

Appendix E explores a wide hyperparameter range. We found that the effect is limited to a valid interval—outside of which responses fail. Alpha remains effective within the range of [-50, 50], while the choice of K depends on the number of hidden heads in the model and shows consistent effectiveness across all intervention tasks. Within the valid range, both hyperparameters affect model performance by no more than 10% on average.

In the revision, we will emphasize this stability and avoid confusion about performance fluctuations from extreme values.

We also demonstrate gains on both TB and FB tasks, as shown by the BOTH metric in Table 1.

Experimental Designs Or Analyses

Testees were not misled. In the video-only condition, core rules—including that closed doors block perception—were clearly explained. The removed text was narrative only and did not affect task understanding. We will clarify this in the revision.

We acknowledge a caption error in Fig. 11. It depicts a first-order belief task for both TB and FB, as correctly stated in the main text (L767–L769). Reviewer ummn also noted this.

Fig. 12 includes two captions for two second-order TB videos, as shown in Fig. 7. All videos include temporal annotations.

In L656, the timing setup refers to the presence of a timeline and corresponding timestamps for events in all scenarios. For example, whether the yellow agent's door is closed at the moment the white agent enters the red room determines whether the white agent can correctly infer the yellow agent's belief.

Fig. 12 and Fig. 13 contain textual errors in the appendix. The second question in Fig. 12 and the first in Fig. 13 should read: "Where does the white character think the yellow character thinks the white character should be?"

The mention of a green character is a typographical error limited to the appendix. The main text and experimental setup are correct. We will fix this in the final version.

Regarding "Question": Fig. 12 and Fig. 13 illustrate second-order belief reasoning, which involves inferring what one agent believes about another agent's belief. The content of the second-order belief is, by definition, the first-order belief itself. This is not an error.

As for Suppl. Sec. B, we acknowledge that some symbols were not explicitly defined. However, their meanings are clearly explained in Section 4.2:

• $y_{protagonist}$: protagonist's belief label
• $y_{participant}$: participant's belief label
• $y_{omniscient}$: omniscient's belief label

The "Belief" row shows true/false belief combinations across perspectives, essential for second-order reasoning. These tasks involve nested beliefs (e.g., what the participant believes about the protagonist's belief), forming a natural 2×2 structure, as shown in Fig. 7 and Suppl. Sec. B.2.

References Not Discussed

Benchmarks like MMBench focus on general MLLM capabilities, while our work targets ToM reasoning—nested beliefs, perspective-taking, and multi-agent cognition—which these benchmarks do not cover. Our task-specific, cognitively grounded framework includes structured annotations for probing reasoning depth. We will mention these benchmarks to highlight the distinction and complementarity of our approach.

Other Comments

Thank you for pointing out relevant related works. We will update Shapira et al. to its EACL 2024 version and add Sap et al. (EMNLP 2022) to the related work section.

We sincerely appreciate your recognition of our work and thank you for the thoughtful suggestions. We respond to your concerns and questions below.

Methods And Evaluation Criteria

Our benchmark is designed as a foundational framework, using binary labels to establish clear distinctions between correct and incorrect beliefs. Introducing a wider range of options and more comprehensive evaluation schemes is indeed a valuable direction. We believe that the evaluation of multiple options could potentially be extended using a binary classification approach, where multiple negative samples are categorized as generalized erroneous beliefs. This will be part of our future research plans.

Experimental Designs

Thank you very much for recognizing the value of using attention heads to distinguish different perspectives and enable second-order belief reasoning. We agree that this approach is highly worthy of further investigation and have planned to conduct broader model validation and more detailed analysis in our future work. At the same time, we will adopt your suggestion to retain the early results and include these preliminary experimental conclusions in the final version of the paper.

Relation To Broader Scientific Literature

We understand your point about the current paper’s insufficient connection to embodied intelligence and the field of robotics. In fact, we are also deeply interested in embodied intelligence. This work represents our initial effort to explore the Theory of Mind (ToM) capabilities of MLLMs. We plan to extend this approach to more interactive and visual scenarios in future work, aiming to build a closer bridge between ToM in MLLMs and embodied intelligence in robotics.

Essential References Not Discussed & Other Strengths And Weaknesses

Thank you for pointing out relevant related works. Although we have not discussed Verma et al. (2024) and Mao et al. (2024) in the current version, we recognize their relevance and value. We will include discussion of these papers in the revised manuscript to better position our work within the broader context of ToM research in LLMs, particularly as it relates to human-machine interaction and embodied intelligence.

Other Comments Or Suggestions

Thank you for the feedback. Our original intention was to use a vivid example to illustrate higher-order beliefs. We acknowledge that ToM benchmarks involve complexities beyond simple analogies. Following your suggestion, we will add a more detailed explanation of ToM in the revision.

Questions for Authors

1. Modality Independence:

We believe that attention-based enhancement is not strictly dependent on a specific modality but rather on carefully designed experiments. Prior work and our own results show the effectiveness of attention enhancement across both text-only and multimodal settings. Our experiments (Sup. G and H) demonstrate generalization across GridToM and MMTOM, suggesting the potential for extending this method to other modalities such as audio. Additionally, We believe attention-based enhancement depends on carefully designed experiments to reduce the impact of noise in attention signals.

2. Binary Testing in ToM Research:

ToM studies often rely on structured binary choices (e.g., true/false) to assess belief reasoning, as seen in classic tasks like the “unexpected location” and “unexpected contents” paradigms. Existing benchmarks for LLMs, such as ADV-CSFB (text-based) and MMTOM-QA (multimodal), also follow this approach. Our study continues this tradition. That said, exploring more open-ended or multi-choice formats could test reasoning flexibility in more complex settings, though this would introduce higher demands in task design and annotation. We see this as a promising direction for future research.

Once again, thank you for your kind support and constructive feedback.

Thank you for your helpful and constructive reviews. We respond to your concerns and questions below.

Methods And Evaluation Criteria

We agree that the construction of the GridToM dataset should be described in greater detail. We provide further clarification here and will include the full pipeline both in the main text and the appendix of the revised version. Our dataset is built primarily through automated generation and verification, with minimal manual annotation and thoughtful quality control. Despite the intrinsic complexity of ToM reasoning, our structured, script-driven approach ensures consistency and reliability across visual input, actions, and narratives.

1. Dataset Construction and Annotation

• Map Design: We manually created 27 distinct 10×7 maps in Excel, each with 3 rooms and unique layouts.

• Automated Checking and Rendering: map validity was verified with Python scripts (e.g., enclosed rooms, door placement). Then, using the MultiGrid library, we rendered maps with:

  • Colors: Assigned from 6 highly distinguishable colors (red, green, blue, yellow, purple, white).

  • Agent Placement: Two groups of agents were randomly placed in hallways with colors distinct from rooms; initial orientations were randomized.

  • Path Planning: Agent trajectories were generated using Breadth-first search to ensure valid, logical movement without dead ends.

• Task Generation: The combination of different variables results in 648 basic samples. For each sample, we generate both “door open” (TB) and “door closed” (FB) conditions, totaling 1296 samples (see L122–127). Second-order belief tasks follow the same structure with minor narrative adjustments.

2. Annotation and Data Quality

• Automation First: Key elements (layout, paths, doors, task type) were generated and verified via script, minimizing subjective error.

• Human Review: We manually reviewed samples for layout issues, trajectory logic, and narrative coherence.

• Staged Execution: Tasks were divided into three stages with controlled timing to ensure logical, coherent event flow.

• Controlled Variables: We used unified logic for all visual and script elements, systematically varying only key factors (room order, agent orientation, colors, door state).

3. On ToM Difficulty and Dataset Validity

• Controlled Scenarios: Carefully constrained scenes reduce noise, allowing clearer focus on ToM and multimodal reasoning.

• Scalability: Current difficulty is moderate and sufficient for analyzing belief reasoning. We plan to expand with more complex scenarios in future releases.

Experimental Designs Or Analyses

We focus on zero-shot evaluation because Theory of Mind research emphasizes testing a model’s ability to reason about others’ mental states without task-specific demonstrations. Zero-shot settings help assess whether such capabilities naturally emerge during pretraining, without overfitting to specific prompts or memorized patterns. In other words, we believe that the abilities revealed by few-shot learning pertain more to the model’s other capabilities rather than ToM abilities themselves. On the other hand, zero-shot remains a widely adopted and informative setting for assessing underlying reasoning abilities in this domain. This approach follows precedent in prior work such as [Shapira et al., 2023; Jin et al., 2024], all of which adopt zero-shot setups to evaluate ToM reasoning in LLMs or MLLMs.

As for model selection, due to the limited availability of MLLMs supporting multi-image inputs at the time, we selected two strong-performing models: LLaVA-Next-Video and Qwen2-VL. We are actively working on extending our experiments to more models and tasks.

Other Comments Or Suggestions

In the first-order tasks shown in Fig. 3, the "Omniscient" label represents the objective ground truth. Since the omniscient perspective observes the entire event sequence, the belief remains "Purple" for both TB and FB conditions.

Second-order belief reflects whether one agent correctly estimates another’s belief. Due to space limitations, Fig. 3 only presents the case where the first-order belief is false, which is why both TB and FB under the protagonist’s view are labeled as "Red." The complete set of cases, including those where the protagonist’s belief is labeled as "Purple," is detailed in Fig. 7.

We acknowledge that there is a mistake in the caption of Fig. 11. Fig. 11 illustrates a first-order belief task to both TB and FB conditions, as correctly explained in the main text (L767-769). Reviewer r21H also pointed this out. We will address this issue in the revised manuscript.