Multi-step Visual Reasoning with Visual Tokens Scaling and Verification

Overall Response

We appreciate the thoughtful feedback and valuable suggestions. In response, we clarified our definition of interpretability, highlighted how our structured tool-use traces offer stepwise, visually grounded transparency, and provided empirical evidence supporting this claim. We also clarified how our verifier differs from prior reflective mechanisms, distinguished the impact of the VTS-V framework from the dataset, and added ablations isolating the reasoner’s contribution. All corresponding updates and clarifications will be incorporated into the final version if the paper is accepted.

Response to W1: Evaluation of the Claim “More Interpretable Reasoning Traces”

We thank the reviewer for raising this important question regarding the interpretability of reasoning traces generated by our model. We clarify below what we mean by interpretability in the context of our method and why it is supported by the proposed framework and results.

1. Nature of Interpretability in Our Setting: Traditional VLMs perform latent, one-shot image encoding and generate answers based on static visual token embeddings. In contrast, our model produces multi-turn reasoning traces composed of explicit tool-use steps:

• Each turn consists of a verbalized subgoal, an invoked tool, and a visual feedback observation.
• These intermediate outputs are grounded in the actual visual content (e.g., cropped regions, OCR text, depth maps), providing a transparent reasoning trajectory.

This tool-augmented, structured output allows both humans and verifiers to inspect the decision process at each step. As a result, the model’s internal reasoning becomes observable and interpretable, rather than remaining a latent process.

2. Examples Beyond Dataset Construction: In addition to the trace examples in our constructed VTS dataset, the trained models retain and generate similar multi-step reasoning traces during inference. These traces are observable in Figure 1 and in Appendix C.3, where models produce dynamic reasoning involving tool selection and verification-guided iteration. Each generation includes a full trajectory with tool actions and outputs.

3. Empirical Support via Multi-Step Trace Lengths and Tool Diversity: Across multiple benchmarks (e.g., BLINK, V*Bench), our models often generate longer and more structured reasoning trajectories compared to vanilla baselines. For instance, average reasoning steps increase from ~1.5 (GPT-4o baseline) to ~3.2 (ours with VTS-V), and tool diversity is observed in the trace logs, including frequent use of grounding, depth, OCR and cropping actions.

4. Future Work on Manual Evaluations: We agree that a manual or crowdworker-based evaluation of reasoning quality (e.g., alignment between trace and final answer) would further strengthen this claim. We plan to include such analysis in future work, but our current framework already outputs verifiable, human-readable sub-decisions, which is a step beyond traditional black-box VLM inference.

We will revise our manuscript to clarify this distinction and to emphasize that the interpretability claim refers to the explicit, grounded, and stepwise trace format enabled by our model, rather than implying subjective human quality judgments alone.

Response to W2: Relation to Existing Feedback-Based Reflective Mechanisms

We thank the reviewer for pointing out the need to better clarify how our verifier differs from existing reflective or feedback-based mechanisms. We agree that our original related work discussion could have better distinguished the motivation and contributions of our verifier design.

While prior works such as V-STaR and step-level math verifiers adopt reflective paradigms for single-domain problems (e.g., mathematical reasoning), our verifier is designed for multi-modal, tool-augmented, and multi-step visual reasoning. Most existing verifiers operate in textual domains, generate scalar scores or binary accept/reject signals, and are trained with task-specific preference data.

Our method differs in the following key aspects:

• We design a verifier that operates over multi-modal reasoning trajectories involving visual tools and structured actions, rather than purely textual rationales.
• We train the verifier using multi-step preference comparisons derived from our large-scale and diverse VTS-DPO dataset, which is constructed from the LLaVA-OneVision corpus and covers over 80 real-world task types.
• Our verifier is not just a reward model but actively controls reasoning flow during inference, enabling early stopping and action rejection.

We will revise our related work section accordingly to better reflect these distinctions.

Response to Q2: Clarifying the Contribution of the Framework vs. the Dataset

Thanks for the thoughtful question. We agree that distinguishing between the impact of the framework (VTS-V) and the curated dataset (VTS-SFT) is important.

Our experimental results (see table below, excerpted from Table 1) demonstrates that the framework alone leads to substantial improvements when applied to a capable model like GPT-4o without any fine-tuning:

This indicates that the VTS-V framework itself is effective, assuming the base model can understand multi-turn instructions and tool-augmented reasoning workflows.

In Table 2, applying the same framework to open-source models without fine-tuning leads to performance degradation. This is not a limitation of the framework, but rather a reflection of the lack of tool-use priors and alignment tuning in these models. Specifically:

• Open-source models have no pretraining exposure to structured reasoning formats like: Plan → Action → Tool → Observation → Continue/Stop.
• They often misinterpret verifier signals or fail to produce valid tool-use outputs without explicit instruction.

By contrast, GPT-4o benefits from strong instruction-following and alignment capabilities, allowing it to integrate seamlessly with VTS-V.

After fine-tuning open-source models with VTS-SFT, they learn to properly use the framework and outperform their original baselines (e.g., Qwen2VL-7B improves from 48.01% to 52.19%).

We will clarify this distinction in the revision.

Response to Q3: Ablation Without the Reasoner

Thanks for the suggestion. To directly evaluate the contribution of the reasoner module, we compare models with:

• w/o Reasoner: The base model uses verifier guidance but no trained reasoner (tool actions are generated directly).
• w/ Reasoner: Our full VTS-V framework, with both trained reasoner and verifier.

We report the results below:

The results demonstrate that incorporating the trained reasoner significantly improves performance across all benchmarks. Without the reasoner (Verifier-only), models struggle to perform reliable multi-step reasoning—especially on complex tasks like MathVista. Adding the reasoner consistently leads to large gains (e.g., +15.72 on MathVista for Qwen2.5VL-7B), highlighting its importance in executing valid tool-use actions and following verifier guidance effectively.

We will highlight this finding more clearly in the revised version.

Response to AQ1 & AQ2: Framework Visualization and Appendix Formatting

Thanks for the feedback. We have added improved visualizations of the framework and corrected the formatting issues in the appendix. These updates will be reflected in the revised version of the paper.

Dear Reviewer,

Could you please check if the authors’ rebuttal adequately addresses your concerns? If so, kindly acknowledge the rebuttal and provide any additional comments. If not, it would be greatly appreciated it if you could engage in a discussion with the authors. Your input at this stage is essential to the review process. Thank you very much for your time and effort!

AC

Dear Reviewer,

Please remember to submitting the Mandatory Acknowledgement. Thank you!

AC

Overall Response

We appreciate the insightful feedback and detailed questions. In response, we clarified key conceptual differences from traditional RL and dynamic attention methods, emphasizing our verifier-guided, multi-step MDP formulation and inference-time visual token scaling. We also updated Figures 1 and 2 for clarity, and clarified the respective roles of our framework and fine-tuning, supported by both GPT-4o and open-source model results. These clarifications and improvements will be included in the revised version.

Response to W1: Clarity of Figures 1 and 2

Thanks for the feedback. We have improved the clarity of Figures 1 and 2, and the revised versions will appear in the updated paper.

Response to W2: Differences from Traditional Multi-Round Reinforcement Learning

Thanks for the thoughtful question. While our method is formally cast as a Markov Decision Process (MDP), it departs significantly from traditional multi-round reinforcement learning (RL) paradigms in both its state formulation and guidance mechanism.

1. Dynamic Visual Token Scaling at Inference Time: Traditional RL agents operate over fixed state encodings (e.g., static image tokens). In contrast, our framework performs inference-time visual token scaling, allowing the model to dynamically generate new visual tokens through actions such as zooming, OCR, grounding, and depth estimation.

• This enables iterative refinement of visual context, rather than reasoning over a one-shot encoding.
• Actions are selected conditionally based on prior steps, supporting context-aware visual exploration.

2. Verifier-Guided Reasoning via Multi-Step DPO (Not Reward Maximization): Instead of using scalar rewards to optimize policy, we guide reasoning using a multi-step DPO-trained verifier, which evaluates entire reasoning trajectories based on preference comparisons.

• The verifier provides step-wise utility signals, assessing both intermediate tool actions and final outputs.
• This enables fine-grained control and inference-time termination based on theoretical bounds (Theorem 3.2), which is typically lacking in RL agents prone to over-exploration or loops.

We will clarify these distinctions in the revised draft.

Response to W3: Comparison with Prior Work on Dynamic Visual Attention

Thanks for raising this important point. We agree that dynamic visual perception is a growing area, and we appreciate the opportunity to clarify how our framework differs from prior methods like LMEye, InstructBLIP, and V*.

Our proposed framework introduces a general, verifier-guided inference-time reasoning process built on a Markov Decision Process (MDP). It enables multi-step refinement of visual understanding via a flexible set of visual tools—something that existing approaches do not fully support.

Comparison with Prior Work:

• LMEye allows LLMs to request new visual evidence, but lacks a structured action space and termination control. In contrast, we model perception as MDP-driven tool selection and pair it with a multi-step DPO-trained verifier that decides when to stop.
• InstructBLIP enhances initial visual encoding using instruction tuning, but performs single-pass perception. Our approach allows iterative, adaptive reasoning during inference, where the model can invoke tools like OCR, zoom, or grounding based on ongoing context.
• V* performs recursive visual search, mainly for locating objects in high-res images. Our framework is more general-purpose, supporting a wider range of tasks (e.g., math, OCR, charts), and termination is based on verifier-guided reasoning quality, not a binary search objective.

Our Core Innovations:

1. Inference-Time Visual Token Scaling – Dynamically adjusts which image regions are processed in more detail via tool-use, not fixed at input time.
2. MDP-Based Multi-Step Reasoning – Each state-action step is formally defined, supporting structured perception and tool-based decision-making.
3. Verifier-Guided Termination – A learned verifier trained via multi-step DPO provides step-wise feedback and enforces bounded reasoning (Theorem 3.2), improving efficiency and robustness.

We will update the manuscript to explicitly include this comparative discussion.

Response to W4: Clarifying Dataset-Wise Improvements and the Role of SFT

Thanks for pointing this out. We agree that some datasets show more modest improvements, and we welcome the opportunity to clarify the role of our framework vs. fine-tuning in these results.

As shown in Table 1, applying the VTS-V framework to GPT-4o—without any fine-tuning—already results in substantial gains:

This demonstrates that the framework itself is effective, particularly when the base model (like GPT-4o) is well-aligned and capable of interpreting multi-step instructions and tool-use signals.

In Table 2, open-source models (e.g., Qwen2-VL, LLaMA3.2) show degraded performance when using VTS-V without fine-tuning. This is not due to the framework’s limitations, but rather:

• These models lack tool-use priors and have no exposure to structured reasoning loops (e.g., Plan → Action → Tool → Feedback → Stop/Continue).
• They also lack the instruction-following robustness of GPT-4o and thus cannot properly interpret verifier feedback or generate well-formed tool commands.

Once fine-tuned with our VTS-SFT dataset, these models learn to align with verifier signals, producing valid reasoning trajectories and improving beyond their baselines (e.g., Qwen2VL-7B improves from 48.01% to 52.19%).

We will clarify this distinction in the revision.

Dear Reviewer,

Could you please check if the authors’ rebuttal adequately addresses your concerns? If so, kindly acknowledge the rebuttal and provide any additional comments. If not, it would be greatly appreciated it if you could engage in a discussion with the authors. Your input at this stage is essential to the review process. Thank you very much for your time and effort!

AC

Dear Reviewer CE1C,

Your expert feedback is crucial to refining this work. While we fully understand the discussion period may pose challenges for your schedule, we would value the chance to clarify any final points with you prior to its conclusion on Aug 8.

We hope we've been able to address your questions and concerns so far. We would be glad to address any further concerns you may have, and we will try our best to clarify promptly.

Thank you again for your feedback and comments; they were really helpful!

Warm Regards, Authors of Submission #9056

Dear Reviewer,

Please remember to submitting the Mandatory Acknowledgement. Thank you!

AC

Overall Response

We appreciate the thoughtful and constructive feedback! In response, we clarified the novelty of our verifier-guided MDP framework beyond prior work, provided visual token scaling analyses, and demonstrated strong generalization of the verifier across unseen domains. We also addressed design choices such as reasoner-verifier separation, the use of SFT over RL, and potential future directions like self-bootstrapping. These clarifications and results will be incorporated into the final version if the paper is accepted.

Response to W1: Novelty Beyond Combining Prior Work

We appreciate this thoughtful feedback. While our work draws inspiration from prior efforts in iterative reasoning [1], reasoner-verifier separation [2], and tool use [3], we believe our contributions go beyond a simple combination. Specifically, we introduce a principled, theoretically-grounded framework tailored for multi-step visual reasoning with tool use, supported by both new methodology and new data.

Key distinctions:

• MDP-based Reasoning Framework: Unlike V*[1], which focuses on task-specific cropping, we formulate reasoning as a Markov Decision Process over a diverse toolset (OCR, grounding, depth, etc.). This allows for flexible, dynamic visual token scaling across tasks and models.

• Verifier for Guided Inference with Theoretical Guarantees: Compared to V-STaR [2], which ranks final answers, our verifier operates at each step, providing real-time guidance and bounded termination (Theorem 3.2), enabling interpretable and efficient inference.

• New Dataset for Multi-Step Visual Tool Use: Prior tool-use works [3] do not support multi-turn visual reasoning. We contribute the VTS-SFT and VTS-DPO datasets, the first to support fine-grained tool selection and verifier training over >80 diverse tasks.

Together, our framework enables strong generalization (e.g., +6.9% on GPT-4o without fine-tuning) and interpretability, establishing a solid foundation for future multi-modal reasoning systems. We will revise the manuscript to more clearly highlight these distinctions.

Response to W2: Visual Token Scaling Mechanism with Optimal-Stopping Verifier

We sincerely appreciate the reviewer’s insightful feedback regarding the need to clarify the visual token scaling mechanism. Below, we present selected experimental results demonstrating the scaling relationship between multi-step reasoning performance and output visual tokens, which reflects our framework’s dynamic behavior.

• Results based on closed-source model APIs:

• Results for the trained open-source models:

The tables above illustrate the relationship between reasoning accuracy and visual token usage for both GPT-4o and our trained models. On average, GPT-4o and our trained models generate approximately 50 and 100 text tokens per reasoning step, respectively, while processing ~100 visual tokens through dynamic image operations. We control the number of visual tokens by limiting the reasoner’s maximum steps, while our verifier selects the optimal stopping point (marked by *).

Key findings:

• Non-monotonic scaling: Accuracy improves with more visual tokens initially, but may decline if reasoning becomes unnecessarily long.
• Verifier-guided optimality: Our VTS-V verifier consistently identifies the best stopping point, balancing depth and efficiency.
• Robust reasoning: Even beyond the optimal step, accuracy remains higher than with under-reasoning, highlighting the benefits of sufficient visual grounding.

We will incorporate these scaling trends and supporting visualizations into the revised manuscript.

Response to W3: Robustness and Generalization of the Verifier Trained on VTS-DPO

We thank the reviewer for raising concerns regarding the verifier’s dependence on the VTS-DPO dataset and its robustness to unseen domains. We provide further clarification below.

The VTS-DPO dataset is constructed based on the LLaVA-OneVision (LLaVA-OV) dataset, which is currently among the most comprehensive multi-modal corpora, covering over 80 distinct vision-language task types. Our DPO dataset comprises 301,028 preference pairs, carefully curated to span a wide range of visual reasoning domains. These include document understanding, scientific diagrams, mathematical problem solving, chart QA, OCR, and general commonsense VQA.

The dataset is divided into the following major categories:

Each category includes examples from highly varied sources.

Notably, all benchmarks used for evaluation in our experiments (e.g., BLINK, V*Bench, MMStar, MathVista) are held-out from the training data. The verifier is therefore evaluated on unseen reasoning tasks and domains. The strong performance observed across these evaluations supports the verifier’s ability to generalize beyond the training distribution.

We will expand on the construction and coverage of VTS-DPO in the revised Appendix, including a full breakdown of dataset sources and domains.

Response to Q1: Why not use Reinforcement Learning (e.g., GRPO) to Train the Reasoner?

We appreciate the reviewer’s suggestion to explore GRPO for training the reasoner. We have conducted preliminary experiments and found that GRPO performs poorly in our multi-tool setting, where the reasoning space is large and tools are diverse. While prior works using GRPO (e.g., DeepEyes [1]) focus on limited-tool environments.

To address this, we adopt supervised fine-tuning (SFT) as a more stable and effective initialization strategy. Our VTS-SFT dataset serves as a strong cold-start foundation for future RL-based methods. While GRPO is not the focus of this paper, we are actively exploring its potential in future work with improved reward design and tool selection policies.

[1] Zheng Z, Yang M, Hong J, et al. DeepEyes: Incentivizing “Thinking with Images” via Reinforcement Learning. arXiv preprint arXiv:2505.14362, 2025.

Response to Q2: Why Separate the Reasoner and Verifier Instead of Using a Single Unified Model?

We thank the reviewer for this thoughtful question. In principle, combining reasoning and verification into a single model is an appealing idea for potential efficiency. However, based on both theoretical motivations and empirical observations, we find that the decoupled design is more effective in practice, especially in the context of multi-modal, tool-augmented reasoning.

1. Current Models Struggle to Jointly Handle Both Tasks Well: Reasoning and verification represent distinct capabilities: the reasoner must generate tool-using action plans conditioned on complex context, while the verifier needs to evaluate the overall quality or utility of each reasoning step. In practice, we observe that when both responsibilities are handled by a single model, the performance on one or both tasks deteriorates.

2. GPT-4o Results Support the Effectiveness of Separation: Our experimental results (see table below, excerpted from Table 1) show that adding a standalone verifier to GPT-4o leads to significant performance improvements:

This demonstrates that adding a verifier as a separate module to an already-strong base model (GPT-4o) yields substantial gains, especially on tasks that require structured multi-step reasoning (e.g., +18.33 on Counting, +12.91 on Functionally-Correlated tasks).

3. Modularity and Reusability of Verifier: Our verifier can be viewed as a plug-in component: it is model-agnostic and can be reused across architectures (e.g., GPT-4o, Qwen, LLaMA) once trained. This improves flexibility, reduces fine-tuning costs, and makes the framework extensible to new tasks or domains.

We will clarify this design decision in the final version of the paper.

Response to Q3: Iterative Bootstrapping of VTS via Self-Teaching

We appreciate the reviewer’s insightful suggestion.

Using the trained VTS-V pipeline to construct additional VTS-style data in a self-bootstrapping fashion is indeed an interesting and promising direction. It aligns well with recent trends in self-improving agents and iterative data refinement.

One potential challenge, however, lies in maintaining supervision quality during self-generated data collection. Our VTS-SFT and VTS-DPO datasets rely on high-quality reasoning trajectories and carefully curated preference signals. If the model-generated traces contain subtle reasoning errors or invalid tool calls, these could accumulate and introduce bias or instability during future training rounds.

That said, combining model-generated traces with selective human or verifier-based filtering may offer a scalable way forward. We see this as an exciting avenue for future work and will consider incorporating this direction into our ongoing research agenda.

Dear Reviewer,

Could you please check if the authors’ rebuttal adequately addresses your concerns? If so, kindly acknowledge the rebuttal and provide any additional comments. If not, it would be greatly appreciated it if you could engage in a discussion with the authors. Your input at this stage is essential to the review process. Thank you very much for your time and effort!

AC

Overall Response

Thanks for the detailed and thoughtful feedback. In response, we provided a quantitative analysis of inference-time latency and memory overhead, demonstrating the efficiency–accuracy tradeoffs of our verifier-guided framework. We also expanded GPT-4o evaluations across V*Bench, MMStar, and MathVista, showing strong gains over both baselines and alternative prompting methods. Finally, we clarified why our framework improves GPT-4o without fine-tuning but requires VTS-SFT to enable open-source models to follow structured, tool-based reasoning. All clarifications and new results will be incorporated into the revised version if the paper is accepted.

Response to Weakness 1: Inference-time Overhead (Latency & Memory Usage)

We appreciate the reviewer’s thoughtful concern regarding inference complexity. To address this, we now provide a detailed quantitative analysis of inference-time overhead, including latency, memory usage, and the number of intermediate tokens generated during dynamic reasoning.

Our method introduces iterative steps composed of:

• Verifier calls, which require only prefill computation (i.e., forward pass over the input context to generate a stop/continue decision),
• Reasoner calls, which involve autoregressive decoding to produce tool instructions.

Each interaction is structured as:

context = input_prompt: Q (image, text_question)
while verifier(context) == 'continue':
	current_output = reasoner(context)
	physical_feedback = tool_using_API(current_output)

We define the total inference latency as:

$\text{Latency} = \sum\_{i=1}^{H} \left( \frac{C\_i}{P\_t} + \frac{O\_i}{D\_t} \right)$

Where:

• $H$: total number of reasoning steps;
• $C\_i$: number of input tokens at step i (verifier prefill);
• $O\_i$: number of generated tokens at step i (reasoner decode);
• $P\_t$: model’s prefill throughput (tokens/sec);
• $D\_t$: model’s decode throughput (tokens/sec).

Example Overhead (Qwen2.5-VL-7B-Instruct with VTS-V):

• Average steps (H): 3.1
• Average verifier prefill tokens: ~10058.0 per step
• Average reasoner decode tokens: ~95.3 per step
• Throughput on 80GB A800 GPUs (float16):
  • $P\_t \approx$ 18493.6 tokens/s
  • $D\_t \approx$ 231.8 tokens/s

Total latency ≈ 3.1 × (10058.0 / 18493.6 + 95.3 / 231.8) ≈ 2.96 seconds

Memory Usage:

• With max_model_len=65536 and dtype=float16, our models can be deployed within 32GB through vLLM or SGLang on 80GB A800 GPUs.
• All tool models (including GroundingDINO, Depth-Anything, etc.) collectively require approximately 23 GB of additional memory during inference.

Tradeoffs and Mitigations:

• We find a clear tradeoff between reasoning depth and computational cost, which is justified by significant accuracy gains (e.g., +7.58% avg. on BLINK for Qwen2.5-VL-7B-Instruct).
• To mitigate latency for deployment, our framework allows parallel verifier batching, early stopping, and adaptive step truncation (via ε-threshold).

We will include this analysis, along with latency and memory tables across models, in the revised Appendix B. Thank you again for the helpful suggestion.

Response to W2: GPT-4o Evaluation on Broader Benchmarks

We sincerely appreciate the reviewer’s insightful feedback regarding the evaluation of our VTS-V framework on additional benchmarks for GPT-4o. We agree that including GPT-4o results on V*Bench, MMStar, and MathVista would better demonstrate the framework’s generalization across diverse reasoning tasks.

To address this, we conducted additional evaluation experiments, including GPT-4o, GPT-4o + CoT[1] and GPT-4o + Visual Sketchpad [2] as baselines. Our results are summarized below:

Key observations:

• Our VTS-V achieved state-of-the-art performance on both V*Bench and MMStar, with absolute improvements of 13.09% and 7.4% over the baseline GPT-4o, respectively, demonstrating the efficiency of our approach.
• Our method outperformed GPT-4o by 2.1% on MathVista. And it achieved nearly equivalent performance to the top-performing GPT-4o+CoT (a mere 0.6% difference), further demonstrating the competitiveness of our approach.

[1] Wei J, Wang X, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[J]. Advances in neural information processing systems, 2022, 35: 24824-24837.

[2] Hu Y, Shi W, Fu X, et al. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models[J]. Advances in Neural Information Processing Systems, 2024, 37: 139348-139379.

Response to W3: Performance Gap Between GPT-4o and Open-Source Models Without Fine-Tuning when integrating VTS-V

We thank the reviewer for raising this important point. While Section 5.2 (lines 306–317) of the main paper briefly mentions this observation, we now provide a more thorough explanation of why VTS-V improves performance in GPT-4o even without fine-tuning, but degrades performance in open-source models like Qwen2-VL and LLaMA-3.2-Vision.

Applying VTS-V directly to GPT-4o yields consistent improvements across benchmarks without any additional training. In contrast, applying the same framework to open-source models without fine-tuning leads to degraded performance.

We hypothesize that this performance discrepancy stems from the following factors:

1. Lack of Tool-Using Priors in Open-Source Models. Unlike GPT-4o, open-source models are not pretrained or aligned with tool-augmented workflows. They have no exposure to structured reasoning formats like: Plan → Action → Tool call → Observation → Continue/Stop. As a result, when presented with verifier signals and expected to issue tool-use commands, these models either ignore the prompt or generate malformed instructions.

2. Robust Instruction Following in GPT-4o. GPT-4o has undergone extensive alignment tuning (e.g., RLHF, tool-use demonstrations) which enables it to robustly interpret multi-turn instructions and unfamiliar prompting formats. This generalization allows it to cooperate with our verifier module out of the box, without dedicated SFT.

3. Verifier Signal Interpretation. Verifier guidance in our framework (e.g., continue vs. stop) depends on implicit task understanding and adaptive response. GPT-4o can interpret these signals semantically, while open-source models require fine-tuning to learn how to react to verifier outputs correctly.

4. Empirical Confirmation via SFT. After supervised fine-tuning on our VTS-SFT dataset, open-source models learn valid tool-use behaviors and align well with verifier guidance. Their performance not only recovers but exceeds the original baseline (e.g., Qwen2VL-7B-Instruct improves from 48.01% to 52.19%).

We will clarify this distinction and analysis in the revision. This behavior gap also underscores the importance of our VTS-SFT dataset in enabling open-source models to benefit from verifier-guided multi-step reasoning workflows.

Dear Reviewer,

Could you please check if the authors’ rebuttal adequately addresses your concerns? If so, kindly acknowledge the rebuttal and provide any additional comments. If not, it would be greatly appreciated it if you could engage in a discussion with the authors. Your input at this stage is essential to the review process. Thank you very much for your time and effort!

AC

Dear Reviewer,

Please remember to submitting the Mandatory Acknowledgement. Thank you!

AC