Latent Chain-of-Thought for Visual Reasoning

We thank the reviewer for their thoughtful comments and constructive suggestions. Below, we provide detailed responses to each point.

W4-1: Perhaps detailed at the beginning of section 3 with the potential inclusion of a new figure and that underlines the key fundamental strengths of the approach that you propose...; Then in Section 3.3 I see the approach in general however I think it could benefit from a bit more detail in the figures and main text. ...; One nitpick, In the intro on the second to last paragraph you mention three points that overlap with your main contributions. Could you simply include this all under one list? It may make things a little clearer.

• We will add a figure at the beginning of section 3 to clearly show our motivation.
• Figure 4 visualizes Equation 8, illustrating how BiN performs Bayesian marginalization over latent chain-of-thought trajectories Z to approximate the likelihood of the answer P(Y|X). We agree that additional detail would improve clarity, and we will revise both the figure and the accompanying explanation in the final version to better highlight the role of latent reasoning and its connection to our overall inference pipeline.
• We will revise the introduction and consolidate the overlapping points into a unified contribution list to improve clarity and readability in the final version.

W4-2: Consider LaCoT on benchmark domains targeting decision making and acting responses (e.g., E3VQA[1], MMIU[2]) that rely on some scene understanding and reasoning from world priors.

We appreciate the suggestion. We attempt to evaluate on MMIU, but find that LaCoT and our baseline models (e.g., Qwen2.5-VL) struggle to generalize due to their limited training on single-image reasoning tasks. Regarding E3VQA, the dataset was not publicly available at the time of writing, which prevented us from conducting a fair comparison.

Although we are unable to follow the reviewer's suggestion, we conducted additional experiments on three other visual understanding benchmarks: MMMU-pro, MMVet, and MME. These benchmarks cover a broad range of tasks, including visual commonsense, fine-grained recognition, and multi-choice QA.

These results demonstrate that LaCoT consistently improves performance across different domains and model scales, suggesting strong generalization beyond mathematical reasoning tasks.

Q4-1: I couldn't see where the reference rationales come from? Could you explain that? If it hasn't been included would it be worth briefly touching on this?

Each training sample in our dataset consists of a tuple {image, query, CoT, answer}, where the CoT (Chain-of-Thought) serves as the reference rationale for the answer. These rationales are generated by teacher models such as GPT-4o or Deepseek-R1, depending on the data source. We will clarify this detail in the revised version of the paper for better transparency.

Q4-2: Did you have any issues with hallucinations in your rationales during inference? Presumably these would be more problematic as the sample size is lowered in BiN?

We appreciate the reviewer’s valuable comment. We observed some hallucination issues in generated rationales when the sample size N=1, where BiN can occasionally produce incorrect or misleading reasoning steps on the MMMU dataset.

To assess this, we used one-shot reasoning as a baseline and varied the number of sampled rationales N during inference. As shown below, increasing N from 1 to 5 significantly mitigates hallucination and improves answer accuracy:

These results suggest that hallucination can indeed be more pronounced at smaller N, but BiN’s sampling strategy is effective in addressing it as N increases. We will include additional qualitative examples in the final version to illustrate this effect.

Q4-3: When doing token level reward approximation I was a bit uncertain where rewards we actually be sampled vs. interpolated from Eq. 4, can you expand on this a bit?

We compute the actual reward for every $\lambda$ steps, and apply linear interpolation to approximate the intermediate token-level rewards. This design reflects a trade-off between accuracy and efficiency: while $\lambda$=1 (i.e., no interpolation) would yield slightly better performance, it is computationally prohibitive in practice. Under our current setting, computing token-wise rewards with $\lambda$=1 takes ~50 minutes per sample due to the long visual CoT sequences. This requires ~400 hours of training for one epoch on a 3k-sample dataset using 8*80G GPUs. Thus, our interpolated approximation provides a practical and effective alternative that maintains strong performance while significantly reducing training time.

[1] Fanqing Meng, Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, et al. Mmiu: Multimodal multi-image understanding for evaluating large vision-language models. arXiv preprint arXiv:2408.02718, 2024

[2] I Lee, W Park, J Jang, M Noh, K Shim, B Shim. Towards Comprehensive Scene Understanding: Integrating First and Third-Person Views for LVLMs. arXiv preprint arXiv:2505.21955, 2025

We thank the reviewer for their thoughtful comments and constructive suggestions. Below, we provide detailed responses to each point.

W3-1: Although the model is compared with SFT and GRPO results. Results on popular inference scaling approaches such as BofN etc. should also be compared with?

We appreciate the valuable suggestion. We provide comparison results of Best-of-N (BofN) approach below. Specifically, during inference, we sample N rational-answer pairs using LaCoT. For each pair, we compute the length-normalized log-likelihood of the generated answer as reward, and then select the final answer corresponding to the highest reward. To ensure a fair comparison, we do not use any external reward model.

We set number of candidates N={5, 10} for BoN and BiN, and we report the highest score of each method.

W3-2: Is this approach the first to adopt Amortizing Variational Inference method, more comparison with related work is needed. Also comparison with existing latent space reasoning approaches?

To the best of our knowledge, our work is the first to apply amortized variational inference to latent visual reasoning with a long CoT chain, where reasoning steps are learned and inferred in a latent space conditioned on both visual and textual input.

Previous works ([1] and [2]) focus solely on the textual domain, applying these methods for latent visual CoT is nontrivial due to long and structured multi-step reasoning.

[1] Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J.E., & Tian, Y. (2024). Training Large Language Models to Reason in a Continuous Latent Space. ArXiv, abs/2412.06769.

[2] Geiping, J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B.R., Kailkhura, B., Bhatele, A., & Goldstein, T. (2025). Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. ArXiv, abs/2502.05171.

Thank the authors for their rebuttal, I keep my positive score about this work.

We thank the reviewer for their thoughtful comments and constructive suggestions. Below, we provide detailed responses to each point.

W2-1: The proposed method is evaluated only on mathematical reasoning tasks. The generalizability to other domains remains unclear.

We appreciate the suggestion. We initially followed the R1-OneVision baseline to evaluate LVLM reasoning performance on mathematical benchmarks. To further assess the generalizability of LaCoT beyond mathematical reasoning, we conducted additional experiments on three diverse visual understanding benchmarks: MMMU-pro, MMVet, and MME. These benchmarks cover a broad range of tasks, including visual commonsense, fine-grained recognition, and multi-choice QA.

These results demonstrate that LaCoT consistently improves performance across different domains and model scales, suggesting strong generalization beyond mathematical reasoning tasks.

W2-2: The proposed method requires sampling multiple rationales at inference time, which is computationally expensive. The paper can benefit from a detailed analysis of computational costs compared to baselines.

We agree that rationale sampling introduces additional inference cost, and we address this by using mini-batching (with batch size k=5) to generate N rationales in N/k forward passes. Below, we report the average per-sample inference time (reasoning + answering) and corresponding performance of different reasoning-LVLM on MathVista and MathVerse:

Where LLaVA-CoT-11B utilizes stage-wise beam search [1] at inference time.

As shown in the table, LaCoT-Qwen-7B consistently achieves stronger performance, even with modest increases in inference time. Compared to other multi-rationale baselines, LaCoT strikes a favorable balance between computational cost and reasoning reliability, thereby improving both the trustworthiness of rationales and the accuracy of final answers.

Q2-1: How sensitive is the performance to the choice of lambda for approximating reward?

We appreciate the valuable comment. We examined the sensitivity of performance to the choice of $\lambda$ in reward approximation by training with values in the range [8,32]. While the approximated rewards showed slight differences in early stages, they rapidly converged and became nearly identical after around 250 training steps, suggesting that LaCoT is relatively robust to the choice of $\lambda$ within a reasonable range.

Q2-2: What are training and inference times compared to baselines (SFT, GRPO)?

We report the total training time of fine-tuning Qwen2.5-VL using different objective functions on 3k samples, all conducted on 8×A100 (80G) GPUs:

Where N indicates the number of explored rationals of BiN.

To avoid out-of-memory (OOM) issues during GRPO and LaCoT fine-tuning, we employed DeepSpeed Stage 3 with gradient checkpointing enabled. While this setting substantially increased training time, it enabled on-policy exploration.

These results demonstrate that while LaCoT incurs additional computational cost due to exploration and token-level reasoning supervision, it remains feasible for practical training pipelines and yields notable performance gains.

Q2-3: How does the proposed reference-guided filtering compare to other exploration strategies like epsilon-greedy or entropy-based exploration?

We studied Epsilon-greedy in our early study, but we found that it does not scale well in training a reasoning LVLM. Specifically, the action space comprises the entire vocabulary (~50,000 tokens). In such a large space, most random actions are semantically nonsensical and low-reward, making exploration inefficient and often harmful. This results in high-variance gradients and slow convergence, leading to a catastrophic forgetting issue in our experiment.

Entropy-based exploration methods such as PPO rely on optimizing policy gradients to maximize expected rewards. However, these methods often struggle to align the learned policy with a target distribution, especially in complex reasoning tasks. In contrast, our approach builds on the GFlowNet framework, which is designed to sample from a target distribution via trajectory-level credit assignment, making it better suited for tasks requiring diverse and calibrated generation [2].

Our reference-guided filtering (RGFN) further enhances the stability of on-policy exploration by selectively retaining high-quality trajectories, ensuring both efficiency and robustness during training. This strategy offers a more principled and controllable alternative to stochastic exploration heuristics.

[1] Xu, G., Jin, P., Li, H., Song, Y., Sun, L., & Yuan, L. LLaVA-CoT: Let Vision Language Models Reason Step-by-Step. ICCV 2025.

[2] Hu, E.J., Jain, M., Elmoznino, E., Kaddar, Y., Lajoie, G., Bengio, Y., & Malkin, N. Amortizing intractable inference in large language models. ICLR 2024

We sincerely thank the reviewer for their thoughtful feedback and constructive suggestions. Please find our detailed responses to each point below.

W1-1: The main concern lies in the experimental section which I find is a bit thin.

We appreciate the suggestion. We conduct additional experiments on widely used benchmarks, including MMMU-pro, MMVet, and MME, where MMMU-Pro is a more robust version of MMMU, designed to assess LVLMs' understanding and reasoning capabilities more rigorously. These benchmarks cover a broad range of tasks, including visual commonsense, fine-grained recognition, and multi-choice QA.

These results demonstrate that LaCoT consistently improves performance across different domains and model scales, suggesting strong generalization beyond mathematical reasoning tasks.

W1-2: I wonder if the comparison with GRPO in Table 1 is fair, as LaCoT uses a different training set as R1-OneVision.

Our SFT data is comparable with R1-OneVision. However, in the RL stage—which we consider the most critical for performance—R1-OneVision (GRPO) utilizes 10k newly collected samples. In contrast, LaCoT reuses only 3k samples from the SFT stage without additional data collection, which highlights its efficiency and generalization.

W1-3: The performance of GRPO in Table 2 is a bit strange, as is contradictory with most recent papers observation. My empirical experience also indicates that GRPO works better than SFT especially in terms of generalization. The details of this ablation is not provided and no analysis on why this abnormal behavior happens.

In Table 2, we compare the effectiveness of different training algorithms on 3k samples, using consistent hyperparameters across all methods (e.g., same batch size, base model, and learning rate). For GRPO and RGFN (our approach), we set the exploration number to 6 and the sequence length to 700. While GRPO performs slightly worse than SFT, we think this is due to GRPO’s higher sensitivity to hyperparameters and its reliance on larger training data for stable generalization. This is also reflected in our previous experiments (R1-OneVision applies 10k training data), where its performance improves with more data.

Q1-1: Since LaCoT introduces an additional neural-based reward model, what would be the computational overhead compared to, e.g., GRPO?

LaCoT introduces no additional GPU memory usage or data preprocessing overhead compared to GRPO during training. However, due to the token-level reward approximation, it does incur a higher computational cost in terms of runtime. Under identical experimental settings, LaCoT requires approximately 90 hours of total training time, compared to 64 hours for GRPO.