How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding

We are encouraged to see that the reviewer finds our work to provide interesting insights into how visual and linguistic information interact across layers in multimodal LLMs through a straightforward and interpretable approach. We appreciate the constructive suggestions and please see below for our response to your questions.

Q: Clarification on semantic reasoning

We agree that this term can benefit from more precise definition. In our context, semantic reasoning refers to the model’s ability to connect linguistic concepts in the prompt (e.g., the noun “animal”) with visual content (e.g., an image of a dog), and to derive a meaningful answer from that alignment. We will clarify this along with other terminologies in the revised manuscript.

Q: A more exhaustive set of experiments with different examples per variant

We thank the reviewer for the insightful suggestion. We have explored additional lexical variations beyond those shown in the main paper, including “animal” → “pet”, “show” → “feature”, adding an extra question mark, and “image” → “picture”. These edits result in accuracy drops in descending order of severity, with more semantically impactful substitutions (e.g., “animal” → “pet”) leading to larger changes in probe performance.

Crucially, these effects are concentrated in layers 5–13 of LLaVA-1.5, supporting our interpretation that these middle layers are responsible for integrating lexical signals with visual content. The consistent correlation between semantic strength of the lexical change and the degree of accuracy shift strengthens the conclusion that this layer range is sensitive to linguistic perturbations. We will include these expanded lexical results in the appendix of the next version of the manuscript.

Q: Relate your results to findings in other interpretability research for LLMs

We thank the reviewer for the thoughtful suggestion. We agree that better situating our work within the broader context of interpretability research for LLMs and MLLMs can clarify the motivation and significance of our approach.

Our study offers a structured and fine-grained view of how multimodal large language models (MLLMs) process visual question answering tasks. Using prompt-based probing on the final token embedding across layers, we identify four distinct processing stages in LLaVA-1.5:

1. Visual grounding (layers 1–4)
2. Lexical integration (5–11)
3. Semantic reasoning (12–15)
4. Decoding (16+)

This decomposition highlights functional specialization across the model's depth, revealing how different layers contribute to different stages of the multimodal pipeline.

In contrast, prior works have not explicitly delineated these intermediate stages. For instance, Basu et al. analyze early layer MLPs and attention blocks to understand general information storage and transfer, but do not map how different linguistic perturbations affect the model’s reasoning flow. Similarly, Neo et al. track visual feature propagation in LLaVA but do not explore how linguistic variations interact with different processing stages or how semantic reasoning and output formatting emerge across layers.

Our work complements these studies by introducing a controlled probing framework that isolates specific reasoning components through prompt variations. Importantly, our method is model-agnostic and comparative: we apply the same setup to Qwen-VL and LLaVA-Next, and observe how the layer-wise boundaries of each processing stage shift with architectural and training choices.

We will expand the related work section to include this discussion and more clearly articulate how our approach builds on and extends prior interpretability efforts in both unimodal and multimodal LLMs.

• Basu et al., Understanding Information Storage and Transfer in Multi-modal Large Language Models
• Neo et al., Towards Interpreting Visual Information Processing in Vision-Language Models

---

We sincerely thank the reviewer for the positive and constructive feedback. We're glad the contribution is seen as a valuable step toward MLLM interpretability, and we appreciate your suggestions to strengthen its framing. We would be grateful for your support during the discussion phase.

We thank the reviewer for acknowledging the strengths of our paper, particularly the clarity of presentation, the multi-model analysis, and the design of our probing-based framework. We also appreciate the constructive feedback.

Q: Wouldn't it be more appropriate to train the probing classifier to answer the binary question “Is this an animal?” in line with the model's actual prompt

We thank the reviewer for the thoughtful suggestion regarding aligning the probing task with the anchor prompt “Is this an animal?” using binary classification. We have explored similar setups early in the project and found important limitations that informed our current design.

Below, we illustrate the reviewer’s proposed setup:

In this binary classification setup, the probe easily achieves near-perfect accuracy across all layers when evaluated with the anchor prompt, as the separation between visual classes (e.g., dogs vs. planes) is broad and easily learnable.

However, we observed that applying lexical variants (e.g., "image" → "object") leads to virtually no change in accuracy across layers:

This outcome occurs because:

• The binary decision boundary between dog and plane is overly coarse. These two categories are easily separable in the embedding space, leading to trivial classification.

• Lexical changes cause only small shifts in the representation space, which are insufficient to move embeddings across such a wide separation. As a result, the probe maintains high confidence and accuracy across layers—even when the prompt is perturbed—making it ineffective for measuring sensitivity to prompt variations.

• Using binary labels for fine-grained classes (e.g., “Labrador” vs. “Beagle”) does not resolve the issue. We explored this option by using prompts like “Is this a Labrador?” for each class; however, we found that models like LLaVA often default to answering “yes” for all dog images, regardless of the specific breed of the image. This makes the supervision signal unreliable and prevents the probe from learning a meaningful decision boundary.

In contrast, our setup uses fine-grained visual classes (e.g., different dog breeds) and a multi-class classification probe. Since these classes are much closer together in the embedding space, even small changes in how the prompt is phrased can shift the final token representation enough to cross class boundaries and result in a drop in classification accuracy. This makes the probe a much more sensitive instrument for detecting how prompt perturbations affect internal representations. Our design thus enables clear attribution of variation sensitivity to specific layers, which would be obscured in the coarse binary classification setting.

We appreciate the suggestion and hope this clarifies the motivation behind our experimental design. We will include a discussion of this experiment and comparison in the next version of the manuscript.

Q: Could the controlled environment oversimplify multimodal reasoning

As the first work to apply probing-based analysis to MLLMs, our goal is to establish a controlled and interpretable experimental foundation. By fixing visual inputs and systematically introducing prompt variations, we demonstrate that probing can be a powerful tool for understanding how multimodal information is processed across layers.

The tasks are carefully designed to isolate specific processing components—such as grounding, semantic integration, and answer formatting—without confounding factors. We believe this level of control is essential for building a precise understanding of internal mechanisms before extending the methodology to more complex and realistic scenarios.

We thank the reviewer for their thoughtful and constructive feedback. We appreciate the recognition of our probing framework and its potential utility for future research. Below we address the concerns raised regarding experiment justification, scope, and presentation.

Q: What about including image variants?

We appreciate the reviewer’s suggestion and would like to clarify the motivation behind our experimental setup. The primary goal of this work is to understand how multimodal large language models (MLLMs) solve image-based tasks internally, and in particular, to dissect the distinct roles that different layers play in this process.

To achieve this, we train probing models using the training split of a dog-breed subset from ImageNet and evaluate on its corresponding test split, which naturally introduces variation in image content between training and testing. During evaluation, we apply multiple prompt variants to the same test set: one with the original anchor prompt and others with controlled variations—such as lexical substitutions, semantic negations, or output format changes. The probing model is trained only with the anchor prompt on the training split, and remains fixed during testing. By comparing accuracy across these prompt conditions on the same set of test images, we can isolate the model’s sensitivity to different types of linguistic change. This design enables us to analyze how different layers contribute to key stages of multimodal processing, including visual grounding, lexical integration, reasoning, and decoding.

Q: Only several simple visual question is studied

We appreciate the reviewer’s suggestion and agree that probing models under different types of visual questions (e.g., OCR) is an interesting direction for future work. However, our decision to focus on simple class-discriminative questions (e.g., “Does this image show an animal?”) is intentional and driven by the design constraints of probing.

Specifically, probing requires a stable, categorical target for supervision. In our setup, we use an image classification dataset (ImageNet dog breeds), where each image has a clearly defined class label, enabling us to train probing models that map internal embeddings to consistent targets. For tasks like OCR, however, the expected output is typically open-ended or continuous (e.g., a word or phrase), and it becomes unclear how to define class labels suitable for probing. Constructing a probing task would require grouping images by identical OCR text content—effectively treating each string as a class—which is impractical given the open vocabulary and long-tail distribution of textual content in natural images.

Moreover, our objective is to understand how different layers of an MLLM contribute to the internal processing pipeline, rather than to benchmark performance on diverse VQA tasks. Using simple visual questions allows us to control the linguistic input precisely, isolate the effects of prompt variation, and cleanly attribute changes in accuracy to distinct processing stages such as visual grounding, lexical integration, and reasoning.

Q:Writing Organization and Redundant Figures

We thank the reviewer for pointing out the structural and organizational issues. We will revise Section 2.2 to introduce prompt structure variants alongside the other variant types for clarity and consistency. Additionally, we will remove the redundant Figure 5 and consolidate relevant insights to avoid repetition. These edits will be incorporated in the next version of the manuscript.

Q: In Figure 2, the image -> picture shows much less accuracy drop comparing with show -> feature

We thank the reviewer for the insightful question. We have explored additional lexical variations, including “animal” → “pet”, “show” → “feature”, adding an extra question mark, and “image” → “picture”. These yield accuracy drops in descending order of severity, with more semantically meaningful changes (e.g., “animal” → “pet”) producing larger drops.

Crucially, these effects are concentrated in layers 5–13 of LLaVA-1.5, consistent with our interpretation that these middle layers are responsible for lexical-visual integration. The fact that the magnitude of the accuracy drop scales with the semantic strength of the change further supports the conclusion that these layers are actively sensitive to linguistic content. We will include this additional analysis in the appendix of the next version of the manuscript.

---

We again thank the reviewer for the constructive feedback and insightful questions. We hope our clarifications have addressed your concerns and helped illustrate the rationale behind our experimental design. We are happy to provide any additional details if needed and kindly hope you to consider revisiting your score in light of these clarifications.

We sincerely thank the reviewer for recognizing the value of our work in probing multimodal large language models (MLLMs), highlighting the generalizability of our findings across multiple model architectures, and appreciating the robustness contributed by our prompt variations. We address the specific comments and questions below.

Q: This study relies on only three probing tasks.

We respectfully disagree that the number of probing tasks is a significant weakness of our study. We believe that the strength and clarity of experimental design lie not in the quantity of tasks, but rather in their appropriateness for clearly conveying the intended insights. Specifically, our experimental setup was carefully tailored to address our core research question: how do multimodal large language models (MLLMs) internally process information when performing visual tasks?

Our probing setup follows a controlled strategy: we train a linear classifier using embeddings from a fixed anchor prompt and then evaluate the same classifier on prompt variants at test time. These variants introduce targeted linguistic changes—such as lexical substitutions, semantic negation, or format reformulations—while keeping the visual input constant. The drop in classification accuracy between anchor and variant reveals which layers are sensitive to each type of change. For example, lexical variants modify surface wording without altering meaning; layers that show large performance drops in this case are likely encoding lexical form. By analyzing these shifts across layers and variant types, we are able to clearly identify distinct processing stages within the model, including visual grounding, lexical integration, semantic reasoning, and decoding.

We acknowledge and fully respect that prior linguistic probing studies (e.g., Conneau et al., 2018; Tenney et al., 2018; Jawahar et al., 2019) employed a broader range of evaluation settings. However, their objective was fundamentally different: to analyze the fine-grained linguistic structure learned by unimodal language models. Such work naturally requires diverse linguistic phenomena and syntactic constructions. In contrast, our focus is on uncovering how multimodal models organize and integrate visual and textual information internally. Our controlled probing setup, combined with carefully designed prompt variants, enables us to isolate specific representational behaviors with clarity. This compact yet diagnostic approach is well suited to our goal of identifying distinct stages of multimodal processing.

Q: it is possible that the some of the observed robustness inherits from prior knowledge rather than true generalization

We would like to clarify that our method is not intended to evaluate what the model has “truly learned” in the sense of traditional NLP probing studies, which typically use out-of-domain tasks to avoid overlap with pretraining data. Prior works such as Conneau et al. (2018), Tenney et al. (2018), and Jawahar et al. (2019) aim to test whether internal representations encode linguistic abstractions—like syntax, sentence structure, or semantics—by using a broad suite of carefully designed linguistic tasks. This broad task coverage is necessary given the complexity and diversity of linguistic phenomena.

In contrast, our objective is to understand how different layers of a multimodal large language model (MLLM) operate during the inference process of visual question answering. We do not rely on distributional shifts or task novelty. Instead, our focus is on measuring relative sensitivity across layers using a consistent probing setup: training linear probes on last-token embeddings from a fixed anchor prompt, and testing on controlled prompt variants. Since both anchor and variant operate within the same visual input and prompt structure, our analysis isolates internal behavioral shifts—revealing which layers respond to surface-level, semantic, or structural changes. The insight comes from these controlled comparisons, not from testing generalization beyond pretraining data.