CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic

We sincerely appreciate your comprehensive review and constructive feedback, and we are truly grateful for your expressed interest in our work. However, we believe there may be some misunderstandings regarding our methodology and experimental setup. We will address each concern below to provide clarification:

The paper does not clearly specify the specific input format during training:

Thank you for this clarification request. Our training input format for huge regions follows a two-part data structure:

Part 1 - Navigation Planning:

• Input: Huge region image (16,000×16,000 downsampled to 1,008×1,008)
• Output: JSON-formatted navigation path with magnification levels and coordinates

Example:

[{
  "step": 1,
  "magnification": 2.5,
  "region_coordinate": [0.4, 0.4, 0.9, 0.9],
  "reasoning": "Initial scan at medium power to get...",
  "need_to_see": "General tissue architecture..."
},
{
  "step": 2,
  ....
},
.....]

Part 2 - Multi-scale Sequential Reasoning:

• Input: Sequential images [Image1@2.5x, Image2@3.5x, Image3@5.0x, ...] with their metadata
• Output: Integrated diagnostic reasoning across all views

Example output:

"Starting with the overall view at 2.5x ([0.4, 0.4, 0.9, 0.9]): The left border shows normal epithelium transitioning to dysplastic tissue. Moving to 3.5x ([0.6, 0.6, 0.95, 0.95]) at the transition zone, I observe ..."

Both parts are trained together. This enables CPathAgent to learn both strategic planning (where to examine) and diagnostic reasoning (interpreting sequence observations). The model never receives pre-generated patch descriptions during training , it learns to generate all reasoning directly from visual inputs. Examples are provided in Appendix Figures D.5-D.7.

The authors adopted the LLaVA-OV model, which can only accept 1008×1008 resolution input. Therefore, the 16000×16000 huge region is downsampled, which raises the "pseudo zoom-in" problem: the model performs zoom-in on 1008×1008 low-resolution images, which is actually not true zoom-in. The reasonable approach would be to obtain zoom-in coordinates, extract corresponding regions from the huge region, and then input them to the model, rather than operating on downsampled images.

Thank you for raising this concern. We do NOT operate on downsampled images for actual analysis. Instead, we implement exactly as you mentioned 'obtain zoom-in coordinates, extract corresponding regions from the huge region, and then input them to the model'

Our Actual Workflow

1. Planning Phase: The 16,000×16,000 huge region is downsampled to 1,008×1,008 solely for navigation planning.

2. Execution Phase: Using the planned coordinates, we return to the ORIGINAL 16,000×16,000 huge region and crop patches at native resolution. These high-resolution patches are then provided as sequential multi-image inputs to the model.

Example:

• Planning output: "Examine region [0.7, 0.7, 0.95, 0.95] at 3.5x magnification"
• Execution: Extract pixels [11,200, 11,200, 15,200, 15,200] from the original 16,000×16,000 image

This ensures that while navigation planning uses a downsampled overview for efficiency, all diagnostic analysis operates on native-resolution patches cropped directly from the original resolution regions.

Lack of detailed experimental setup descriptions, and corresponding PathMMU experimental details are not found in the supplementary materials. CPathAgent has already exceeded Expert performance, and it needs to be clearly stated whether it was trained on the PathMMU training set.

Thank you for raising this important point. We clarify that CPathAgent was only evaluated zero-shot on PathMMU.

Why CPathAgent exceeds expert performance:

1. Domain specialization gap: Individual pathologists typically specialize in 1-2 disease areas, while PathMMU spans the entire pathology spectrum. This creates an inherent disadvantage for human experts on out-of-specialty cases, whereas LMMs can learn across all disease types.
2. Known LMM shortcuts: The PathMMU authors found their benchmark contains text-based shortcuts that LMMs can exploit. To address this, they created PathMMU-Pro with more rigorous data cleaning.

To verify this, we conduct additional experiments on PathMMU-Pro:

On this cleaner benchmark, CPathAgent no longer exceeds human performance but still significantly outperforms CPath-Omni.

Are the performance gains from patch and patch descriptions (as shown in Figure 2) or from reasoning-focused instruction data? Please provide detailed analysis and experiments.

Thank you for this insightful question. The performance gains come from reasoning-focused instruction data, not patch descriptions.

In fact, the comparison between CPathAgent and CPath-Omni inherently serves as the ablation experiments. As both models share identical Stage 1 & Stage 2 (patch-level perception) training data. They differ only in Stage 3:

• CPath-Omni: Outputs only final diagnostic results
• CPathAgent: Trained with reasoning-focused instruction data including:
  • Rationale for zoom/navigation actions
  • Step-by-step diagnostic feature identification
  • Multi-scale multi-image reasoning chains

Clarification on Figure 2: Figure 2 shows our data generation pipeline, not inference (shown in Figure 1). The patch descriptions serve solely as auxiliary information to help Gemini-2.5-Pro produce more accurate training data for CPathAgent.

To further validate CPathAgent's advantages, we conducted additional experiments to compare CPathAgent and CPath-Omni on two datasets:

PathMMU-HR²:

BRACS (Held-out High-Resolution Dataset for breast carcinoma classification):

The 16.9% improvement on PathMMU-HR² and 6.4% improvement on the independent BRACS dataset demonstrate that performance gains stem from our pathologist-like reasoning approach in Stage 3.

Lack of training method descriptions for WSI classification tasks, and relevant experimental details are not found in supplementary materials C.2. The MIL method results are conventional, but the evaluation method for MLLMs is unclear. Need to clearly specify key information such as input image, resolution and prompt design.

Thank you for highlighting this important omission. We adopt the same training methodology and MLLM evaluation implementation as CPath-Omni to ensure fair comparison. We will add these details in the revised paper.

This is a super-resolution evaluation dataset, but CPathAgent cannot directly process super-resolution images, and input images will be resized to 1008×1008. Under these circumstances, **do the evaluation dataset images only need to be 1008×1008? **Since CPathAgent achieves 88.6 performance at 1008×1008 resolution. Can I consider this not strictly a huge region benchmark?

Thank you for raising this important question. I understand your concern, but there's a key misunderstanding about how CPathAgent processes huge regions. CPathAgent DOES process super-resolution images, it analyzes multiple original-resolution sub-cropped regions, not a single downsampled image.

Here's how it works (as shown in Figure 1 and detailed in Section 3.1):

1. Navigation Planning Phase:

• Input: 16000×16000 huge region → temporarily downsampled to 1008×1008
• Purpose: Only to plan WHERE to look (like a pathologist viewing at low magnification)
• Output: Navigation path with coordinates and magnifications

2. Multi-scale Diagnostic Reasoning Phase:

• Returns to the ORIGINAL 16000×16000 image
• Crops multiple regions based on planned path at native resolution (e.g., 1000×1000 pixels from the original image)
• Reasoning across ~10 different views sequentially

**The 88.6% accuracy comes from analyzing multiple native-resolution patches across the huge region, NOT from a downsampled 1008×1008 image. **In addition, as shown in Table 2, when other models adopt our agent-based approach, they gain ~3% on average (Gemini-2.5-Pro: 73.2%→76.4%, GPT-4.1-mini: 60.1%→62.3%). This confirms PathMMU-HR² is indeed a huge region benchmark requiring multi-view analysis.

The results showing CPathAgent far exceeding Gemini-2.5-Pro need detailed analysis**, which would bring greater value to readers than simply presenting data.

This is a very good question. The reason lies in the training data generation, Gemini-2.5-Pro had access to ground truth WSI reports as guidance, enabling it to produce significantly higher quality navigation paths and diagnostic reasoning than its zero-shot capabilities. CPathAgent, trained on this high-quality data, can therefore surpass Gemini-2.5-Pro's zero-shot performance.

Quantitative Evidence for WSI classification (Table 3):

• Upper Bound (synthetic training data generated by Gemini-2.5-pro): 91.7%
• Zero-shot Gemini-2.5-Pro: 72.1%
• CPathAgent: 82.8%

The 19.6% gap between upper bound and zero-shot Gemini-2.5-Pro quantifies how WSI report guidance dramatically improves data quality. CPathAgent successfully distills this guided knowledge, outperforming zero-shot Gemini by 10.7%.

Thank you for your comments. We appreciate your engagement and would like to address each point with specific references to our manuscript.

1. Clarification on Figure D.5 and the Navigation Planning Pipeline

Figure D.5 actually showcases the final outputs of our two-stage inference process, rather than the training procedure. By presenting both the planned navigation paths and final diagnostic reasoning together, we aim to help readers visualize how the navigation strategy contributes to final diagnostic insights.

Our two-stage process is described in detail throughout the manuscript:

1. Navigation Planning Stage (Lines 165-167): We state "For each preserved huge region, the model plans a navigation path across spatial coordinates and magnification levels. The goal is to plan sequentially examine areas that are most likely to yield diagnostic insights." This states that we first plan the complete navigation path.

2. Multi-scale Sequential Reasoning Stage (Lines 168-171, 203-209):

   • Lines 168-170: "Along the planned navigation path, the model observes each field of view, integrates visual features across scales and generates step-by-step reasoning descriptions"
   • Lines 203-204: "Multi-Image Understanding: We extract image crops at specified magnification levels along the generated viewing paths..."

   As we describe in our manuscript, we crop ALL images along the path at their native resolutions (specified coordinate and magnification levels) and then feed them to the model for ONE-SHOT multi-image reasoning. The "9 operations" are planned waypoints, not separate inference cycles.

   We hope these specific line references help clarify our implementation approach, which is detailed throughout the manuscript.

2. DeepEyes Comparison

DeepEyes's operation numbers are much lower because: DeepEyes addresses targeted visual QA tasks (e.g., "Is the window black and square?", "What color is the blazer?" as shown in their examples) on standard-resolution images, where the visual targets are clearly defined and 1-3 zoom-ins sufficiently to answer the question.

In contrast, CPathAgent analyzes 16,000×16,000 pathology images containing extensive diagnostic features that require systematic multi-scale examination across tissue regions. This is exploratory diagnostic reasoning, not simple verification. The number of navigation steps is determined by inherent task requirements and image complexity, not the choice of RL vs. SFT.

These highlight the novelty of our setting, which, to our knowledge, has not been explored in prior work. While the current approach is not perfect, the empirical results suggest that this direction is promising and merits further investigation.

3. Training Data Clarification

We would like to clarify that we do not train on the patch descriptions shown in Figure 2. These patch descriptions serve as intermediate materials used to guide Gemini in generating navigation planning data. As noted in Figure 2's caption, this figure illustrates "the generation of instruction-tuning data for CPathAgent," rather than the inference process. Figure 1 presents the actual CPathAgent inference workflow.

Regarding the relationship with CPath-Omni, it appears there may be a different interpretation of Reviewer EKpp's original concern. CPathAgent was not " further trained based on CPath-Omni model" as stated in Lines 255-256, we only inherited patch-level training data (CPathInstruct and CPathCaption, appendix line 218-227) that CPath-Omni uses. This is unrelated to TCGA-specific data or WSI-level training.

This follows common practice in the field: similar to how general vision-language models require pre-training on datasets like CC3M, LAION or PMC-15M (e.g., LLaVA, LLaVA-Med) before being fine-tuned on their constructed data, we use patch-level data for basic histological understanding before navigation and reasoning training. This does not affect evaluation validity, as the patch-level data contains no WSI-level or TCGA-specific information.

Because the patch-level descriptions shown in Figure 2 were not used during training, we are unsure which specific comparison needs to be done.

To address this potential concern, we conducted additional experiments using CPathAgent checkpoints pre-trained only on the inherited patch-level data (without our reasoning data). We hope these results help address your question:

These results demonstrate that navigation planning and reasoning data provide substantial gains beyond patch-level pretraining, especially on tasks requiring multi-scale integration.

We greatly appreciate your feedback and welcome any further suggestions you may have.

We greatly appreciate your thoughtful feedback and are grateful for your recognition of CPathAgent's innovative approach in mimicking pathologists' multi-scale diagnostic reasoning and our contributions to dataset construction for pathology model training, and we address all the concerns you have point by point below:

I think this paper places too much emphasis on dataset construction during the writing process.

We appreciate the reviewer's valuable feedback. Due to space limitations, technical details are currently placed in Appendix C.1 (model architecture and training strategy). We will move this part to Section 3, and restructure the paper to prioritize the agent-based architecture and multi-scale reasoning framework in the main paper.

Many of the training steps in this paper overlap with CPath-Omni. However, the differences from CPath are not sufficiently highlighted, and the paper should better emphasize its innovations.

We appreciate the reviewer's important observation. While CPathAgent inherits some patch-level training components from CPath-Omni, it represents a fundamentally different paradigm for WSI and huge region analysis:

The critical innovation lies in Stage 3 of our training pipeline, the agent-based reasoning framework for WSI/huge region analysis:

1. Architectural Innovation:

• CPath-Omni: Static MIL-based approach that embeds all patches and produces a single output
• CPathAgent: Dynamic agent framework that performs hierarchical examination: first planning at low magnification, then strategically navigating to regions of interest and reasoning across multiple scales and multiple images.

2. Interpretability & Clinical Utility:

• CPath-Omni: Black-box predictions without reasoning traces
• CPathAgent: Full interpretability with:
  • Real-time natural language explanations for each navigation decision
  • Step-by-step diagnostic reasoning (see Appendix Figures D.5-D.7)
  • Verifiable examination paths that clinicians can validate

In cancer diagnosis, AI can't replace doctors since no model is 100% accurate. With black-box models like CPath-Omni that only give final answers, doctors still need to manually check the entire slide, as they can't tell whether the AI actually "saw" the cancer or just made a lucky guess. In contrast, CPathAgent shows its WSI navigation and reasoning step-by-step, so doctors can quickly verify the AI's reasoning, transforming AI from a black box into a collaborative diagnostic tool.

The dataset used for training the navigation module is mainly generated by large models. It remains unclear whether the generated navigation paths align with the actual behaviors of human pathologists when reading pathology slides. This needs further validation through experiments or expert analysis.

This is a good question. We ensure synthetic data quality through:

1. Expert-Guided Prompt Development: During prompt engineering, board-certified pathologists iteratively refined our prompts to ensure generated paths reflect clinical examination patterns (detailed prompts in Appendix D.8-D.16).

2. Empirical Performance Validation: The quality of synthetic data is demonstrated by directly using synthetic 'ground truth' data for WSI classification, indicated by upper bound results in Table 3:

• TCGA-ESCA: 100% accuracy
• TCGA-NSCLC: 97.9% accuracy
• TCGA-BRCA: 97.0% accuracy

These results confirm that our synthetic diagnostic descriptions contain accurate and complete clinical and diagnostic information.

3. Expert Validation: Following your suggestion, we conducted further validation where pathologists evaluated 150 randomly sampled navigation paths and reasoning traces on three criteria:

• Diagnostic reasoning validity
• Coverage of critical diagnostic regions
• Clinical soundness

Result: 92.7% (139/150) met clinical standards, confirming strong alignment with actual pathologist workflows.

We will add these validation results to the main paper to strengthen our claims about clinical relevance.

The paper lacks an analysis of the functional contributions of different agent components; specifically, it does not include ablation studies on individual modules, making it difficult to assess their respective impacts on overall performance.

Thank you for this valuable suggestion. Following your suggestion, we conducted systematic ablation studies to evaluate the functional contribution of each agent component, we isolated the impact of individual modules by introducing controlled perturbations:

1. Global Screening Ablation: We replaced the strategic region selection with random selection of 70% of WSI regions. This 70% threshold aligns with the global screening module's design objective of identifying diagnostically relevant regions from WSI thumbnails while filtering out ~30% uninformative areas.
2. Navigation Planning & Multi-scale Reasoning Ablation: Replaced dynamic navigation paths and multi-scale reasoning with direct whole-region description.

Results

Random Global Screening Impact (-2.5%): The modest performance drop validates that strategic region selection effectively balances efficiency and accuracy, successfully mimicking pathologists' selective attention.

Navigation & Multi-scale Reasoning Impact (-7.8%): The substantial performance degradation reveals:

• Direct processing of high-resolution regions (16000×16000) causes critical information loss through forced resizing
• Absence of pathologist-like reasoning that strategically integrates multiple fields of view at varying magnifications impact model performance

These results demonstrate that our agent-based approach derives its effectiveness from the synergistic interaction of all components, with navigation and multi-scale reasoning being particularly crucial for diagnostic accuracy.

Compared with CPath-Omni, the proposed method requires multiple invocations of VLMs, yet the paper does not provide a comparative analysis in terms of time efficiency and computational cost.

Thank you for this important question about computational efficiency. We conducted a comparative analysis using 100 randomly sampled WSIs to evaluate the time and computational costs between CPath-Omni and CPathAgent on H800 GPU.

The two models exhibit fundamentally different computational profiles:

CPath-Omni: Requires extensive preprocessing, which extracts and encodes patches at multiple scales across the entire WSI. For a single region, it processes three different scales, resulting in substantial preprocessing overhead (153.6s).

CPathAgent: Eliminates vision preprocessing by directly operating on regions of interest. To be specific, we leverage vLLM for parallel region processing. On H800 GPU, CPathAgent processes each 16,000×16,000 region in 12.7 seconds (averaged over 1,500 regions), with a total of 237.4 seconds per WSI.

While CPathAgent shows some increase in total processing time (83.1s), this represents a worthwhile trade-off. The agent-based approach provides Interpretable diagnostic reasoning and superior accuracy through explicit navigation paths. In this work, we focus on proposing a novel paradigm for pathology analysis, with efficiency optimization left for future work.

Will the dataset constructed in this paper be open-sourced?

Yes, we are committed to fully open-sourcing our complete dataset, including all synthetic navigation paths, diagnostic reasoning data, and associated annotations to support reproducibility and future research in interpretable pathology AI.

We greatly appreciate your thoughtful feedback and are grateful for your recognition of CPathAgent's clinical relevance and for addressing the gap between AI systems and actual pathologist workflows by mimicking their examination process. However, some concerns may stem from misunderstandings, and we address these one by one below:

It essentially repackages traditional multi-scale image analysis as an agent concept. The so-called "agent behaviors" are fundamentally predefined image cropping and scale selection, lacking genuine autonomous decision-making. Compared to existing attention mechanisms or multi-scale feature fusion, the conceptual innovation is limited.

We appreciate the reviewer's concern but respectfully disagree that CPathAgent merely repackages traditional multi-scale analysis.

1. Most agent frameworks operate within constrained action spaces, classical agent systems define constrained action sets: WebGPT [1] (browser commands), ReAct [2] (search/lookup/finish), MM-ReAct [3] (crop/enhance). Autonomy in agents is characterized by learning policies for action selection, not by discovering new actions. This is commonly recognized in agent works.

2. CPathAgent exhibits genuine autonomous decision-making Unlike static multi-scale fusion, CPathAgent:

• Dynamically plans navigation paths based on visual observations.
• Learns when and where to zoom and move.
• Generates action rationales explaining why each decision was made

This differs fundamentally from predetermined attention mechanisms lacking intermediate reasoning and interpretability.

3. Quantitative evidence The significant performance gaps between agent-based and non-agent approaches in Table 2 (Gemini-2.5-Pro: 73.2%→76.4%, GPT-4.1-mini: 60.1%→62.3%) demonstrate that dynamic navigation policies benefits model performance.

Notably, several pathology agent works have emerged after our submission [4,5], further supporting our agent paradigm.

[1] Nakano et al. WebGPT. arXiv:2112.09332, 2021.

[2] Yao et al. ReAct. ICLR 2023.

[3] Yang et al. MM-ReAct. arXiv:2303.11381, 2023.

[4] Li et al. OmicsNavigator. bioRxiv, 2025.

[5] Chen et al. Evidence-based diagnostic reasoning with multi-agent copilot for human pathology. arXiv:2506.20964, 2025.

How is the quality of such synthetic data ensured? Do the generated navigation paths and diagnostic reasoning genuinely reflect pathologists' actual workflows?

This is a good question. We ensure synthetic data quality through:

Prompt Refine: During prompt engineering, board-certified pathologists provided iterative feedback to refine our prompts, resulting in comprehensive prompts (Appendix Figures D.8-D.16).

Empirical Validation: The quality is demonstrated by upper bound performance in Table 3: achieving 100% acc on TCGA-ESCA, 97.9% on TCGA-NSCLC, and 97.0% on TCGA-BRCA when directly using synthetic 'ground truth' for WSI classification.

To further validate the synthetic data, we invite pathologists evaluate 150 randomly sampled navigation paths and diagnostic reasoning examples based on: (1) diagnostic reasoning validity, (2) coverage of critical diagnostic regions, and (3) clinical soundness. 92.7% (139/150) met clinical standards, which confirms the data quality.

Fairness of Evaluation In the PathMMU-HR² benchmark, only CPathAgent uses the agent approach, while other models not. Part of the performance improvement may stem from the inherited CPath-Omni training data rather than the agent mechanism. The pass@k evaluation uses only 40 samples.

Thank you for this important concern.

1. Regarding evaluation fairness, as shown in Table 2, we provided the same agent-based prompting strategy to GPT-4.1-mini and Gemini-2.5-Pro. Under identical prompts, CPathAgent significantly outperformed both (88.6% vs. 62.3% and 76.4%). These models also improved with our agent approach (GPT-4.1-mini: 60.1%→62.3%, Gemini-2.5-Pro: 73.2%→76.4%).

2. Performance attribution: To isolate our agent mechanism from inherited data, we evaluated CPath-Omni on PathMMU-HR²:

The 16.9% improvement shows that performance gains come from our pathologist-like reasoning (Stage 3's data), not inherited training data.

3. Statistical significance: We expanded pathologist evaluation to 100 samples. Results show navigation path satisfaction reached 85.0% and reasoning satisfaction reached 89.0% at pass@8. Due to rebuttal format limitations, we will present the figures in the revised paper.

How is "navigation planning" actually implemented? Is it merely selecting image patches according to predefined rules? There is a lack of analysis on computational complexity and inference time.

Thank you for this important technical question. Our navigation planning is not rule-based but fully autonomous. The model:

1. Observes each huge region at low magnification
2. Analyzes visual features to decide next actions (zoom locations and scales)
3. Outputs navigation decisions in JSON format for execution This autonomous navigation is learned during training, not predefined.

Regarding computational complexity, we leverage vLLM parallel processing to analyze multiple regions simultaneously. On H800 GPU, CPathAgent processes each 16000×16000 region in 12.7 seconds on average (two VLM calls, averaged over 1,500 huge regions), with 237.4 seconds for each WSI. While slower than MIL, this trade-off is worthwhile as the intermediate reasoning process is crucial for interpretability and diagnostic accuracy. In this work, we focus on propose a novel paradigm for pathology analysis, with efficiency optimization left for future work.

Q1 How do your proposed AI agent adapt the newly incomed training data?

Thank you for this important question. CPathAgent handles new data through two approaches:

1. Zero-shot Generalization: CPathAgent demonstrates strong generalization by learning pathologists' inherent reasoning logic rather than memorizing dataset-specific patterns. We evaluated CPathAgent on the CPTAC-lung (non-TCGA, holdout data) without fine-tuning:

Despite pretrained on significantly fewer WSIs, CPathAgent achieves superior generalization to unseen datasets.

2. Domain Adaptation: When needed, our framework can automatically generate high-quality agent-based training data for fine-tuning using WSI reports (easily available in practice), without costly manual annotations.

Q2 During training, Gemini generates navigation paths based on WSI reports, but during inference without such reports, how does the model determine where to examine next?

We need to clarify that the WSI reports are used only during training data generation, not during model training or inference.

In the Data Generation Phase, we use WSI reports to guide Gemini-2.5-Pro in generating high-quality navigation paths and reasoning sequences. Without this guidance, Gemini-2.5-Pro often produces low-quality data with incorrect diagnoses and navigation patterns.

During the Training/Inference Phase, the model never uses WSI reports. It learns to navigate and reason based purely on visual features and autonomously decides where to examine like pathologists do without prior reports.

Q3: Does the model memorize navigation patterns or genuinely learn visual reasoning? If the system merely memorizes training navigation patterns, how does it handle novel pathological presentations?

Thank you for this insightful question. We provide several empirical evidence against pure memorization:

1. Low Navigation Path Similarity: We analyzed navigation paths across 1,000 test samples using step-wise Region of Interest (ROI) Intersection over Union (IoU) as the similarity metric. The average correlation between navigation paths from different images was only 0.12, demonstrating that the model generates content-adaptive navigation strategies rather than memorizing fixed patterns.
2. Performance on Diverse Test Cases: TCGA WSIs exhibit significant heterogeneity across different cases. If model only memorized patterns, performance would deteriorate dramatically. Instead, CPathAgent maintains high performance compared to MIL methods and CPath-Omni.
3. Generalization capability: As shown in Q1's response, CPathAgent achieves 88.1% on the unseen CPTAC-lung dataset, outperforming models trained on 100× more data. This strong generalization proves the model has learned genuine visual reasoning rather than memorized patterns.

Q4 Base model unspecified: The paper merely mentions "open-source VLMs" without detailing the specific architecture.

Thank you for highlighting this omission. We apologize for the lack of this information. Due to space constrains, we placed the detailed model architecture and training in Appendix Section C.1. We will move it to the main paper.

Each agent can make mistake. The author did not provide the detailed information about the performance when errors may accommulates. Can the system return to previously examined locations for re-evaluation? .

That is correct, error accumulation in multi-step agent systems is indeed a common challenge in agent systems. Currently, CPathAgent follows a forward-only navigation strategy without backtracking. We agree that backtracking would be valuable future work to mitigate error accumulation.

We will provide detailed error cases in the revised appendix (as figures cannot be included in the rebuttal) to better show when and how errors occur in our system.

Dear Reviewer e2cS,

We have carefully considered each of your concerns and hope that our responses adequately address them. We would be grateful if you could share your feedback on our responses, and we are more than happy to provide any further clarification or engage in additional discussion if needed.

Many thanks for your time and effort!

Thank you for your thorough review and constructive feedback. We are grateful for your recognition of CPathAgent as an interesting approach by mimicking pathologists' navigation. Below, we provide detailed responses to each of your concerns:

W1a & Q1. CPathAgent inherits training data from CPath-Omni (pretrained on TCGA), making TCGA evaluation not truly zero-shot and similar to traditional MIL approach.

Thank you for this important question. Although CPathAgent inherits patch-level pretraining from CPath-Omni, its WSI-level reasoning is fundamentally different.

Key Distinction: While CPath-Omni and MIL methods operate as black-box models outputting only final predictions, CPathAgent introduces a paradigm shift through agent-based reasoning that mimics pathologists' diagnostic workflow, dynamically navigating, zooming, and generating interpretable step-by-step explanations of observed morphological features and diagnostic rationale across WSIs.

Zero-shot Validation: To demonstrate true generalization, we evaluated on the out-of-domain CPTAC-Lung cohort using balanced accuracy:

Despite training on only 5,254 WSIs, CPathAgent outperforms PRISM (trained on 100× more data) and CPath-Omni (trained on 2× more data) by learning intermediate reasoning processes rather than memorizing dataset-specific patterns.

W1b. PathMMU-HR2 is developed using the same method as the training data for CPathAgent, so I expect CPathAgent to have an advantage on it.

You raise an important point. While we used similar methods for both training data and benchmark construction, this was unavoidable as no comprehensive ultra-high resolution pathology benchmarks existed.

To further validate our model's generalization, we try our best to find a completely held-out, highest resolution pathology dataset, BRACS (breast carcinoma classification), the only available dataset with regions up to 10K pixels, and evaluate CPathAgent on it:

The 6.4% improvement over CPath-Omni on this independent dataset demonstrates the robustness of CPathAgent's agent-based reasoning capabilities.

W2a&2c&Q1. Can CPathOmni be evaluated on PathMMU-HR2 and the TCGA WSI classification tasks as to properly validate CPathAgent's agentic improvements?

Thank you for this valuable suggestion. We have conducted the requested evaluations:

PathMMU-HR² Result

The 16.9% performance gap occurs because other models directly resize ultra-high resolution images (16000×16000) to their fixed input size (e.g., 336*336). In contrast, CPathAgent uses strategic navigation planning and dynamic zoom-in to examine critical diagnostic details without losing important information.

WSI Classification Result

Direct comparison on the TCGA WSI tasks is unfair as CPath-Omni was trained on the larger TCGA dataset (11,728 cases), while CPathAgent used only cases with WSI reports (5,254 cases) with different train/test splits.

For fair evaluation, we retrained CPath-Omni's stage 3 using our identical training split:

CPathAgent achieves a significant 5.4% improvement, demonstrating that learning pathologist-like diagnostic reasoning and intermediate processes leads to better generalization and learning compared to weakly-supervised approaches.

W2b. Greatly exceeding expert pathologist performance on PathMMU makes the quality of this dataset suspicious, while the PathCLS subset shows nearly identical results.

Thank you for raising this important concern. We provide clarification on several key points:

The PathCLS subset evaluates simple perceptual classification tasks with low-resolution images (e.g., PatchCamelyon: 96×96 patches where simple vision encoders can achieve 99%+ accuracy), fundamentally different from other complex reasoning questions. Since CPathAgent inherits patch-level classification data from CPath-Omni, similar PathCLS performance is expected.

Regarding Expert Performance, CPathAgent exceeds reported expert baselines due to:

• Limited Expert Coverage: PathMMU experts specialize in 1-2 areas while the dataset spans diverse tissues. No single expert masters all domains.
• Text-based Shortcuts: PathMMU authors acknowledged LMMs can exploit strong text-based guessing abilities that humans cannot easily leverage.

To address the text-based shortcut, PathMMU authors created PathMMU-Pro with more rigorous question design. We conduct further evaluation on PathMMU-Pro:

CPathAgent no longer exceeds expert performance on this cleaner benchmark while still outperforming CPath-Omni.

Q2a : Is it necessary to scan around the image sequentially and zoom in and out? Does this have any advantage over a model that takes the full region or WSI at multiple scales as input? Is there any additional training signal in the sequential setup that cannot be used in the everything-at-once input approach?

This is an excellent question that highlights a fundamental design choice in our approach. Sequential navigation offers critical advantages:

1. Interpretable Clinical Reasoning

Our method generates step-by-step natural language explanations of the diagnostic process, unlike MIL/CPath-Omni that produce only final predictions. For life-critical cancer diagnosis, AI must provide transparent reasoning that pathologists can verify.

2. Rich Sequential Supervision Signal

• Traditional Multi-scale approaches use weakly supervised WSI-level labels with MIL, which: (1) Cannot explicit learn which regions are diagnostically important. (2) Have no guidance on optimal magnification levels (3) Fail to capture the diagnostic reasoning process. Therefore, MIL models can easily learn spurious correlations from limited training data, especially when the training signal is very weak (only slide-level labels).
• Our approach provides explicit supervision at each step: where to navigate, when to zoom, and what to examine, this creates a fundamentally stronger learning signal that enables robust reasoning.

3. Superior Generalization

• CPathAgent learns the diagnostic process rather than dataset-specific patterns, enabling stronger generalization to new datasets and achieving better performance with less data (experiments in W1a&1b)
• Gemini-2.5-Pro improves from 73.2% to 76.4% when equipped with our sequential navigation approach (Table 2)

Q2b: Could you explain why "fail to replicate the diagnostic process of pathologists" is a problem? Could you explain what you mean by "poor alignment with clinical workflows" and why this undermines the credibility of validated model performance?

As mentioned in Q2a's response, cancer diagnosis is life-critical and requires human verification. Models that only output final predictions force pathologists to manually re-examine entire slides to ensure diagnostic accuracy, as they cannot trace the AI's reasoning to verify critical findings weren't missed.

"Poor alignment with clinical workflows" refers to models that don't mirror how pathologists systematically examine tissue: starting with low-magnification overview, then strategically zooming into regions of interest. This misalignment creates two problems:

1. Verification inefficiency: Without interpretable reasoning paths, pathologists cannot quickly validate AI findings
2. Limited generalization: As our experiments demonstrate (W1a&1b), models aligned with clinical workflows learn the general diagnostic process rather than dataset-specific patterns, enabling better performance with less data.

Q 3a. When all parameters are unfrozen, does this include CPath-Clip? If so, then all of it (including Virchow2) or part of it?

Yes, all parameters are unfrozen during Stage 3 training, including the entire CPath-CLIP module (with Virchow2). This allows end-to-end optimization for agent-based reasoning.

Q 3b. How are images passed to Gemini when evaluating it without "CPathAgent's prompting strategy"? Do they still cover a mix of magnifications?

For non-agent evaluation, models receive the full large region image (16000×16000) directly as a single input, without sequential navigation and do not cover a mix of magnifications. As shown in Table 2, Gemini with "CPathAgent's prompting strategy" compared to without it can improve performance by 3.2%.

Q3c. How do differences between models look when computing statistical significance in the tables?

While we acknowledge the importance of statistical significance testing, LMMs during evaluation typically use a single final result, as the computational cost of running multiple trials with large pathology models is time-consuming and impractical within the rebuttal timeframe; we will address this in future work. However, our method demonstrates consistent improvements across four diverse evaluation settings, providing empirical evidence of effectiveness.

Thank you for your thoughtful follow-up questions and for considering raising your score. We also appreciate your recognition of CPathAgent's data efficiency and performance improvements. Below we address your specific questions and comments:

Comment 1: Comparison with TITAN on CPTAC

Following your suggestion, we evaluated TITAN on CPTAC:

While TITAN achieves higher accuracy, it requires 64× more training data than CPathAgent. Data quantity is a critical constraint in pathology. This aligns with TITAN's own ablations on TCGA-OT-8K, which show using 12.5% of pretraining data (41,955 WSIs) reduces the accuracy from 0.818 to 0.784.

If TITAN's pretraining data were reduced to CPathAgent's scale (1/64, 5,254 WSIs), the performance gap would likely diminish substantially or even reverse.

More importantly, beyond performance metrics, CPathAgent's core contribution is proposing a fundamentally new paradigm, which introduces an explainable and traceable AI agent pathological diagnosis framework. Performance optimization, while important, is reserved for future iterations.

Q1: Is this the original CPathOmni used in PathMMU-HR2 and BRACS evaluations?

Yes, it is the original CPathOmni version trained on the larger dataset (11,728 WSIs), not the retrained version.

Q2: More information about PathMMU-Pro and how was it created?

PathMMU-Pro was recently created by the PathMMU authors to address text-based shortcuts in the original PathMMU. The dataset is available at their GitHub repo provided in their paper [1] (rebuttal policies prevent direct URL sharing).

PathMMU-Pro was created by:

1. Training text-only models (Phi3 & Llama3) on PathMMU's training set to equip them with stronger question-guessing capabilities, enabling identification of questions that could be answered through text patterns alone without visual information
2. Filtering out these "guessable" questions from the test set to ensure visual understanding is necessary
3. Using GPT-4o to generate more confusing options that are textually similar but require careful visual examination to distinguish

[1] Sun Y, et al. PathBench: Advancing the Benchmark of Large Multimodal Models for Pathology Image Understanding. IEEE TMI, 2025.

---

Q3: In principle, region-level supervision could also be used when training a model that takes multiple resolutions of a full region or slide at a time.

This is a good question. In theory, it's absolutely right. However, there are significant practical challenges that make this approach infeasible:

1. Computational Infeasibility: Processing 4 magnifications (1×, 2×, 4×, 6×) for a single region requires 57 input images. When scaled to WSIs containing thousands of regions, this becomes computationally prohibitive. Current open-source LMMs struggle with this many inputs due to token limits and degraded understanding.
2. Dynamic Flexibility: CPathAgent enables dynamic magnification selection and adaptive crop shapes. For instance, ductal structures often require rectangular crops rather than fixed square sliding windows, providing crucial flexibility for accurate diagnosis.

---

Q4: I guess the main claim here is that it is faster to review the agentic process?

Yes, this is one of the main claims. However, based on preliminary feedback from collaborating pathologists, we hypothesize that CPathAgent offers several potential advantages beyond review speed:

• MIL attention maps, due to their weak supervision nature, highlight areas that are frequently diagnostically irrelevant or misleading. Pathologists reported spending considerable time trying to understand why certain areas were flagged.
• While attention heatmaps show where the model focuses, they cannot explain why those regions matter diagnostically.

CPathAgent aims to address these issues through transparent decision-making and explainable rationale. We agree that formal clinical validation studies are needed to quantify these potential benefits, and we plan to conduct such evaluations in future work.

---

Q5: Does mixing magnifications alone improve Gemini's performance?

This is a great question. Following your suggestion, we conducted ablation studies to isolate exactly this effect:

• Multi-magnification does help: Simply providing all magnification regions to Gemini yields a 1.8% improvement, confirming that multi-scale is valuable.
• But it's not enough: Even with all magnification regions, this approach still underperforms compared to CPathAgent's strategic exploration (76.4%) while feeding all magnifications of all regions simultaneously requires substantially more computational resources.

N1: How does CPathAgent handle multiple slides per case?

Multiple slides per case (e.g., gastric tissue biopsies) are indeed common clinically. Our approach naturally handles this scenario: since CPathAgent outputs detailed textual descriptions with spatial locations, we can synthesize findings across multiple blocks using an additional LLM. We plan to evaluate this multi-slide aggregation in future clinical studies.

---

N2: Are TCGA subsets no longer part of the CPathAgent training data set (unlike that of CPathOmni)?

CPathAgent's training (CPathAgent-Instruct Dataset) does include TCGA data, as it remains the largest publicly available WSI dataset and excluding it would significantly limit training resources. However, evaluation datasets (CPTAC, BRACS) are completely held-out test sets with no overlap with training data. Our 7.8% improvement over CPath-Omni on CPTAC and 6.4% on BRCAS demonstrates CPathAgent's superior generalization capability.

---

N3a: How does CPathAgent's representation quality compare to retrained CPath-Omni?

While direct feature-level comparison is challenging since LMMs generate text rather than embedding vectors, CPathAgent's representation quality can be assessed through downstream performance. The improvement across diverse tasks suggests that our CPathAgent learns more robust and generalizable visual representations.

---

N3b: How does CPathAgent perform on BRACS without navigation?

Thanks for the suggestion. We tested CPathAgent on BRACS without sequential navigation:

Performance drops significantly without navigation (4.7% decrease), though still exceeds CPath-Omni, likely due to exposure to various sizes of fields of view during training.

---

N3c: Can CPathAgent be adapted for survival prediction or biomarker detection?

While CPathAgent currently focuses on diagnostic classification, extending to prognostic or biomarker detection tasks is theoretically feasible and represents an exciting future direction. For example, for prognostic applications, we could incorporate training data linking morphological observations to outcomes (e.g., "Based on these observed morphological features, predict 5-year survival → output: 0.78").

We acknowledge this requires substantial validation. Creating an explainable, LMM-based prognostic/biomarker detection system would be groundbreaking for precision oncology, but rigorous clinical studies are needed to establish reliability and clinical utility.

---

We hope these additional responses fully address your questions. We sincerely appreciate your thorough evaluation and thoughtful follow-up questions that have significantly strengthened our work.