GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution

Q1: This work is motivated by efficiency, but the efficiency of the method itself is not measured. Could the authors provide a brief analysis of the computational overhead of your method? A small clarification here would make the practical benefits clearer and provide a more complete picture of the practical trade-offs involved.

Thanks for your constructive comments. We fully agree that it is necessary to analyse the efficiency. Therefore, we provide inference speed and FLOPs for each compression ratio. We employ the tool calflops and report Tera FLOPs (TFLOPs). We report FLOPs of the LLM side to exclude the computation incurred by the vision encoder, which in our model, accounts for more than a half of the computation. The results of Inference Speed are tested using a image of a resolution of 10,000 $\times$ 10,000 which results in 332k image patches for our 8k model.

The table presents compression ratios applied consistently during both training and testing—for instance, a 24× ratio is used throughout training and evaluation on XLRS-Bench. Overall, performance declines with higher compression, with the best result—54.5 accuracy—achieved at 24×, the setting adopted by GeoLLaVA-8K. GeoLLaVA-8K supports resolutions up to 8K × 8K, generating a large number of visual tokens. With a 24× compression ratio, about 14K tokens are produced, resulting in higher FLOPs and slower inference. Increasing the ratio to 96× significantly reduces tokens, boosts TFLOPs and inference speed, while maintaining strong performance—49.7 accuracy, outperforming Qwen2.5VL-7B's 47.4. As our focus is model performance, we adopted the 24× setting. In future work, we plan to optimize inference speed and better balance performance and efficiency.

Q2: The use of [CLS] attention for selecting important tokens is a key design choice. Could you select 1-2 examples from the XLRS-Bench where the question involves reasoning about multiple small objects or their spatial relationship. Does the attention successfully capture all task-relevant objects, or does it sometimes focus on a single, most prominent one?

We offer comprehensive experiments on small targets to demenstrate that the [CLS] attention mechanism captures all task-relevant objects in GeoLLaVA-8k, it rarely focuses on the most prominent one. Future work could further improve [CLS] attention by incorporating fine-grained visual encoders from CLIP-based models such as FG-CLIP or FineCLIP.

Unfortunately, the latest NeurIPS rebuttal policy does not allow image submissions. Therefore, we designed a new experiment to address your concern regarding whether token compression leads to the loss of dense small-object information. We extracted samples that meet the definition of small objects from relevant subtasks in XLRS-Bench to create a new small-object subset of the benchmark. We then evaluated both the baseline LLaVA-Next and our method on this subset. The detailed experimental procedure is outlined below.

We selected six small-object-related tasks from XLRS-Bench—object classification, object color, motion state, global counting, regional counting, and spatial relation reasoning. After manual verification, we curated a dedicated small-object subset, which will be open-sourced to support related research.

• Small Object Criteria: COCO dataset defines small objects by absolute pixel area (<32² pixels), which is less suitable for high-resolution remote sensing images. Given COCO's typical image size (400–650 pixels), small objects occupy roughly 2.5%–5% of the image area. Inspired by this, we adopt a relative area-based classification: Extremely Small: <0.1% of image area, Very Small: 0.1%–2.0%, Normal Small: 2.0%–5.0%, Large: ≥5.0%.
• For object classification, object color, and motion state tasks—which include bounding boxes—we calculated the pixel area of each object. We identified 1,450 samples where objects occupy less than 5% of the image. A detailed breakdown is provided in the table below.

• For counting and spatial relation tasks without bounding boxes, we manually annotated targets and calculated their area ratios. Samples with any object exceeding 5% of the image area were excluded, yielding 40 test samples per subtask. Each sample contains only small objects, categorized by the smallest object's area ratio into Very Small(<2.0%) and Normal Small(2.0%–5.0%).
• Experimental Results: GeoLLaVA outperforms the baseline across all six tasks in the newly constructed small-object test set, raising overall accuracy from 34.05% to 40.25% (+6.20%). The most significant gain is in spatial relation reasoning (+15%), followed by global counting (+10%) and motion state (+8%). Color recognition shows the lowest accuracy (34.43%), highlighting the difficulty of extracting color features from small objects. These results suggest that high-level cognitive reasoning tasks hold greater potential for improvement than basic perceptual ones.

Overall, GeoLLaVA-8K, trained on SuperRS-VQA and HighRS-VQA, significantly outperforms the baseline LLaVA-Next on small-object tasks, highlighting the substantial benefits of higher resolution(8kx8k). However, as noted in your comment on “information loss in scenarios with dense small targets,” GeoLLaVA-8K still struggles—achieving only ~34% accuracy on some tasks. This indicates limited capability in extracting features from dense small targets, falling short of full recognition and information capture. Addressing this limitation should be a key focus for future research. Thanks for your constructive comments.

Your feedback is highly constructive and deeply encouraging to us. We sincerely thank you once again. If you have any further questions or concerns, we would be truly grateful for your guidance.

Q1: Show comparison with other datasets.

We agree that more benchmark datasets should be included. At the time of submission to NeurlPS'25, UHR remote sensing MLLM benchmarks were scarce, with XLRS-Bench being the only open-source, formally published option.

During the rebuttal phase, and following your suggestion, we identified LRS-VQA—a newly accepted ICCV 2025 benchmark—well-suited for evaluating MLLMs on ultra-high-resolution remote sensing tasks. However, LRS-VQA only offers open-ended QA formats, which are incompatible with standard evaluation platforms like VLMEval.

To address this, we created a multiple-choice version of the LRS-VQA dataset and shared it with the LRS-VQA authors and VLMEval platform to support broader community evaluation. Our processing pipeline for LRS-VQA is as follows:

1. Plausible Distractor Generation: We employed a large language model (GPT-4o) to generate three semantically plausible distractors for each question. The prompt explicitly instructed the model to ensure that the distractors:
   • Are relevant to the question context,
   • Belong to the same semantic category as the correct answer (e.g., vehicle types, object status),
   • Do not overlap with the correct answer.
2. Binary Question Handling: For binary or contrastive questions (e.g., “rural or urban”, “yes or no”), we automatically restricted the number of answer choices to two, ensuring alignment with the question format and avoiding unnecessary ambiguity.
3. Strict Format Enforcement and Post-validation: We enforced a strict JSON structure during generation, requiring each output to contain:
   • The original question,
   • Exactly two or four answer choices labeled with keys "A", "B", (and optionally "C", "D"),
   • A valid answer key pointing to the correct choice. All generated outputs were further post-processed to validate their structural consistency and ensure that the correct answer was included and accurately labeled.

As a result, we successfully generated 7,145 MCQ-style QA pairs out of the 7,333 original questions. This allowed us to perform additional quantitative evaluations using a multiple-choice format that complements the original open-ended benchmark.

We evaluated LLaVA-Next and our GeoLLaVA-8K on LRS-VQA, with GeoLLaVA-8K achieving 56.28% accuracy—1.21% higher than the LLaVA-Next baseline which achieving 55.07%—demonstrating our generalization. Looking forward, our data generation pipeline which was used for generating HighRS-VQA can be readily adapted to augment LRS-VQA, further boosting GeoLLaVA-8K’s performance. We will open-source the data generation pipeline to support the community in generating high-quality training data for ultra-high-resolution remote sensing tasks.

Q2:What is the difference between the tasks that GeoLLaVA-8K performs better or worse, and what could be the potential reason?

Due to space constraints, the main text lacks detailed subtask analysis. Therefore, we present the following analysis:

Most subtasks benefit from our datasets and methods, though performance declines are observed in Overall Counting, Regional Counting, Object Color, Object Motion State, and Anomaly Detection，comparing with baseline. These drops stem from the datasets, which lack sufficient or relevant samples for these subtasks. Our proposed method helps alleviate this issue. For example, the Regional Counting task in XLRS-Bench requires counting small objects within red circles—a scenario rarely found in our SFT data. Addressing this requires generating samples that improve the model’s ability to follow such instructions to detect red circles and better capture small-object features.

While extending the GeoLLaVA-8k to five compression ratios, we discovered an intriguing finding: Several underperforming subtasks (OC and RC) in GeoLLaVA-8K showed significant improvement with higher compression ratios.

The table shows that compression ratios were applied consistently during training and testing. Overall performance declines with higher compression, with the best accuracy—54.5—achieved at 24×, the setting used in GeoLLaVA-8K. Interestingly, some subtasks improve with increased compression. We believe different RS tasks have distinct optimal compression levels—OC, RC, OCC, OCL, and OSR each exhibit unique preferences. Notably, OC and RC achieve substantial gains at 96× compression, outperforming the baseline (LLaVA-Next). This suggests that downstream tasks may benefit from tailored compression ratio. For instance, in OC and RC, higher compression may remove redundant information, helping the model focus more effectively on targets. As GeoLLaVA-8K is a unified model, we use a single global compression ratio (24×) for evaluation. These results will be included in future versions. We encourage future research to explore task-specific compression strategies, a promising direction for impactful standalone work.

We sincerely thank you again for your constructive feedback. If you have any further questions or suggestions, we would be more than happy to address them.

Thank you, authors, for addressing all my questions. I will maintain my already positive score.

We greatly appreciate your constructive feedback and meticulous review. We’re pleased to share a brief new about the Q1: the LRS-VQA authors appreciated our newly added Q1 experiments and plan to update their website with our provided MCQ version. Thank you again for your valuable feedback to the RS UHR MLLM community. In summary, we would like to express our sincere gratitude for your positive score and further support towards our work!

Q1:More ablation results

We fully agree with your suggestion. The ablation studies in appendix, along with the additional experiments and deeper analysis completed during the rebuttal, should be incorporated into the main text. As your recommended, we evaluated the baseline model LLaVA-Next on our high-resolution datasets (SuperRS-VQA and HighRS-VQA).

Experimental Setup: We followed LLaVA-Next’s original settings and used the same hyperparameters as in the GeoLLaVA-8K experiments (epoch=1, lr=5e-6, Vision_tower_lr=1e-6, warmup_ratio=0.1, etc.). 'Dataset' means SuperRS-VQA and HighRS-VQA, 'BTP' means the background token pruning method, 'ATS' means anchor token selection method.

The results show that while training the baseline model on HighRS-VQA and SuperRS-VQA improves performance, further gains are achieved by adding our BTP and ATS strategies. Most subtasks benefit from our datasets and methods, though performance declines are observed in Overall Counting, Regional Counting, Object Color, Object Motion State, and Anomaly Detection.These drops stem from the datasets themselves, which lack sufficient or relevant samples for these tasks. Our method helps alleviate this issue. For example, the Regional Counting task in XLRS-Bench requires counting small objects within red circles—a scenario rarely found in our SFT data. Addressing this requires generating samples that improve the model’s ability to follow such instructions to detect red circles and better capture small-object features.

The background token pruning and anchor token selection strategies are strongly coupled. The first strategy alone causes OOM issues, prompting the need for the second. As an adaptive, iterative method with learnable parameters, the background token pruning struggles to handle numerous background tokens in early training, improving only in later training stages. Therefore, we will conduct additional ablation studies to examine their joint performance under varying compression ratios in the following responce.

Q2:Experiments on different token compression ratios

The appendix includes an ablation study on compression ratios, but only at three levels. We extended the study to five compression ratios, reporting both computational efficiency and semantic preservation (accuracy). We also detail subtask-level performance across these ratios with comprehensive discussion.

The table presents compression ratios applied consistently during both training and testing. Overall, performance declines with higher compression, with the best result—54.5 accuracy—achieved at 24×. Interestingly, some subtasks improve with higher compression, supporting your commnets “there are optimal ratios for different RS tasks”. Notably, OC and RC see substantial gains under 96× compression, significantly outperforming the baseline (LLaVA-Next). As GeoLLaVA-8K is a unified model, we use a single global compression ratio (24x) for evaluation. We encourage future work to explore task-specific compression strategies, a promising direction for impactful standalone research. We also provide inference speed and FLOPs for each compression ratio. We employ the tool calflops and report Tera FLOPs (TFLOPs) of the LLM side to exclude the computation incurred by the vision encoder. The results is tested using a image of a resolution of 10,000 $\times$ 10,000 which results in 332k image patches for our 8k model.

GeoLLaVA-8K supports resolutions up to 8K×8K, generating a large number of visual tokens. With a 24× compression ratio, about 14K tokens are produced, resulting in higher FLOPs and slower inference. Increasing the ratio to 96× significantly reduces tokens, boosts TFLOPs and inference speed, while maintaining strong performance—49.7 accuracy, outperforming Qwen2.5VL-7B's 47.4. As our focus is model performance, we adopted the 24× setting. In future work, we plan to optimize inference speed and better balance performance and efficiency.

Q3: Whether performance improvements primarily derive from data resolution or strategy efficiency.

Your feedback is both constructive and timely. As noted in our response to your first question, we conducted separate experiments on the baseline model using the HighRS-VQA and SuperRS-VQA. The results clearly validate the effectiveness of our datasets and token compression strategies.

Q4: Failure cases to identify whether token compression causes information loss in scenarios with dense small targets?

Unfortunately, the latest NeurIPS rebuttal policy does not allow image submissions. Therefore, We address this issue from two perspectives, backed by extensive experiments.

(1) GeoLLaVA-8K does not cause significant loss of small-object information.

We designed a new experiment to address your concern regarding whether token compression leads to the loss of dense small-object information. We extracted samples that meet the definition of small objects from relevant subtasks in XLRS-Bench to create a new small-object subset of the benchmark. Specially, we selected six small-object-related tasks from XLRS-Bench—object classification, object color, motion state, global counting, regional counting, and spatial relation reasoning. After manual verification, we curated a dedicated small-object subset, which will be open-sourced to support related research.

• Small Object Criteria: COCO dataset defines small objects by absolute pixel area (<32² pixels), which is less suitable for UHR RS images. Given COCO's typical image size (400–650 pixels), small objects occupy roughly 2.5%–5% of the image area. Therefore, we adopt a relative area-based classification: Extremely Small: <0.1% of image area, Very Small: 0.1%–2.0%, Normal Small: 2.0%–5.0%, Large: ≥5.0%.

• For object classification, object color, and motion state tasks—which include bounding boxes—we calculated the pixel area of each object. We identified 1,450 samples where objects occupy less than 5% of the image.

• For counting and spatial relation tasks without bounding boxes, we manually annotated targets and calculated their area ratios. Samples with any object exceeding 5% of the image area were excluded, yielding 40 test samples per subtask. Each sample contains only small objects, categorized by the smallest object's area ratio into Very Small(<2.0%) and Normal Small(2.0%–5.0%).

• Results: GeoLLaVA-8k outperforms the baseline across all six tasks in the newly constructed small-object test set, raising overall accuracy from 34.05% to 40.25% (+6.20%). The most significant gain is in spatial relation reasoning (+15%), followed by global counting (+10%) and motion state (+8%). Color recognition shows the lowest accuracy (34.43%), highlighting the difficulty of extracting color features from small objects. These results suggest that high-level cognitive reasoning tasks hold greater potential for improvement than basic perceptual ones.

Overall, GeoLLaVA-8K significantly outperforms the baseline LLaVA-Next on small-object tasks, highlighting the substantial benefits of higher resolution(8kx8k). However, as noted in your comment on “information loss in scenarios with dense small targets,” GeoLLaVA-8K still struggles—achieving only ~34% accuracy on some tasks. This indicates limited capability in extracting features from dense small targets. Addressing this limitation should be a key focus for future research.

(2) Failure Cases

In a regional counting task, GeoLLaVA-8K failed to accurately count ships within a red-circled of small harbor. Although 53 ships were present within harbor and 38 within the red circle, the model misinterpreted the instruction, counted the entire harbor, and returned an incorrect result. These errors likely stem from a lack of relevant training samples with red circles. Addressing this will require generating more targeted samples to enhance the model's ability to follow spatial cues and detect small-object features. While GeoLLaVA-8K generally outperforms the baseline, occasional failure cases—as you pointed out—highlight areas for future improvement.

All of your suggestions are truly constructive, and our team unanimously agrees that they will greatly improve the quality of our work. We sincerely appreciate again for your valuable guidance.