Multimodal Causal Reasoning for UAV Object Detection

Thank you for your thoughtful review and for recognizing the merits of our work. We have carefully addressed the concerns raised and hope that our answers meet your requirements. We kindly ask you to consider raising the score if possible . Thank you!

Q1: Interestingly, it looks Yolov8 is already a very strong baseline for VisDrone, if you directly compare the results from Table 3 with Table 1. One potential weakness of the paper is the novelty, overall, it is more like an okay, incremental work for the UAV object detection.

A: We would like to summarize the significance and novelty of this work again (please also see our responses to reviewer nLTh): (1) We agree that YOLOv8 is a strong baseline on the VisDrone dataset, which makes further improvements on top of it particularly challenging. Nevertheless, our method enhances this baseline with causal reasoning and a lightweight architecture. This combination helps to effectively reduce inter-class inconsistency caused by complex scenes and scale variation. As shown in Table 1, our approach achieves significant improvements in both detection accuracy and inference speed.

(2) We would like to take this opportunity to reiterate the key innovations of our work. Unlike incremental improvements that simply build upon existing methods, our approach introduces a novel causality-aware reasoning framework tailored specifically for the unique challenges in UAV object detection, such as inter-class inconsistency caused by complex scenes and scale variations. This fundamental shift in methodology, combined with lightweight design choices, distinguishes our work form prior efforts and leads to significant gains in both accuracy and efficiency. Therefore, we believe our contribution goes beyond a mere incremental step.

Q2: One question is - how is the importance of the pretrained CLIP model in proposed method? if changing the CLIP models to a different pretrained models, since two parts of the framework depend on CLIP text encoder.

A: The proposed method does not strongly depend on CLIP itself. In our framework, CLIP serves mainly as a vision-language embedding tool to construct the confounder dictionary for causal reasoning. We chose CLIP for its wide adoption and strong generation, but our framework remains compatible with other VLMs, provided they can generate aligned text-image embeddings. We replaced CLIP with OpenCLIP[1] in our experiments, and as shown in Table 1, the results confirm that our method remains effective with alternative VLMs, demonstrating its flexibility and generalizability.

Table 1. Comparison of using CLIP and OPENCLIP.

Q3: What is the difference between Yolov8 in Table 3 and UAV-YOLOv8 in Table 1?

A：Sorry for the confusion. UAV-YOLOv8 [2] is an enhanced version of YOLOv8 specifically tailored for UAV-based object detection. Unlike the original YOLOv8 baseline reported in Table 3, UAV-YOLOv8 (Table 1) integrates three key improvements: a more robust regression loss (WIoU v3), an attention mechanism and a five-scale detection head with FFNB.

Q4: Limitations: No limitation section for the submission.

A: Thank you for the comment. We would like to clarify that the previous version already contains a detailed discussion of the limitations in the appendix (lines 128–135), which may have been overlooked. We will add a concise summary of this section to the main text for better visibility.

References

[1] Cherti M,et al. “Reproducible scaling laws for contrastive language-image learning”,CVPR2023.

[2] Wang G, et al. “UAV-YOLOv8: A small-object-detection model based on improved YOLOv8 for UAV aerial photography scenarios”. Sensors, 2023.

Dear Reviewer,

The authors’ rebuttal has been posted. Please check the authors’ feedback, evaluate how it addresses the concerns you raised, and post any follow-up questions to engage the authors in a discussion. Please do this ASAP.

Thanks.

AC

Dear Reviewer,

Could you please look at the authors' rebuttal and comments from other reviewers and make your final rating with justifications? Please do this asap as the deadline is approaching.

Thank you.

AC

Thank you for your constructive feedback and for acknowledging the strengths of our work. We have carefully addressed the concerns raised and sincerely hope our responses clarify the contributions and improvements made. We would greatly appreciate your consideration of a higher score if you find the revisions satisfactory.

Q1: The incorporation of CLIP with detection models is not new and has been demonstrated in other works.

A: We would like to clarify our contribution is not a simple fusion of CLIP and detection models. (1) CLIP usage is different. Unlike previous works that focus on integrating CLIP directly into detection models, our approach leverages CLIP more as an auxiliary tool to enable causal reasoning in challenging UAV scenarios. Our core idea is to utilize a vision-language model to generate embeddings of diverse textual descriptions, which are then used to construct a confounder dictionary. This dictionary facilitates causal reasoning to address inter-class inconsistencies caused by complex scenes and scale variations in UAV data. So our method is independent to CLIP at inference stage. Of course, other VLMs can also be used instead of just CLIP.

(2) Our innovation lies in introducing a causality-aware framework that goes beyond fusion of CLIP. By embedding language semantics into LGRE and and causal reasoning(BACR), the method reduces visual ambiguity and improves generation, especially for small or confusing UAV objects. This presents a novel and effective use of VLMs beyond conventional fusion-based approaches.

Q2: The connection with UAV-based visual data is weak and in this regard it is not clear why the particular application domain was selected.

A: We appreciate the reviewer’s comment and would like to clarify the motivation behind choosing the UAV-based visual domain. As mentioned in lines 24-32 on the first page, UAV scenarios are characterized by complex scenes, frequent scale variations, and a high degree of inter-class visual ambiguity, which are challenges that often degrade detection performance. Our framework is specifically designed to address these issues through causality-aware reasoning and enhanced semantic alignment. Therefore, UAV-based detection is not only a representative but also a highly relevant application domain for validating the effectiveness of our method. Experimental results on VisDrone and UAVDT further support this choice.

Q3: While the comparisons with vision models are adequate, the paper is missing comparisons with text based YOLO models such as (Yolo-world CVPR, YOLOE).

A: Thank you for your valuable comments. We have added comparison experiments with YOLO-World[1] and YOLOE[2].The results demonstrate that our method still achieves superior detection performance compared to other text-based YOLO detectors.

Table 1. Comparison of YOLO-World and YOLOE.

Q4: In page 7 lines 285-286 the initial learning rate is reported as 0.001 and the final is 0.01. Usually, the learning rate reduced over the duration of the training. Is this a typo?

A: We apologize for the confusion caused by this typo, we will correct it in the paper. In our experimental settings, we set lr0 (the initial learning rate) to 0.001 and lr1 to 0.01, resulting in a final learning rate of lr0*lr1.

Q5: With reference to Table 4. Why does the parameters of the MCR-UOD models are lower than the original models? Don’t the language model parameters increase the total count?

A: Thank you for your valuable comments. As stated on page 8, line 329 (Main text), and page 6 (Appendix), line14. The reduction in model parameters primarily stems from compression both visual and text features to a unified channel dimension (256), whereas YOLOv8 uses higher dimensions (1024 and 512) in its detection head. This adjustment leads to a more lightweight structure. Importantly, despite the lower parameter count, MCR-UOD maintains competitive detection performance, effectively meeting the demands of UAV-based detection tasks. We will provide additional details in the relevant sections.

Q6: Limitations: Perhaps a discussion on the impact of biases introduced because of the language model could be useful.

A: Thank you for the suggestion. We agree that discussing potential biases from the language model is important and will add this in the revised version. Such bias often stem from imbalanced or incomplete training data, which may affect semantic guidance and lead to skewed reasoning in some scenarios. However, systematically analyzing these bias is challenging, as it requires controlled experiments, access to training data distributions, and clear fairness definitions across tasks and categories. While these bias exists, large scale dataset pretraining allows the model to capture useful semantics, particularly for common UAV categories like vehicles and pedestrians, where it has helped improve detection.

References

[1] Cheng T, et al. “Yolo-world...”.CVPR2024.

[2] Wang A, et al. “Yoloe...”. ICCV2025.

Dear Reviewer,

The authors’ rebuttal has been posted. Please check the authors’ feedback, evaluate how it addresses the concerns you raised, and post any follow-up questions to engage the authors in a discussion. Please do this ASAP.

Thanks.

AC

We thank the reviewer for the very encouraging comments on the originality of MCR-UOD, the effectiveness of MCR-UOD, and the overall good presentation. We hope to provide satisfying answers to the concerns raised. We kindly ask you to consider raising the score if possible . Thank you!

Q1: Ablation Study of top-N selection mechanism.

A: We added ablation experiments on VisDrone to assess the top-N selection mechanism, as shown in Table 1. Even without it, the model still improves performance.

Table 1. Ablation study on whether using top-N selection mechanism.

Q2: Ablation Study on whether updating mean CLIP feature.

A: We added an ablation study on whether to update the CLIP features. As shown in Table 2, experimental results show that without updating the mean CLIP features, BACR can still improve performance. However, the improvement is slightly weaker compared to the setting where the feature dictionary is updated.

Table 2. Ablation study on whether updating mean CLIP feature.

Q3: What is the source of performance improvement—classification accuracy or more precise bounding box regression? What specific problem does the proposed method solve?

A: We conducted a more detailed analysis on experimental results which confirm that both classification accuracy and bounding box regression contribute to the performance gain. As shown in Table 3, the reduction in cls_loss indicates improved classification accuracy, while the decreases in box_loss and dfl_loss reflect more precise bounding box regression. These are further supported by the higher precision and recall, showing an overall enhancement in detection quality.

Table 3. Analysis on performance improvement.

Q4: The YOLOv8 baseline in the paper uses category text embeddings. How does it compare to a learnable classifier (as in traditional detectors), and why were category text embeddings chosen?

A: (1) We compared the baseline using a learnable classifier with our method based on CLIP classification. As shown in Table 4, using CLIP text embeddings for classification yields a noticeable performance improvement over the learnable classifier.

Table 4. Performance comparison of learnable classifier with CLIP text embeddings.

(2) We chose CLIP text embeddings because text and image features of the same class are tightly clustered, while those of different classes are clearly separated as CLIP model is pretrained on millions of text-image pairs. By employing these embeddings in both the LGRE and BACR modules and optimizing classification loss end-to-end, the model better integrates semantic information and enhances feature consistency.

Q5: Is there any other CLIP enhance detector could chose as a more fair baseline?

A: We have added comparison results with YOLO-World[1] and YOLOE[2], which are also CLIP enhanced YOLO detectors. As shown in Table 5, our method achieves better detection performance compared to other CLIP enhanced detectors.

Table 5.Comparison of different CLIP enhanced detectors.

Q6: Limitations:No. The authors should discuss whether the work and its application methods remain generalizable to other tasks or categories.

A: Thank you for the suggestion. We would like to clarify that the limitations of our method are discussed in the appendix (lines 128–135). Regarding generalizability, although our framework is developed for UAV detection, its core components, such as multimodal semantic guidance and causal reasoning, are not restricted to this task. Theoretically, our method can also be applied to other scenarios, including transfer learning and zero-shot detection. We consider moving this discussion to the main text for easier reader access.

References

[1]Cheng T, et al. “Yolo-world...”.CVPR2024

[2]Wang A, et al. “Yoloe...”. ICCV2025

Dear Reviewer,

The authors’ rebuttal has been posted. Please check the authors’ feedback, evaluate how it addresses the concerns you raised, and post any follow-up questions to engage the authors in a discussion. Please do this ASAP.

Thanks.

AC

The authors' response has addressed most of my concerns. Overall, this is a rather interesting piece of work. I decide to keep my score. I did not further increase the score because, in my view, the most common approach to handling different scenarios in practical applications is to augment data related to specific aspects. The method of extracting features using language models, which requires alignment in the feature space, generally yields inferior performance compared to the former. This limits the deployment of this work in real-world scenarios. However, the idea behind this work is still quite interesting, so I have kept the score unchanged overall.