Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology

Q4: Lack of video results for visual demonstration

A4: Thank you for your valuable suggestion. To provide a more intuitive understanding of the benchmark, we have provided visual comparisons between the ground truth trajectory and the model's predicted trajectory. Additionally, we have included first-person perspective videos for further comparison at this link.

Q5: Reliance on current observation and comparison with related methods

A5: Thank you for your insightful question. As mentioned in A3, to address this challenge, we incorporate the UAV’s "historical displacement" and "global position" information into the text input to the large model, allowing the UAV to maintain an awareness of the goal's orientation specified in the instructions. We have validated the effectiveness of this design through ablation experiments.

Regarding the use of Llama-vid, EVA-CLIP, and Vicuna-7B, Llama-vid is an excellent pre-trained multimodal model, and we have built our approach on top of it. Therefore, the image encoder (EVA-CLIP) and the large language model (Vicuna-7B) align with Llama-vid in terms of their architecture.

Unlike Navid, which uses long image sequences for grounded-based vision-language navigation, we supplement our model with text-based UAV state information and introduce a learnable special token <waypoint>, that interacts with the visual-linguistic information for feature extraction.

In terms of model output, Llama-vid generates dialogue text, and Navid outputs action types in text form. In contrast, our model uses the features extracted by the <waypoint> token to predict the next trajectory point for the UAV.

Q6: OSR significantly higher than SR

A6: The discrepancy where OSR is significantly higher than SR is indeed an issue worth noting. This phenomenon is generally caused by difficulties in endpoint localization, leading to the inability to terminate the task.

In UAV-based VLN tasks, the scene space is large, and the agent has a high degree of freedom in flight. However, some target endpoints are small and difficult to detect accurately, such as pedestrians in urban environments or small animals in forested scenes. These challenges make it difficult to accurately localize the endpoint, which results in OSR being higher than SR. This phenomenon is also observed in existing UAV-based VLN works. For example, in the AerialVLN task, the SR is 2.9%, while the OSR is 10.2%; in the CityNav task, the SR is 8.69%, while the OSR is 35.51%. These studies also demonstrate a similar discrepancy between OSR and SR.

Q7: Existing LLM-Based method (NavGPT)

A7: Although NavGPT is indeed a valuable work in the field of indoor navigation, it differs fundamentally from our approach. In NavGPT, the action space of the agent is limited to a set of predefined discrete viewpoints that form a navigation graph, and the agent can only move between these viewpoints. This method corresponds to a discrete VLN task, as clearly stated in their paper: "the agents' action space is confined to the navigation graph G." In contrast, our goal is to directly output arbitrary points in a continuous space, rather than discrete actions or fixed locations, making it a task of realistic continuous trajectory VLN. Therefore, NavGPT does not align with our task setup.

References:

[1] AerialVLN: Vision-and-Language Navigation for UAVs

[2] CityNav: Language-Goal Aerial Navigation Dataset with Geographic Information

Q1: Lack of experimental validation for DAgger

A1: Thank you for your valuable suggestion. We have taken the reviewers' feedback into account and have made necessary additions and revisions to the content of the paper.

In fact, we have presented the model performance after incorporating DAgger in the original version of Table 2 (L447, "Ours-DA"). Compared to the base model, we observed improvements across various metrics, which demonstrates the effectiveness of integrating DAgger into our approach.

Q2: SR is not high even with L1 assistant

A2: Thank you for your insightful comment. We understand your concern about the relatively low SR despite the use of the L1 assistant. We believe the result of SR can be attributed to several factors:

Complexity of UAV VLN Tasks: UAV-based VLN tasks are inherently complex. Previous works have reported relatively low success rates, such as AerialVLN[1] with an SR of 7.2% on its simple dataset and 2.9% on the full dataset, and CityNav[2] with an SR of 8.69%. Notably, AerialVLN uses step-by-step language instructions, and CityNav provides auxiliary information about landmarks near the endpoint on the map. Therefore, research on UAV-based VLN tasks in complex 3D environments is still in its early stages.

Understanding of Language Instructions: Although the L1 assistant provides ground truth instructions, these commands still need to be interpreted by the model. For instance, the meaning of a "turn left" command depends on the context: it could be a slight turn if there is an obstacle on the right or a sharp 90-degree turn at an intersection. Thus, the model needs to correctly understand the language instructions. The performance of the Fixed baseline in Table 2 (L439) also shows that directly mapping instructions to a fixed trajectory does not yield ideal results. We provide visualizations of the fixed baseline trajectoriy and MLLM-predicted trajectory here.

Stricter Failure Settings: Previous UAV tasks had insufficient collision detection. For example, AerialVLN places the UAV at a fixed position in the simulation, ignoring potential collisions during movement. CityNav focuses on high-altitude flight and does not consider interactions with objects. In contrast, we perform realistic UAV flights in the simulator and implement strict collision detection, accounting for obstacles such as power lines in urban environments and trees in villages and forests. This challenges both the model’s visual capabilities and the UAV's flight safety, which affects the overall task success rate.

Q3: Navigation history information

A3: We apologize for not clearly stating in the paper that our strategy was to add "historical displacement" and "global position" information in the coordinate system of the 0th frame in the textual input (we mentioned in the task setup that the UAV's state information can be accessed), allowing the UAV to understand changes in its own state over time.

To validate this approach, we conducted ablation experiments on a small batch of data and analyzed performance across multiple dimensions, including success rate, inference time, and memory usage, for different configurations.

However, the results show that when only historical image information is used, the model's ability to understand historical data remains limited, resulting in some performance degradation. When both historical images and textual information are used, long sequences of multi-view images may interfere with the model's understanding, leading to slight performance drops. This might mean that the decision made based on the current frame is more crucial, and that historical visual information may play a limited role in complex, long trajectories.

Moreover, in the dataset we collected, each trajectory consists of an average of 264 waypoints (as shown in Table 1, L119), meaning that extended steps are required to complete the task. Additionally, for object search tasks, the UAV needs multi-view images, with each frame in our setup containing 5 images. The large amount of image input results in increased inference time and higher memory usage, which also impacts the real-time nature of computation in UAV applications.

Q11: Clarification of main contributions and ablation studies on continuous actions or hierarchical predictions.

A11: Thank you for your feedback. We understand your confusion and would like to clarify the differences between our contributions and existing work:

Platform – Existing UAV VLN platforms often lack APIs that control the continuous and realistic flight dynamics of UAVs, and their extensibility is limited. To address this, we have designed a platform that supports realistic, continuous UAV trajectories. Our platform includes a rich set of expandable scenes and objects, offering features for object placement, UAV control, data collection, and closed-loop simulation.

Dataset – Existing UAV VLN datasets consist of discrete action sequences, which differ significantly from the data collected during real UAV flights in actual environments. Therefore, we have collected 12k trajectories and instruction data, sampled at equal time intervals and at the waypoint granularity.

Benchmark – Existing UAV VLN benchmarks are designed around discrete action sequences, and their challenging task settings often result in low success rate. To address this, we implemented multi-level assistant mechanisms and obtained real-time sensor data from continuous UAV flights with real poses.

Model – Most UAV VLN models rely on traditional VLN baseline models, while some ground-based VLN tasks have begun utilizing large models for high-level decision-making or discrete control outputs. In contrast, we propose a framework that combines both small and large models to collaboratively plan UAV trajectory sequences with hierarchical outputs.

We answered the question regarding continuous action space in A10. We further conducted an ablation analysis on the hierarchical trajectory generation model in terms of performance and computational efficiency. The table below shows that using MLLM alone for long-distance prediction results in a decrease in success rate with similar inference times. Fine-grained predictions using MLLM improve performance by receiving assistance information more frequently. However, this comes at the cost of a significant increase in inference time, making it unsuitable for real-time navigation.

Q12: Abbreviation of UAV

A12: We noticed the abbreviation issue with UAV. Thank you for pointing this out. We will make corrections in the upcoming version update.

Q13: Human baseline

A13: Our human baseline was tested after multiple attempts and training. Under L3, even humans need sufficient exploration to find the target described by the task. This indicates that the drone navigation model must have strong environmental awareness and autonomous exploration capabilities to complete the task.

Reference:

[1] AerialVLN: Vision-and-Language Navigation for UAVs

[2] CityNav: Language-Goal Aerial Navigation Dataset with Geographic Information

[3] OpenCDA: An Open Cooperative Driving Automation Framework Integrated with Co-Simulation

[4] Design and use paradigms for Gazebo, an open-source multi-robot simulator

Q9: UAV autonomous setting

A9: Currently, research on UAV-based VLN tasks is still limited, and significant challenges remain in achieving visual-language understanding and long-term 3D spatial planning in dynamic environments. In designing our assistants, we were primarily motivated by the low success rates of previous UAV VLN methods. For instance, AirealVLN achieved a success rate (SR) of only 7.2% on its simple dataset and 2.9% on the full dataset, while CityNav had an SR of 8.69%, even when assistance was provided by indicating landmarks near the destination on the map. Additionally, our task imposes higher demands, such as more realistic trajectory generation and stricter collision avoidance. As a result, we developed assistants at multiple levels. We envision the highest level of assistance to be similar to a navigation system used when driving in an unfamiliar environment, while lower levels provide more lightweight prompts to gradually guide the UAV toward autonomo us planning.

It is worth noting that the L3 assistant does not require an oracle path to generate its instructions. Instead, it relies solely on depth map information to issue steering commands when the UAV faces imminent danger (e.g., a potential collision), guiding the UAV to avoid obstacles. The decision of how to reach the target remains autonomously made by the UAV. As shown in the results of Table 2, the success rate with L3-level assistance is relatively low, highlighting the challenges of the task. We have added a performance comparison without any assistance using a small batch of test data. The experiment shows completing long-trajectory VLN tasks without guidance still presents challenges.

We have also explored the impact of temporal input and memory mechanisms on model performance. In our dataset, the average trajectory contains 264 points, and search tasks require multi-view images. When using historical images, the visual input to the large model becomes quite extensive. Therefore, we only added the current position and historical displacement information to the input of the large model, while focusing on the current visual information for decision-making. We conducted ablation experiments using historical image information, as shown in the table below.

The experiment shows that when only historical image information is used, the model's ability to understand historical data remains limited, resulting in some performance degradation. When both historical images and textual information are used, long sequences of multi-view images may interfere with the model's understanding, leading to slight performance drops. On the other hand, the growth of image sequences increases inference time and memory usage, which also impacts the real-time nature of computation.

Q10: Discrete baseline and qualitative example

A10: Thank you for your suggestion. We have defined a "Fixed" baseline in the Comparison Baselines section (line 420) and provided the corresponding evaluation results in Table 2. As shown, there is a clear difference between the fixed baseline and the MLLM model using continuous trajectory prediction. We provide visualizations of the fixed baseline trajectory and MLLM-predicted trajectory here, along with a detailed analysis to illustrate the shortcomings addressed by the continuous action space. Additionally, there may have been some misunderstanding. The continuous, realistic trajectories are part of our task setup. Therefore, even in the fixed action setting, as shown in our visualizations, the predicted points are still flown by the UAV along realistic trajectories.

Q4: OpenUAV platform naming

A4: Thank you for your valuable feedback regarding the naming of our platform. To address this concern, we have decided to rename our simulation platform to TRAVEL, which stands for Towards Realistic Aerial Vision and Exploratory Language Navigation. This new name better reflects the unique focus and goals of our platform.

Q5: Clarification of scenario

A5: Thank you for your suggestion. In the paper, "scenarios" refer to different environments or maps available within the platform. Currently, the platform supports 22 distinct scenarios, and we provide descriptions of each scenario in this link.

Q6: Dataset splits details

A6: Thank you for your valuable suggestion. We have provided visual images of objects, as well as the split infomation of scenes and objects in this link.

Q7: Clarification of input to Trajectory Completion module

A7: Thank you very much for pointing that out. The description of the input to the Trajectory Completion module in the paper was indeed unclear. Our intention was to indicate that the Trajectory Completion module takes the front-view image as input, where the front-view RGB image is encoded by ViT to generate the front-view token $T_{img}^f$. We have revised the paper to clarify this point.

Q8: Confusion on the teacher model in DAgger

A8: Thank you for your question. The teacher model is an expert policy that guides the UAV during the DAgger process. Specifically, in our DAgger implementation, the teacher model directly receives the closest position to the current location from the ground truth (GT) trajectory and computes a trajectory sequence, which serves as the UAV's path for the next time step. We have revised the paper to clarify this point.

Q1: Similarities with AerialVLN/CityNav and the need for a new platform and dataset

A1: Thank you for your insightful question.

While AerialVLN[1] and CityNav[2] are valuable benchmarks, they focus on different aspects of the problem. AerialVLN primarily focuses on sentence-by-sentence language instruction navigation in urban environments and uses discrete action sequences. In its specific implementation, they ignore the drone's flight process and directly set the drone at a specified position based on the model's current decision. This approach: 1) overlooks potential collisions during motion (e.g., a single action displaces the drone by 5 meters), and 2) keeps the drone always horizontally oriented, with only the yaw angle changing, while the roll and pitch remain unchanged. CityNav also uses a city-level simulator for goal-directed language instruction navigation with discrete action sequences, but it sets the drone's flight altitude very high (100-150 meters), resulting in a perspective more similar to remote sensing imagery, and the simulator does not handle interactions with objects (such as collisions).

In contrast, we focus on target-oriented language instruction navigation in urban and natural environments, where the flight altitude in the reference trajectories is relatively low. In urban scenes, the drone needs to navigate between buildings, with strict collision detection implemented. Additionally, we use waypoint-level control granularity to guide the UAV in continuous flight, capturing sensor data based on realistic UAV attitudes to complete the VLN task.

In terms of the platform, AerialVLN achieves excellent simulation results using the combination of UE and AirSim, but there are several aspects that still need to be expanded:

Scene and object assets: AerialVLN open-sourced the precompiled environment for Linux, which limits secondary development, especially the ability to add object assets. Many target-related tasks, such as search and perception, cannot be conducted. In contrast, we aim for the platform to support a broader range of drone tasks, which is why we plan to open-source the project source code and compiled versions of various scene files. Additionally, our data collection framework includes a user-friendly API for placing target objects in scenes.

Continuous and realistic trajectory data: AerialVLN currently adjusts the UAV's position by "directly setting its posture"， which lacks the changes in attitude angles during flight. In contrast, we have developed a framework for continuous trajectory collection based on time intervals, inspired by the OpenCDA[3] in the autonomous driving field. This framework captures poses at equal time intervals rather than fixed action sequences, significantly reducing the domain gap between simulated and real UAV flight trajectories.

We provide a comparison of the trajectories between AerialVLN and our dataset in an anonymous link to offer a more intuitive understanding of the task differences.

Q2: Difference from AirSim and clarification of existing vs. new function

A2: Thank you for pointing out the unclear contribution description. We will clarify this in the paper. As you mentioned, AirSim is indeed a powerful drone control plugin, and we used its core functionalities to build our platform. However, the control link using the remote controller-QGC-PX4-simulation environment that AirSim supports is relatively complex, especially when it comes to tasks like environment setup and remote controller calibration.

To address this, we extended the existing functionality by developing user-friendly APIs. This feature allows users to simulate remote controller input via the keyboard, and we have integrated it into our data collection framework.

Q3: Issues in describing related work and differences with UE + AirSim platforms

A3: We sincerely apologize for the confusion caused during your reading. We intended to highlight that widely used simulation platforms like Gazebo[4], which are commonly employed for UAV control, suffer from limitations in visual fidelity and rendering accuracy. Unfortunately, this point was not clearly conveyed in the original version of the paper. We have now updated and clarified the description in the revised version for better clarity.

Regarding the differences with existing works that combine UE and AirSim, our focus lies primarily on the platform's task-specific capabilities and algorithmic support. As mentioned in A1, we aim to address specific challenges in continuous realistic trajectory-based VLN tasks, where previous platforms may have lacked support for such tasks.

Dear Reviewer 7iwn,

Thank you for your valuable suggestions. In response to your comments and concerns raised in the previous round of review, we have provided further clarifications and revisions in the following areas: Comparison to Existing Work, Details of Descriptions / Clarity, Evaluation / Benchmark, Clarification of Main Contributions, and Ablation Studies.

We would greatly appreciate it if you could consider our additional explanations and revisions.

Thank you again for your time and thoughtful review.

Best regards,

Authors of Paper 611

Q4: Ablation Studies

A4: Thank you for pointing this out. Considering the extended time required for full model training and testing, we have supplemented various ablation experiments on a small-scale seen test set. In A2, we provided a performance comparison under different assistance levels.

We further conducted an ablation analysis on the hierarchical trajectory generation model in terms of performance and computational efficiency. The table below shows that using MLLM alone for long-distance prediction results in a decrease in success rate with similar inference times. Fine-grained predictions using MLLM improve performance by receiving assistance information more frequently. However, this comes at the cost of a significant increase in inference time, making it unsuitable for real-time navigation.

We also conducted ablation experiments with fewer sensors. We believe that for search tasks, the front view and top view are more important for detection, so we kept only these two perspectives for the VLN tasks. The experimental results show a slight performance drop across various metrics after reducing the number of sensors, indicating that visual information plays a supportive role in decision-making.

Q5: Comparison with Ground-based VLN

A5: Our method primarily addresses the realism of drone flight. Previous VLN tasks used fixed action sets, positioning agents directly at desired locations, which is reasonable for ground robots. However, drones require attitude adjustments in the air, necessitating waypoint-level output and continuous flight control. Thus, we proposed a waypoint-level UAV navigation model that uses hierarchical output to balance performance and efficiency, aiming to better align with real UAV VLN tasks.

Some classic ground-based VLN methods, like Seq2Seq, can be adapted to our task, but they must be guided to produce waypoint-level outputs. In contrast, models like DUET and NavGPT2 focus on discrete VLN tasks, relying on topological navigation graphs for decision-making, which significantly differs from our task and poses application challenges.

Therefore, we trained a Seq2Seq network and tested it using a small batch of test data. The experimental results indicate that understanding multimodal information remains a challenge for Seq2Seq.

Q6: Extensibility to Other Tasks

A6: Thank you for noting the extensibility of the platform and dataset. Yes, we have made target search one of the tasks supported by the platform. The platform offers a rich set of object assets, interfaces for scene object setup, user-defined scene customization, drone control, and asynchronous parallel data collection, making it ready to expand to more UAV tasks. The continuous data collection framework with equal time intervals and closed-loop simulation support facilitates training UAV trajectory planning models and end-to-end large models.

Our dataset supports target-oriented drone tasks, with drone trajectories that can be reused for various tasks by applying targeted language annotations. For example, annotating UAV action commands to decompose drone trajectories and train low-level instruction-to-trajectory models.

We have also collected depth maps from multiple drone camera perspectives within the dataset, which can be used for mapping tasks. Additionally, using the platform's asynchronous data collection feature, more sensor data, such as LiDAR point clouds, can be collected based on existing trajectories.

References:

[1] Air-M: A Visual Reality Many-Agent Reinforcement Learning Platform for Large-Scale Aerial Unmanned System

[2] AerialVLN: Vision-and-Language Navigation for UAVs

[3] CityNav: Language-Goal Aerial Navigation Dataset with Geographic Information

Q1: Real-world Deployment

A1: Thank you for your thoughtful and detailed review. The transition from simulation to the real-world deployment is indeed a critical issue. For UAVs, we can leverage the ROS framework to connect with real drones, deploying large models on ground stations to receive sensor data such as IMU and RGB data, and return the expected path to complete the flight. Some work like Air-M[1] has implemented hardware-in-the-loop simulation, supporting the transition to real environments.

However, the exploration of UAV VLN tasks is still limited, and achieving vision-language understanding and long-sequence planning in dynamic environments remains challenging. In the latest work, AirealVLN[2] (which gives predefined step-by-step navigational instructions) has an SR of 7.2% on its simple dataset, 2.9% on the full dataset , and 8.69% for CityNav[3] (which gives position of markers near the endpoint in the map). Therefore, we are currently focusing on simulating realistic UAV operations in a simulated environment, aiming to enhance navigation performance before applying it to real-world experiments.

Q2: Autonomy and Reliance on Assistance

A2: In designing the assistance, we considered the low success rates (<10%) of previous UAV VLN methods and the increased complexity of our task. CityNav uses an aerial perspective, and AirealVLN directly moves the drone's position, reducing the need for obstacle avoidance. CityNav assists navigation by providing the location of landmarks near the endpoint. Therefore, we implemented different levels of assistance. We aim for the highest level of assistance to be like a navigation tool used when driving in an unfamiliar environment, while lower levels provide lighter guidance to help the drone gradually achieve autonomous navigation.

We conducted an ablation experiment without assistance. The experiment showed a sharp decline in performance without assistance in the current long-sequence trajectory search tasks. Notably, our L3 setup only uses the drone's depth camera sensors and does not rely on external information like GT trajectories, allowing the drone to autonomously navigate and complete tasks. However, completing long-trajectory VLN tasks without guidance still presents challenges.

Q3: Efficiency and Real-time Performance

A3: Thank you for raising this point about efficiency and real-time performance. Indeed, the complexity of LLMs makes real-time deployment on resource-limited drones challenging. A more feasible approach is to deploy the LLM-based navigation model on the drone's ground station, while traditional flight control algorithms run on the drone itself. They exchange multimodal perception data and flight paths via a communication link. In this setup, the real-time performance of drone navigation depends on the inference and transmission latency of the LLM navigation algorithm.

Testing shows that using a bf16 precision on a single 4090 GPU, the average inference time of the navigation model is 0.407 seconds. With techniques like model quantization, this time can be further reduced. The model can predict waypoints for the drone to fly 5 meters ahead within 0.5 second. When the drone flies at a safe speed, such as 2 m/s, the inference time remains shorter than the execution time, ensuring real-time drone navigation.

Dear Reviewer LL2n,

Thank you for your valuable suggestions on our manuscript. Based on your comments and questions in the previous round of review, we have provided additional clarifications on the following aspects: Real-world Deployment, Autonomy and Reliance on Assistance, Efficiency and Real-time Performance, Ablation Studies, Comparison with Ground-based VLN, and Extensibility to Other Tasks.

We would greatly appreciate it if you could consider our further explanations and revisions.

Best regards,

Authors of Paper 611

Q1: Size of the dataset and authenticity of the simulated data

A1: Thank you for noting our simulation dataset. We believe that the current dataset is sufficient to support the training of MLLM-base models. We would like to address your concerns as follows:

• In Table 1, we compare our dataset with mainstream vision-language navigation datasets. While the total number of trajectories in our dataset is comparable to current VLN benchmarks, our trajectories contain the highest number of actions per sequence, represented as detailed drone trajectory points.
• We segment each full trajectory into shorter training trunks during model training at every timestep. This method results in 405k trunks.
• We train our model on a pre-trained foundation model with basic vision-language capabilities, utilizing LoRA fine-tuning. This approach requires less data compared to training from scratch.

Regarding the authenticity of the data, during data collection, we established a control chain connecting the remote controller, QGC, PX4, and the simulation environment, aligning it with the control chain used in real-world drone operations. Furthermore, our data was collected entirely by human pilots rather than generated using algorithms such as shortest-path planning, ensuring its authenticity.

Q2: Efficiency and real-time performance

A2: Thank you for raising this point about practical drone navigation. We agree that further investigation would be valuable, particularly regarding trade-offs between inference speed and model performance.

After testing, the average inference time for a single forward pass of the model using bf16 precision on a single NVIDIA 4090 GPU is 0.407 seconds. If model quantization or other acceleration techniques are applied, the inference time can be further reduced. The model is capable of predicting the trajectory points for a 5-meter flight within 0.5 second. At a safe drone flight speed, such as 2 m/s, the model's inference time is shorter than the execution time, ensuring real-time navigation.

Additionally, we conducted an ablation study on the hierarchical trajectory generation model by sampling a subset of seen test set, considering the extended time required for full testing. As shown in the table below, we compared the model's performance and computational efficiency.

The results indicate that using MLLM alone for long-range predictions results in a slight decrease in success rate with similar inference times. Fine-grained predictions using MLLM improve performance by receiving assistance information more frequently. However, this comes at the cost of a significant increase in inference time, making it unsuitable for real-time navigation.

Q3: Concern about success rate metrics

A3: Currently, the exploration of UAV VLN tasks is still limited, and achieving vision-language understanding and long-sequence planning in dynamic environments remains challenging. In the latest work, AirealVLN[1] (which gives predefined step-by-step navigational instructions) has an SR of 7.2% on its simple dataset, 2.9% on the full dataset , and 8.69% for CityNav[2] (which gives position of markers near the endpoint in the map), that's why we add assistance in this task.

On the other hand, our model achieves an OSR of 44%, indicating that the model is capable of reaching areas near the destination with assistance. However, due to limitations in detection accuracy, the model struggles to identify targets such as humans, animals, and small objects like road signs. Compared to the latest UAV VLN task, we have achieved a great improvement in OSR with the assistance.

Q4: Ablation studies on different conditions

A4: Thank you for raising this question about performance under different scenarios and target categories. We resampled the test set from the perspectives of scenarios and objectives to evaluate drone navigation performance under different conditions, yielding the following experimental results.

Open natural environments like TropicalIsland experience fewer collisions and have a higher success rate. Smaller maps such as Carla_Town01 and ModernCityMap, also achieve high success rates. However, complex environments remain highly challenging. For example, the narrow streets of NewYorkCity, with higher buildings on both sides and obstacles like streetlights and power lines, result in lower success rates.

Testing two common target objects on the easy (less than 250 meters) test set of the large city map NYCEnvironmentMegapa showed that the salience of the target significantly affects the success rate (32.86% for cars and 14.29% for humans).

Q5: Safety mechanism of navigation system

A5: Thank you for noting the safety mechanisms. The main contribution we claimed in this paper is the UAV simulation platform and the assistant-guided UAV object search benchmark, focusing on advancing UAV visual-language navigation tasks. However, we agree with your suggestion. In our closed-loop simulation, we employ strategies such as hovering or immediate landing in response to severe navigation failures like collisions. Besides, we need to further investigate safety mechanisms to enhance the navigation system during experimental validation involving physical UAVs.

References：

[1] AerialVLN: Vision-and-Language Navigation for UAVs

[2] CityNav: Language-Goal Aerial Navigation Dataset with Geographic Information