Embodied Crowd Counting

Q1: The paper does not explain ‘VPS’ and ‘ATE-VPS’ in Table 5, which makes this experiment confusing. This experiment seems to demonstrate the effectiveness of ATE viewpoint; however, a detailed explanation is necessary.

A1: We apologize for this typo. 'w/o VPS' refers to not using view point selection (VPS) but select a point along the normal vector with ε. w/o ATE-VPS refers to not using the ATE- view-vector-based view point selection (ATE-VPS), but randomly select a navigation vector from the potential navigation vectors. We will add these explanation to the revised paper.

Q2: ZECC contains numerous calculation steps and components such as ATE, NLBN, VPS, etc. It is suggested to include an algorithm chart for clarity.

A2: We provide the algorithm in Algorithm 1 and Algorithm 2. We will add these explanation to the revised paper.

Algorithm 1 Pseudo-Code of ATE

Require: Global distribution $D$, HAE map $M_H$, LAE map $M_L$, Prompt $I$

Ensure:

1: $t \leftarrow 0$;

2: $done1 \leftarrow False$;

3: $D \leftarrow None$

4: $M_H \leftarrow None$

5: $M_L \leftarrow None$

6: while not $done1$ do

7:   $O_t, p_t \leftarrow getState()$;

8:   $s_t \leftarrow MLLM(O_t; I)$;

9:   $M_H \leftarrow updateMap(O_t; p_t)$;

10:   if $s_t > 0.5$ then

11:     $toLAE()$;

12:     $done2 \leftarrow False$;

13:     while not $done2$ do

14:       $O_f, p_f \leftarrow getState()$;

15:       $d_f \leftarrow P(G(O_f), p_f)$;

16:       $M_L \leftarrow updateMap(O_f; p_f)$;

17:       $D \leftarrow updateDistribution(D, d_f)$;

18:       $done2 \leftarrow toUnexplored(M_L)$;

19:     end while

20:   end if

21:   $done1 \leftarrow toUnexplored(M_H)$;

22:   $t \leftarrow t + 1$;

23: end while

24: return $D$

Algorithm 2 Pseudo-Code of NLBN

Require: Cluster size $\epsilon$, Navigation vector degree $\zeta$, Navigation point range $\eta$, Density map threshold $\kappa$, Global distribution $D$

Ensure:

1: $D \leftarrow \text{ATE}(D)$;

2: $\{ \mathbf{x}_i^{\text{cluster}}, \ldots, \mathbf{x}_N^{\text{cluster}} \} \leftarrow \text{Cluster}(D)$;

3: $done \leftarrow False$;

4: for $i = 1$ to $N$ do

5:   $d_i^{\text{cluster}} \leftarrow \text{FitPlane}(\mathbf{x}_i^{\text{cluster}})$;

6:   $\mathbf{x}_i^{\text{ATE}} \leftarrow \text{getViewpoint}(D)$;

7:   Compute $\{ d_{i1}^{\text{view}}, \ldots, d_{im}^{\text{view}} \}_i$ using Eq. (4);

8:   Compute $d_i^{\text{ATE}}$ using Eq. (5);

9:   Compute $d_i^{\text{view}}$ using Eq. (6);

10:   Compute $\mathbf{x}_i^{\text{view}}$ using Eq. (7);

11: end for

12: while not $done$ do

13:   $done \leftarrow \text{toNextNaviPoint}(\{ \mathbf{x}_i^{\text{view}}, \ldots, \mathbf{x}_N^{\text{view}} \})$

14: end while

Q3: In the failure case study, the author mentioned that ATE is vulnerable in some occlusion cases. This may restrict performance of ZECC. A further discussion on this limitation would strength the paper.

A3: We believe that this is mainly due to the fact that current ATE lacks the ability to identify positions that can effectively observe positions relevant to targets, such as roads or squares, from a top-down view, resulting in navigation to irrelevant areas, such as rooftops. We will study this challenge in our future work.

Q4: Minor weaknesses: The authors should define the task names in Table 1. ‘ZSON’ in line 132 is not clearly defined when first time used. Also, the reference is missing.

A4: We apologize for this typo. ZSON is defined in [1]. We will add this definition to the revised paper.

Q5: In line 225, a space is missing between ‘Figure 2’ and ‘(b)’.

A5: We apologize for this typo. We will revise this part in the paper.

[1] Majumdar, Arjun et al. “ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings.” ArXiv abs/2206.12403 (2022): n. pag.

Q1: All evaluations are confined to the synthetic ECCD simulator.

A1: Thank you for your review to improve our work. We have provided a real-world application case in ‌Appendix 6‌ to test ZECC's capability in reducing occlusions in densely crowded scenarios. Besides, we provide results in another real-world scenario, which is denser compared to the existing one. The drone model used is the Taobotics Q300. The captured images are transmitted back to a ground server using a remote communication protocol and fed into VGGT [1] to estimate relative point clouds and poses. The proposed NLBN is then employed to calculate relative navigation points. The drone's absolute geographic coordinates and poses are recorded in real-time by its GPS. By leveraging the relationships between absolute and relative geographic coordinates and poses, the navigation points calculated by NLBN are mapped to absolute navigation points. After the ground server controls the drone to fly to these absolute navigation points, it captures images of the crowd. We utilize Grounding DINO and the crowd counting model Generalized Loss to perform crowd detection or counting on images captured from both distant and close-up perspectives, respectively. The ground truth for crowd annotations is manually labeled.

Q2: Please show the reasons for building embodied crowd counting instead of embodied small object detection. I think the latter is more valuable with location information.

A2: It is indeed that ZECC leverages the concept of small object detection. However, we think that crowds often appear with heavy density and occlusion, which makes conducting embodied tasks on crowds more challenging. These complex distributions require additional techniques like NLBN to generate fine viewpoints to gain accurate counting results. We will broaden our scope to general small object detection in our future work.

Q3: The crowd generator relies on a simple Poisson point process, which ignores structured formations common in real gatherings (queues, flows, social spacing).

A3: The current ECCD includes scenes that simulate real crowd distribution, such as queues and flows. As mentioned in ‌line 159‌ in the manuscript, the Poisson point process only determines the number and distribution of individuals ‌in a block‌, and the shape and distribution of the blocks are adjusted by human experts according to the environment semantics. In this way, crowds in street scenes appear to be in queues, and crowds in intersections appear to be in flows.

Q4: Comparative experiments omit recent multi-view crowd-counting methods ("Cross-View Cross-Scene Multi-View Crowd Counting," and "CountFormer: Multi-View Crowd Counting Transformer"), making it hard to judge the improvement of ZECC.

A4: Since no open source code for [2] is found, we compare with [3] using the same settings as MVC. The weight of the models are fixed during testing. The results are shown in Table 1. Although recent MVC methods show improvement, they still suffer from significant degradation when the number of cameras decreases. MVC methods cannot fundamentally solve the occlusion issue because of fixed cameras. This result illustrates the necessity of embodied crowd counting.

Table 1: Comparison with different MVC methods

Q5: ZSON appears without definition.

A5: We apologize for this typo mistake. ZSON is defined in [4]. We will add this definition to the revised paper.

[1] Wang, Jianyuan et al. "VGGT: Visual Geometry Grounded Transformer." Computer Vision and Pattern Recognition (2025).

[2] Zhang, Qi et al. "Cross-View Cross-Scene Multi-View Crowd Counting." 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021): 557-567.

[3] Mo, Hong et al. "CountFormer: Multi-View Crowd Counting Transformer." ArXiv abs/2407.02047 (2024): n. pag.

[4] Majumdar, Arjun et al. “ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings.” ArXiv abs/2206.12403 (2022): n. pag.

Dear Reviewer,

Thank you again for your time and effort invested in the evaluation of our paper. We would like to ensure that we have addressed all your concerns satisfactorily. If there are any additional feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you again for your time and effort in reviewing our paper.

The authors have been resolved my concerns. Although I think it is far away to real embodied application, the research is exploratory study. I revise my rating to positive.

Q1: Could the authors include statistical metrics.

A1: Thank you for the reviews to improve our work. We have updated the main experimental results with standard deviation in Table 1, Table 2, and Table 3. In each experiment, we generate three random starting positions within the navigable area. Additionally, the random seed is changed three times for each starting position, resulting in a total of nine trials for every experiment. The results show that the performance of ZECC is consistent.

Table 1: Statistical performances.

Table 2: MAPE under different crowd density

Table 3: TD under different crowd density levels

Q2: Could the authors provide visualizations or examples of failure modes.

A2: We provide a failure case in ‌Section 5 in the Appendix‌. It shows that a rooftop occludes the MLLM and cannot realize the crowd beside the building. This is mainly due to the fact that current ATE lacks the ability to identify positions that can effectively observe positions relevant to targets, such as roads or squares, from a top-down view, resulting in navigation to irrelevant areas, such as rooftops. We will study this challenge in our future work.

NLBN sometimes chooses suboptimal positions. In our experiments, this is mainly caused by the wrong detection results of the crowd counting model used to estimate the crowd distribution. It incorrectly identifies trees as a crowd in some cases, making NLBN generate navigation points near these target irrelevant areas. This can be solved by introducing a better target detection model to filter these cases or training a more robust crowd distribution estimator.

Q3: But how consistent are its decisions? Is it queried once per scene or at every step?

A3: We provide statistical metrics of ZECC, and from the results in Table 2, Table 3 and Table 4, it shows that the result is consistent. We believe this is because crowds can be easily identified during HAE in most cases, as the drone's top-down view provides a clear sight, enabling the MLLM to make consistent decisions.

The MLLM used in ZECC is used to decide whether to conduct LAE in each HAE step.

Q1: The method and experiments are entirely conducted in a synthetic environment (ECCD), without any real-world evaluation.

A1: Thank you for your review to improve our work. We have provided a real-world application case in ‌Appendix 6‌ to test ZECC's capability in reducing occlusions in densely crowded scenarios. Besides, we provide results in another real-world scenario, which is denser compared to the existing one. The drone model used is the Taobotics Q300. The captured images are transmitted back to a ground server using a remote communication protocol and fed into VGGT [1] to estimate relative point clouds and poses. The proposed NLBN is then employed to calculate relative navigation points. The drone's absolute geographic coordinates and poses are recorded in real-time by its GPS. By leveraging the relationships between absolute and relative geographic coordinates and poses, the navigation points calculated by NLBN are mapped to absolute navigation points. After the ground server controls the drone to fly to these absolute navigation points, it captures images of the crowd. We utilize Grounding DINO and the crowd counting model Generalized Loss to perform crowd detection or counting on images captured from both distant and close-up perspectives, respectively. The ground truth for crowd annotations is manually labeled.

The results in Table 1 demonstrate that the methods with NLBN perform significantly better than baselines, which illustrates the real-world generalization ability of our method. We will add these results to the revised paper.

Table 1: Performance before and after applying NLBN in the real scenes.

Q2: The method is primarily a composition of existing components.

A2: We agree that this work mainly focuses on system-level innovation; however, it also provides novel algorithms to address the occlusion issue and a height adjustment method to improve exploration efficiency. Also, we propose a new task, ECC, and a new platform, ECCD, that can inspire future research in this area.

‌Occlusion issues in crowds degrade the performance of detection and counting.‌ Occlusion is one of the key challenges in crowd counting, which requires an algorithm to actively adjust the viewpoint to mitigate this challenge. ‌To address this, the key component of the method, NLBN, is proposed to generate viewpoints according to the crowd distribution automatically.‌ It converts the difficult viewpoint selection problem into a relatively simple surface detection problem and generates viewpoints that reduce the degree of occlusion among the crowd. ‌Table 2 in the manuscript‌ shows that, when equipped with NLBN, ZECC significantly outperforms existing methods. ‌Table 1 in the Appendix‌ shows that NLBN reduces the obstruction rate of the line of sight, which contributes to the improved performance of ZECC.

‌A strategy utilizing height adjustment to improve exploration efficiency is proposed.‌ We explicitly consider the advantage of height adjustment in the exploration task. By using HAE, the agent obtains broad views, and irrelevant areas can be rapidly filtered out. This method mitigates redundant exploration caused by limited views at a fixed height and reduces the occlusion avoidance trajectory. ‌Figure 5 (a) in the manuscript‌ shows the efficiency improvement.

‌The system-level design serves as a foundation for future work.‌ ZECC not only serves as a baseline for further improvement but is also an automatic trajectory generation tool, which is important given the data scarcity in the embodied agent community [2]. Training-based methods can be developed based on ZECC to further improve performance.

‌A new task and dataset that supports this research.‌ ECCD is proposed, and it supports large-scale scenes and large object quantities simultaneously, enabling more complex algorithm designs.

Q3: Can the authors provide insights or preliminary evidence on real-world applicability?

A3: We show a real application case in response ‌A1‌. The challenge of real-world deployment lies in dataset generation and long-range UAV control. These challenges are being solved by the community gradually. [2] shows that 3D Gaussian splatting can be used to construct simulation scenes based on the real world, and [3] has provided possible solutions for real-world drone deployment in large-scale environments. We can use these solutions to extend this work.

We still would like to emphasize that this work mainly focuses on solving the occlusion issue in crowd counting and improving exploration efficiency by using height information. We have demonstrated the effectiveness of the proposed methods in both simulation and a real-world application case. Sim-to-real techniques indeed are important to the community, and we are trying to improve these aspects in future works.

Q4: How critical is the vision-language model (GPT-4V) to the success of ZECC?

A4: We conduct experiments on replacing GPT-4V with the open-source Qwen-VL-Max model and the tiny MiniGPT model. The results are shown in Table 2. It shows that the performance is consistent with different MLLMs. We believe this is because the images captured during HAE lack complex occlusion compared to LAE, making it easy to analyze, which results in consistent results for different MLLMs

Table 2: Performance of different MLLMs.

[1] Wang, Jianyuan et al. “VGGT: Visual Geometry Grounded Transformer.” Computer Vision and Pattern Recognition (2025).

[2] Gao, Yunpeng et al. “OpenFly: A Comprehensive Platform for Aerial Vision-Language Navigation.” (2025).

[3] Lou, Jiabin et al. “Air-M: A Visual Reality Many-Agent Reinforcement Learning Platform for Large-Scale Aerial Unmanned System.” 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2023): 5598-5605.

Dear Reviewer,

Thank you again for your time and effort invested in the evaluation of our paper. We would like to ensure that we have addressed all your concerns satisfactorily. If there are any additional feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you again for your time and effort in reviewing our paper.