Test-time Correction with Human Feedback: An Online 3D Detection System via Visual Prompting

1. The rationale behind this task setup requires further discussion. Since the proposed task involves human feedback, this process is often uncontrollable, making it challenging to ensure real-time performance. This limitation affects the feasibility of applying the task in real-world autonomous driving scenarios.
2. The rationale behind the EDS evaluation metric requires further discussion. Classification accuracy is also crucial for autonomous driving, and focusing solely on localization performance while neglecting classification performance is not realistic.
3. The proposed method is only applicable to vision-based autonomous driving solutions, limiting its generalizability to LiDAR-based autonomous driving systems.

1. The proposed can only handle missing objects. It seems it cannot reduce the FP in test time.
2. Although the authors explain the differences between proposed TTC and single object tracking, the explanation is unconvincing. The visual and object prompt can be easily used in SOT setting. Such discussion and at least comparisons with bbox annotations are must. This is the key concern in my evaluation.

1. Re-entering the regions of undetected targets as visual prompts introduces human knowledge, which may lead to potential biases and affect the fairness of comparative experiments.
2. The TTC method needs to maintain a buffer of visual cues and solve the matching problem between cues and target objects, which increases the complexity.
3. The experimental section lacks a description of how the visual prompts used during testing are obtained.

The paper lacks self-containment. For example, in lines 217-231, where the authors describe various visual prompts, they mainly reference numerous other methods without offering sufficient detail. This heavy reliance on external sources renders the paper somewhat incremental, as it fails to clearly articulate the novel contributions and context of the visual prompts within the proposed framework. Furthermore, this lack of clarity results in the use of many notations, such as "visual features" and "image features," without providing clear definitions.

Rather than referring to it as "visual prompts," the pipeline developed in this paper essentially provides a template containing location and size information in the buffer, enabling generic tracking during test time without any additional training. Therefore, the authors are encouraged to clarify whether this pipeline fundamentally differs from a single-object tracker. Additionally, it would be beneficial to include an experiment comparing state-of-the-art (SOTA) trackers for test-time correction as part of the evaluation

1. Motivation

Pretrained static 3D detection modules have clear limitations due to issues like domain gaps, and I agree with the goal of improving these. However, I find it difficult to fully empathize with the motivation behind TTC. If issues in the predictions of the 3D detection module indeed pose a significant impact on safety, as stated, would there realistically be an opportunity to perform online correction in such scenarios?

Additionally, I am skeptical about the feasibility of interventions like visual prompting during driving. Operating devices such as navigation systems manually while driving is likely a legal violation in most countries, and in practice, the difficulty level for performing such tasks during driving seems exceedingly high.

2. Comparison with TTA or TTT

In this field, there are various approaches for online improvement of pre-trained static models, such as test-time adaptation (TTA) and test-time training (TTT). Notably, most of these methods function without the need for human feedback. A thorough methodological and performance comparison with these approaches is essential. Additionally, while TTT may be somewhat challenging, in the case of TTA, it seems feasible to utilize human feedback as a direct learning guidance. I would appreciate a more in-depth discussion on this aspect as well.

3. Robustness

It is unrealistic to expect that user corrections will always be accurate. Depending on the situation, incorrect user interventions could potentially worsen the proposed TTC. It would be beneficial to model the noise that might exist in visual prompting and demonstrate that TTC can operate robustly even when this noise becomes more pronounced.

1. OA is one of the core modules of the model, serving as a bridge between prompts and offline-trained 3D detectors. However, the explanation of OA in the method section is somewhat abstract; adding simple illustrative diagrams could aid understanding.
2. In the Related Work section, the Online 3D Detection System subsection discusses online 3D detectors. Expanding on offline 3D detectors would help readers better understand the development of offline versus online 3D detection.
3. There are some minor typos in the text that need correction.