PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer

We sincerely thank the reviewer for the positive assessment and thoughtful comments. We are glad you found the PANDA framework compelling and clearly presented. Below, we address each concern in detail.

Q1: Real-time performance concern.

We thank the reviewer for this detailed analysis and fully agree that real-time inference is vital for practical VAD systems. We provide a detailed clarification on the real-time performance below.

1. Measured throughput and drift analysis
   As shown in Supp. Tab. 1c, PANDA achieves the following average inference speeds across different datasets:

   Dataset	Average Inference Speed (FPS)
   UCF-Crime	0.82
   XD-Violence	0.86
   UBnormal	0.79
   CSAD	0.53

   Notably, CSAD is an extreme stress-test benchmark, composed entirely of visually degraded videos. This is not reflective of typical real-world anomaly distributions, but is used to evaluate robustness under worst-case conditions.
   Across the other three realistic and diverse datasets, PANDA achieves an average speed of 0.823 FPS, meaning each sampled frame is processed in approximately 1.22 seconds, resulting in a mild temporal drift under the 1 FPS sampling setting — as the reviewer insightfully observed.

   In practice, this level of delay is often acceptable for real-world anomaly detection, as abnormal events typically unfold over several seconds. PANDA is often able to raise an anomaly alert during the early stages of most anomalies, which satisfies the latency requirements of many surveillance and safety-critical scenarios. Moreover, we plan to further reduce PANDA’s computational cost by introducing multi-threaded tool pipelines and adopting lighter-weight LLM/VLM backbones in future work.

2. Comparison with existing training-free LLM/VLMs-based methods

   Real-time VAD using LLM/VLMs is inherently challenging due to their reasoning complexity. Existing SOTA training-free MLLM-based methods like LAVAD and AnomalyRuler involve manual intervention (e.g., pre-processing prompts or post-filtering predictions) and require offline access to global video information, making them unsuitable for online deployment**. PANDA, to our knowledge, is the first system to support online, training-free VAD in an agentic manner.

   To illustrate this comparison, we measured the average inference FPS on the UCF-Crime dataset with existing SOTA training-free LLM/VLM-based VAD baselines, all evaluated on the same hardware setup using a single A6000 GPU. As shown in the table below, PANDA outperforms previous methods by 6~10 in speed, while also supporting training-free, manual-free, and fully autonomous online detection.

   Method	Avg. FPS	Online
   LAVAD (CVPR 2024)	0.08	✘
   AnomalyRuler (ECCV 2024)	0.13	✘
   PANDA	0.82	✔

   Note: For LAVAD and AnomalyRuler, the inference process involves multiple sequential stages, where each stage must be completed before the next can begin. For example, LAVAD first generates captions for all video frames, then performs caption cleaning and summarization, followed by initial scoring and anomaly score refinement based on the captions. Therefore, in our evaluation, we report the average FPS across all inference stages to provide a fair and comprehensive comparison.

This discussion and table will be included in the revision.

Q2: Concerns regarding reliance on an anomaly-rule knowledge base, generalization to unseen anomalies, and rule number sensitivity

We sincerely thank the reviewer for these insightful questions. Below, we respond to each aspect in detail:

1. On the construction and cost of the anomaly-rule knowledge base

We would like to clarify that PANDA does not require any manual annotation or dataset-specific rule design. As shown in Supp. Fig. 1 (top), the anomaly knowledge base is automatically constructed at runtime based on the user’s specified anomaly detection needs.

• This process is driven by MLLMs (e.g., Gemini 2.0 Flash), using dynamically composed prompts based on the user requirement;
• For example, generating 20 rules for each of 13 anomaly categories on the UCF-Crime dataset took approximately 95 seconds in total — demonstrating low overhead and strong scalability, even when extended to hundreds of categories.

Importantly, PANDA does not rely rigidly on the initial rule base. It performs:

• Scene-aware knowledge retrieval in the perception stage, selecting only context-relevant rules;
• Dynamic rule augmentation during reflection, where new rules may be added or revised based on ambiguous cases and context.

This ensures that PANDA’s reasoning process remains adaptive, even when the initial knowledge base lacks coverage for the current anomaly type.

2. On generalization to unseen anomalies

Since PANDA is fully training-free, it starts with no pre-learned notion of known anomaly categories. The only guidance comes from user requirement at runtime.

Because anomaly is inherently context-dependent — the same action (e.g., “running”) may be normal in a park but abnormal in a hospital. PANDA reflects this principle and building the anomaly rule base on demand, according to the specific detection goal and scene context.

Moreover, PANDA’s adaptive mechanisms — self-adaptive scene perception, tool-augmented self-reflection, and chain-of-memory — allow it to handle anomalies not explicitly defined in the initial rule base.

To further verify this, we conducted an open-set evaluation on the UBnormal dataset:

• The anomaly rule base was constructed only using categories from the training set;
• The test set included completely disjoint anomaly types;
• As shown in the table below, PANDA’s performance only dropped ~1.5% AP, and still outperformed all SOTA methods — demonstrating robust generalization to unseen anomalies.

3. On rule number effect

We have analyzed this factor in Tab. 2(b) of the main paper. When too few rules are retrieved (e.g., k = 1), the system lacks diverse contextual cues to support robust reasoning, resulting in performance degradation. Conversely, setting k too high (e.g., k = 9) may introduce noisy or irrelevant rules that dilute reasoning quality. Finally, PANDA achieves optimal performance when setting k = 5.

Q3: Clarification on contribution scope beyond model reuse

Thank you for this remark. While it is true that PANDA leverages existing MLLMs and VLMs, we respectfully argue that its contribution lies in the novel agentic AI paradigm and system-level architecture designed to unlock training-free, manual-free, and generalist video anomaly detection — a capability not achieved by prior work.

1. Paradigm-level innovation

PANDA is not a naive wrapper over existing models. It introduces a deliberately designed four-stage reasoning stack —
Perceive → Reason → Reflect → Memory(Learn) — mimicking a human detective’s workflow. Each module equips PANDA with essential abilities:

• Self-adaption scene-aware strategy planning → addresses open-scene adaptation;
• Goal-driven heuristic reasoning → supports goal-driven, explainable, and reliable decisions;
• Tool-augmented self-reflection → enhances robustness under visual degradation;
• Chain-of-memory → enables temporal abstraction and learning from prior experience.

This transforms MLLMs from static responders into situationally aware, reflective agents that operate sequentially over video data — enabling a new class of VAD agents.

2. First to unify training-free + online VAD

PANDA is, to our knowledge, the first training-free MLLM-based VAD system that supports online detection, addressing a critical practical need. In contrast:

• ZS CLIP, LLAVA-1.5, LAVAD, AnomalyRuler — all require offline processing and manually tuned or learned prompts;
• None support self-reflection, dynamic rule retrieval, or temporal memory;
• None operate without scene-specific annotations or tuning.

3. System-level design as contribution

PANDA unifies RAG, tools, reasoning, and memory into a dynamic and autonomous agent, shifting from static pipelines to reasoning-driven learning. This design is non-trivial and central to PANDA’s contribution.

We will further emphasize these points in the revised Introduction and Section 3.

Q4: Typos.

We sincerely thank the reviewer for pointing out the typos. We will correct all typos such as "planing" → "planning" and "Kowleage" → "Knowledge". We will also thoroughly revise and proofread the entire paper and supplementary materials to ensure a polished and professional presentation in the final version.

Q5: Privacy and societal impact.

Thank you for the reminder. While PANDA does not collect personal identifiers, we acknowledge potential risks if used inappropriately (e.g., unauthorized surveillance, biased rules).

We will update the societal impact section to discuss:

• The importance of transparent and auditable rule design;
• Ethical deployment boundaries;
• Bias mitigation through automated rule auditing.

Dear Reviewer wQDY,

I would be grateful if you could kindly review the authors' rebuttal and let them know if you have any further questions or concerns. Your feedback will help ensure a constructive and productive discussion.

Best regards,

I thank the reviewers for their comprehensive response. My only concern is still about the FPS. I understand that this work has potential, and thus I would like to consider this to be further improved. While the authors are proposing solutions, they could respond with a more scientific approach, and discuss their steps towards achieving ~real-time speed.

Thank you for your thoughtful and critical review. We appreciate your recognition of the motivation and comprehensiveness of our proposed PANDA. We address your concerns in detail below.

Q1: Why we must use those modules to model the desired ability? Are they optimal? What are the other solutions?

1. Why those modules are necessary.

Thank you for the insightful question. We would like to clarify that the four modules in PANDA — namely: (1) self-adaptive scene-aware strategy planning, (2) goal-driven heuristic reasoning, (3) tool-augmented self-reflection, and (4) a self-improving chain-of-memory — are not arbitrary choices, but are deliberately designed to achieve generalist VAD that is fully training-free and manual-free even when facing novel scenes or anomaly types — a capability not supported by existing VAD methods. Next, we elaborate on the necessity of those modules below.

(1) Functional Necessity

Each module equips PANDA with a key cognitive capacity essential for generalist VAD:

• Self-adaptive scene-aware strategy planning (via RAG): Real-world anomalies are scene-dependent, and thus require context-specific rules. The RAG module lets PANDA dynamically retrieve only relevant rules from a anomaly knowledge base based on perceived context. Static prompts or fixed rules (used in prior work like VERA or AnomalyRuler) cannot generalize.

• Goal-driven heuristic reasoning: Existing methods that rely on manually constructed or tuned fixed prompt templates for MLLM reasoning are brittle and poorly scalable. PANDA learns to synthesize them adaptively from environment + user goals + rules.

• Tool-augmented self-reflection: When facing ambiguous or degraded input, PANDA “knows what it doesn’t know” and selectively invokes tools (e.g., object detection, web search). This reflects a meta-cognitive ability essential to generalization.

• Chain-of-Memory: Many anomalies involve temporal cues (e.g., shoplifting, riot). PANDA maintains both short-term reasoning traces and long-term historical experiences, allowing it to refine decisions across clips — which neither VERA nor other baselines support.

(2) Empirical Justification

• As shown in Table 2 (main paper), PANDA outperforms existing SOTA methods.
• Table 3 (main paper) demonstrates that the performance of PANDA improves consistently and substantially as each module is incrementally added.
• As addressed in Q2, even when compared to VERA — a recently proposed method trained with coarse-grained prior labels — PANDA still achieves superior results on the XD-Violence and CSAD datasets.

These empirical evidence indicates that these modules are not redundant but instead contribute orthogonally to generalization and robustness in VAD.

2. Are these modules optimal? What are the other solutions?

Thank you for the insightful question. While we do not claim these modules represent the “optimal” design, they provide a minimally sufficient and extensible structure to achieve training-free, manual-free, and generalist VAD across diverse benchmarks.

We acknowledge that these modules proposed in PANDA represent just one plausible design path toward generalist VAD. Another feasible solution is to design reinforcement learning (RL)-based agents — for example, using RL to train a policy that decides how to analyze videos (e.g., which tools to invoke or how to interpret anomaly signals). However, since real-world anomalies are rare and highly context-dependent, this poses significant challenges for reward function design and generalization to complex, dynamic environments.

In summary, while other solutions exist, we emphasize that PANDA offers a training-free, manual-free, and generalist design that balances flexibility and practicality. We believe it provides a compelling trade-off between generalization, interpretability, and ease of deployment in real-world VAD scenarios.

Q2: The paper lacks comparison with VERA (CVPR 2025), which uses learned prompts to achieve strong results without fine-tuning. Does this mean the proposed modules are unnecessary?

Thank you for raising this important point. We fully acknowledge that VERA is a strong, recent method in the MLLM-based VAD paradigm. However, we respectfully clarify several key distinctions and provide both theoretical and empirical evidence supporting the necessity of PANDA’s modular design:

1. PANDA is Fully Training-Free and Manual-Free — VERA Requires Dataset-Specific Training and Human Intervention

A core distinction is that PANDA is a generalist that is fully training-free and manual-free, requiring no training phase, no data annotation, no optimization of prompt parameters, and no human intervention. The entire pipeline is globally automated — from rule generation to detection — making PANDA highly suitable for deployment in novel environments with no prior data or supervision.

In contrast, VERA is a specialist model that requires:

• Coarsely labeled training videos (weakly-supervised);
• An optimizer-learner loop to iteratively tune prompt questions;
• Manual inspection for convergence and selection of prompt questions.

This makes PANDA significantly more lightweight and deployment-ready in unseen, zero-data environments, while VERA remains more aligned with weakly-supervised prompt learning and dataset-specific adaptation.

2. Evaluation Metric Consistency — Our AUC and AP on XD-Violence is Higher than VERA

VERA reports AUC on the XD-Violence dataset, while our main paper follows the original XD-Violence paper [HL-Net, Wu et al., ECCV 2020] in reporting AP, which is more robust for class-imbalanced datasets as highlighted by XD-Violence authors.

To ensure a fair comparison, we additionally report PANDA’s AUC on the XD-Violence dataset. Furthermore, using the official VERA codebase — including the released video features, initial scores, and AP computation scripts — we reproduce VERA’s AP on XD-Violence (which was not reported in their paper). As shown in the comparison table, PANDA outperforms VERA on both AUC and AP metrics. This demonstrates that PANDA’s modular design does not sacrifice accuracy despite being fully training-free and manual-free, whereas VERA requires training on coarsely labeled video data.

3. PANDA Outperforms VERA in Complex and Degraded Scenarios (e.g., CSAD)

We further evaluate VERA using its released codebase on our proposed CSAD benchmark — a complex-scene anomaly detection set characterized by poor lighting, low resolution, and long-range temporal anomalies.

As shown in the table above, VERA achieves only 64.52% AUC on CSAD, whereas PANDA significantly outperforms it with 73.12%. This highlights that prompt-based adaptation alone is insufficient for handling real-world degradation, where PANDA’s self-adaptive scene-aware, tool-augmented self-reflection, and chain-of-memory modules play a critical role in restoring visual clarity and improving reasoning confidence.

In summary, while we commend VERA as an elegant prompt-learning method, we emphasize that:

• PANDA is completely training-free, manual-free, and generalizable — capable of handling arbitrary new scenes and anomaly types, whereas VERA requires coarse-grained annotations and training on specific datasets;
• PANDA surpasses VERA on complex and unseen scenes without dataset-specific tuning;
• PANDA achieves superior accuracy on the XD-Violence under a fair comparison on the same evaluation metric;
• PANDA offers fully training-free, manual-free, and globally automated solution, which is essential for real-world generalist anomaly detection systems.

Finally, we thank the reviewer again for highlighting this important comparison. We will include detailed discussion on VERA in the revised version of our paper.

Q3: What's the additional computation overhead will be brought by these modules?

Thank you for pointing out this important concern. We agree that a discussion on computation cost is necessary, and now provide additional clarification and quantitative analysis.

1. Inference Speed Across Datasets

We have reported the inference speed of PANDA on each dataset in the supp. material (Section C.3, Table 1c):

Among them, CSAD is a purposefully extreme benchmark composed entirely of visually degraded scenes, and thus its speed reflects a worst-case condition. For the other three datasets — UCF-Crime, XD-Violence, and UBnormal — which represent real-world surveillance distributions and open-set scenarios, PANDA achieves an average speed of 0.823 FPS, which closely aligns with the 1 FPS processing rate and supports timely anomaly detection in practical deployments (1 FPS sampling settings).

2. Trade-off Justification

We acknowledge that PANDA introduces additional reasoning and reflection steps compared to fixed-prompt baselines. However, this is a conscious trade-off:

• PANDA trades marginal compute for robustness under uncertainty, scene adaptivity, and open-set generalization;
• Unlike training-dependent systems, PANDA avoids all training costs;
• Unlike VERA, PANDA does not require pre-optimized prompts or dataset-specific training to adapt.

We believe that this modest overhead is justified by PANDA’s real-world usability and improved generalization in complex and unseen scenarios. We also plan to reduce PANDA’s computational load and further improve its inference speed in future work by incorporating multithreaded tool invocation and adopting lighter LLM/VLM backbones.

Thank you for your thoughtful and detailed feedback. We appreciate your recognition of PANDA’s “detective agent” paradigm and its advantages of operating without training or manual prompt engineering. Below, we address each of your concerns point by point:

Q1: Inference latency and FPS concerns.

We respectfully clarify that PANDA’s actual speed is significantly higher than the initial estimate (< 0.1 FPS), and is broadly sufficient to meet the practical requirements under 1 FPS sampling settings, which aligns with our experimental setup.

As shown in the supplementary material (Sec. C.3, Tab. 1c), the measured average inference speed across datasets is:

Among them, CSAD is a purposefully extreme benchmark composed entirely of visually degraded scenes, and thus its speed reflects a worst-case condition. For the other three datasets — UCF-Crime, XD-Violence, and UBnormal — which represent real-world surveillance distributions and open-set scenarios, PANDA achieves an average speed of 0.823 FPS, which closely aligns with the 1 FPS processing rate and supports timely anomaly detection in practical deployments (1 FPS sampling settings).

Importantly, PANDA is a training-free and manual-free VAD framework that supports both online and offline inference, while existing SOTA training-free LLM/VLM-based methods (e.g., LAVAD, AnomalyRuler) rely on multi-stage processing involving manual prompt design, external visual parsing, or result aggregation, making them unsuitable for online operation.

To illustrate this comparison, we measured the average inference FPS on the UCF-Crime dataset with existing SOTA training-free LLM/VLM-based VAD baselines, all evaluated on the same hardware setup using a single A6000 GPU. As shown in the table below, PANDA outperforms previous methods by 6~10 in speed, while also supporting training-free, manual-free, and fully autonomous online detection.

Note: For LAVAD and AnomalyRuler, the inference process involves multiple sequential stages, where each stage must be completed before the next can begin. For example, LAVAD first generates captions for all video frames, then performs caption cleaning and summarization, followed by initial scoring and anomaly score refinement based on the captions. Therefore, in our evaluation, we report the average FPS across all inference stages to provide a fair and comprehensive comparison.

We will include these clarifications and the comparison table in the revised version.

Q2: The novelty is limited; the core idea is to replace the prompts in existing MLLM-based VAD methods (such as SUVAD, LAVAD, VERA, etc.) with RAG and chain-of-thought techniques.

We respectfully clarify that PANDA is not a simple replacement of prompts with RAG and CoT techniques, nor an incremental extension of existing VAD methods. Instead, it is a fundamentally new agentic AI framework, built from scratch using LangGraph, to enable generalist VAD — meaning it can automatically adapt to new scenes and novel anomaly types in a training-free and manual-free manner. In contrast, existing methods typically require retraining or manual intervention when faced with such open-world scenarios.

Specifically, PANDA introduces a paradigm-level shift from training or manual dependence to training-free, manual-free, and generalist paradigm. It is built as a detective-like agent and is empowered with the following cooperating abilities:

• Self-adaptive scene perception with environment-aware rule retrieval;
• Goal-driven heuristic reasoning that aligns anomaly rules with real-time vision-language observations;
• Tool-augmented self-reflection, allowing the PANDA to detect uncertainty and improve its own perception pipeline dynamically;
• Chain-of-memory experience alignment, enabling context tracking across sliding windows and continue learning from history experience;
• Online processing ability, where the system makes decisions sequentially without accessing future frames — whereas existing methods, such as SUVAD, LAVAD, and VERA, require access to the entire video for global context and can only operate in an offline manner.

In summary, these prior works like SUVAD, LAVAD, and VERA rely on handcrafted or fine-tuned prompts and equire offline access to global video information, they lack four critical capacities:
① self-adaptive scene perception, ② self-reflection under degradation, ③ long-term memory across temporal sequences, and ④ online processing — all of which are essential for generalist VAD in complex, real-world settings.

We believe PANDA is the first agentic VAD framework that not only performs well across open-set and degraded scenes, but also does so fully autonomously, without training or handcrafted tuning.

Q3: Model hallucinations concern.

Indeed, hallucination remains one of the most recognized challenges when deploying large models (LLMs/VLMs), especially in lengthy reasoning process. In PANDA, we fully acknowledge this concern and have explicitly designed several key components to mitigate hallucination propagation throughout the pipeline. We elaborate below:

1.Scene-Aware Planning with RAG Mitigation

During the strategy planning phase, PANDA integrates scene perception with RAG. Instead of relying on free-form hallucinated outputs, the system uses explicit environmental cues (e.g., scene type, weather condition, potential anomalies) to retrieve relevant anomaly rules from a curated knowledge base. This constraint dramatically reduces the likelihood of hallucinated, scene-irrelevant planning responses.

2.Goal-Driven VLM Reasoning with Multi-State Output

During Reasoning, PANDA explicitly passes potential anomaly types, inferred from scene-aware planning, into the reasoning prompt — thus narrowing the decision space. Moreover, instead of forcing binary outputs (“normal” vs. “anomaly”), we introduce an "Insufficient" reasoning state, allowing the model to abstain from uncertain predictions rather than hallucinate a confident but incorrect answer.

3.Tool-Augmented Reflection to Ground Ambiguities

If a clip is deemed “insufficient” by the VLM, PANDA triggers tool-augmented self-reflection, which includes reliable enhancement tools (e.g., image deblurring, brightness enhancement, object detection) that provide objective, grounded information. These tools act as external signal correctors, significantly reducing the reliance on speculative internal reasoning.

4.Chain-of-Memory for Decision Consistency

PANDA maintains both ShortCoM and LongCoM memory. These modules serve as explicit external memories, enabling the system to reference prior decisions and reflection rationales — reducing the chance of making inconsistent judgments across similar cases, a common hallucination symptom.

Overall, these mechanisms above enable PANDA to effectively mitigate model hallucinations, as further supported by the component ablation study in Table 3 of the main paper. Nonetheless, hallucination remains a stubborn challenge for large models. Despite our efforts to alleviate it through multiple strategies, we still observed occasional hallucination cases during the final experiments. We will add hallucination case studies in the revised version to illustrate remaining failure cases and reflect on potential future improvements. Notably, despite these challenges, PANDA — as a training-free and manual-free generalist model — still significantly outperforms existing methods across diverse scenario datasets, as shown in Table 1 (main paper), indicating its overall reliability and effectiveness.

Q4: Missing implementation details — RAG and tool invocation.

Thank you for pointing this out. We appreciate the opportunity to clarify the technical implementation behind PANDA's core modules, and we will add further elaboration in the revised version.

1. RAG Details

PANDA encodes both the pre-built anomaly rule base and environment information using the all-MiniLM-L6-v2 sentence embedding model.

• The rule base is indexed using FAISS for fast similarity retrieval.

• During the scene-aware planning phase, the environment information is embedded and used as a query to retrieve the top-k most relevant rules, which are then fed into the MLLM for reasoning.

2. Tool Invocation Details

PANDA integrates a modular toolset, where all tool functions are pre-defined with their input/output formats and functional descriptions. These descriptions are made available to the MLLM during the reflection stage — as illustrated in Supp. Fig. 6 ({tool_description_text}).

During reflection, the MLLM considers the VLM’s output, historical context, and available tool descriptions to determine whether additional information is needed. It then outputs a structured field tools_to_use, specifying:

• The tool name to invoke

• Optional parameters (e.g., query strings)

Each tool name directly maps to a callable function in PANDA’s toolset. Finally, PANDA parses this tools_to_use field and executes the corresponding tool function with the provided parameters.

We thank the reviewer for their insightful comments and thoughtful questions. We are pleased that you found PANDA’s design, scope, and generality valuable. Your excellent rating on the quality of our work is truly encouraging and deeply appreciated. Below we respond point-by-point to the concerns raised.

Q1. How does PANDA handle unresolved “insufficient” cases after the maximum 3 reflection rounds?

As stated in the main paper (L196–197):

“If after r rounds the result is still Insufficient, PANDA skips the current segment and continues to the next timestep.”

Thus, if a segment remains “Insufficient” after the maximum r = 3 reflection rounds, PANDA assigns a default anomaly score corresponding to the "Insufficient" status. This strategy is a deliberate design choice to balance computational efficiency with performance stability.

As shown in Tab. 2(a) of the main paper, increasing r from 3 to 5 brings minor gains, resolving some hard cases but significantly increasing inference time, making r = 3 a reasonable trade-off in practice.

Q2. Why PANDA outperforms other offline methods in online mode?

We agree that a method typically performs better in the offline setting due to access to future information. As shown in Tab. 1 (main paper), PANDA’s offline mode consistently outperforms its online counterpart. However, due to fundamental design differences, online and offline methods are often not directly comparable — a well-designed online system may outperform rigid offline ones.

Our agentic AI framework, PANDA, is deliberately designed to function robustly in the online setting, and notably, it outperforms existing training-free offline methods, which we highlight as one of our key contributions. Below, we further explain PANDA’s advantage by analyzing the capabilities enabled by its core components.

• Self-adaptive scene perception: PANDA uses RAG to retrieve context-aware anomaly rules based on current scene information (e.g., scene overview, potential anomalies), aligning reasoning goals with environment.
• Goal-driven heuristic reasoning: PANDA performs CoT-style planning guided by retrieved rules, enhancing grounding between task objectives and VLM predictions.
• Tool-augmented self-reflection: Upon insufficient reasoning confidence, PANDA autonomously invokes enhancement tools (e.g., object detection, image deblurring) to improve clarity before re-reasoning.
• Chain-of-memory: PANDA leverages memory mechanisms (ShortCoM and LongCoM) to accumulate prior context, enabling temporally consistent decisions in streaming settings.

These mechanisms enable PANDA to make robust anomaly detection — even without access to future frames — much like a human expert acting with present information and past experience. In contrast, SOTA methods like LAVAD and AnomalyRuler rely on static prompts and lack adaptation, reflection, or memory, limiting their reasoning robustness even in offline mode.

Q3. Statistics about reflection mechanism.

Thank you for this detailed suggestion. We analyzed PANDA’s reflection behavior and summarize key statistics below (full results will be added to supp. material).

1. Frequency of reflection

As shown in the table above, the average reflection trigger rate across all four datasets is 19.0%. Notably, the CSAD dataset exhibits a trigger rate nearly 2–3 times higher than the other three. This aligns with our expectations, as CSAD is composed entirely of complex scenes, where PANDA is more likely to encounter uncertainty during reasoning and therefore invokes reflection more frequently.

2. Transition outcomes

Among clips initially classified as “Insufficient”, their transitions after each round are as follows (UCF-Crime dataset):

This shows that over 87.6% of uncertain clips are resolved within 3 rounds. We also observed diminishing returns after the second round, which supports our default choice of r = 3 as a trade-off between effectiveness and computational overhead.

Q4. Fig. 2 lacks clarity in reflection flow.

Thank you for this constructive feedback. To improve clarity, we will revise Fig. 2 in the following ways:

• Explicitly depict the “Insufficient” reasoning path, with a clearly labeled arrow from the VLM block to the MLLM reflection module;
• Visually separate normal reasoning and reflection flows via distinct line styles;
• Number key steps (1–6) to clearly present PANDA’s sequential reasoning process: perception → planning → initial reasoning → reflection trigger → tool-enhanced retry → memory update.

Q5. How does LongCoM behave at cold-start (no history)?

Thank you for the insightful question. At the start of a video, LongCoM is empty by design, and PANDA relies on ShortCoM’s local window memory for initial reasoning and reflection. As more clips are processed, LongCoM gradually accumulates traces, supporting memory-consistent reasoning and reflection planning. This progressive warm-up is intentional to ensure robustness under cold-start conditions. We will clarify this in Sec. 3.4.

Q6. L241–242 ambiguity.

Thank you for catching this. We will revise this sentence in the final version. The correct statement should be:

“In offline reasoning mode, the perception phase is sampling M = 300 frames uniformly for the whole video, while only the initial M = 10 frames are sampled in online mode.”

Q7. Tool selection.

As detailed in Sec. 3.3 of main paper and Supp. Fig. 1, PANDA selects tools based on two signals:

• VLM reasoning feedback: When the VLM returns “Insufficient”, it provides a natural language rationale (e.g., “too blurry”, “too dark”), which helps identify the root cause.
• Memory-guided experience: PANDA retrieves similar past “Insufficient” cases and their successful tool usage from LongCoM, informing effective tool reuse.

These cues are combined into a reflection prompt, along with all tool descriptions. The MLLM then selects the most appropriate tools, which are invoked accordingly. We will clarify this more explicitly in the final version.

Q8. Is the anomaly knowledge subjective to each dataset?

We would like to clarify that PANDA’s anomaly knowledge base is not subjective to specific datasets, but is dynamically built based on:

• User-defined anomaly categories
• Scene context perceived at test time.

Given anomaly is context-dependent, PANDA embraces this by adapting the rule base per instance:

• It first constructs a candidate rule set based on the user’s goals;
• Then, during the scene perception stage, it retrieves context-relevant rules using environmental cues (e.g., scene overview, potential anomalies);
• During reflection, it further refines or expands rules based on reasoning ambiguities or failures.

This design ensures the knowledge base is context-grounded, adaptive, and generalizable, rather than statically tailored to any dataset — enabling open world anomaly detection.

Q9. Missing ShortCoM in Supp. Fig. 1.

Thank you for pointing this out. While ShortCoM is implicitly integrated into the construction of both the Reasoning Input and Prompt and the Reflection Prompt, we agree it is not explicitly shown in Supp. Fig. 1. We will revise the Supp. Fig. 1 to clearly depict ShortCoM.

Q10. Supp. Fig. 5, where does the “{formatted_enhancement_prompt} come from?

Thank you for raising this point. {formatted_enhancement_prompt} refers to the enhanced prompt generated after the reflection stage, as described in Lines 148-152 and 188–193 of the main paper. It integrates three components: {Text Enhancement Info, New Anomaly Rule, New Heuristic Prompt}, where Text Enhancement Info summarizes the outputs of invoked tools (e.g., object detection, web search). We will clarify this in the caption of Supp. Fig. 5.

Q11. PANDA VS "Generative cooperative learning for unsupervised VAD (GCL)" (CVPR 2023)

Thank you for highlighting this reference. PANDA differs from GCL in both its goal and capabilities. While GCL focuses on cooperative training and learning from unlabeled videos, PANDA is designed to generalize to unseen scenes and anomaly types without any training, which GCL cannot handle due to its training dependency. Specifically:

• PANDA adopts an agentic paradigm with scene perception, rule-based retrieval (RAG), heuristic reasoning, and tool-augmented reflection, enabling zero-shot and training-free VAD.
• In contrast, GCL requires fixed model structures and offline training on large unlabeled data, limiting adaptability when deployed in open-world, complex environments.

Quantitatively, on the shared evaluation benchmark UCF-Crime, PANDA achieves an AUC of 84.89%, significantly outperforming GCL’s reported 71.04%, while training-free and zero-data requiring. We will add this comparison to the revised version.

Q12. Failure case analysis.

Thank you for the helpful suggestion. We will add a failure case visualization in Supp. Sec. C.4.

The case from UCF-Crime (Shoplifting033_x264.mp4) involves a man discreetly steals a watch from a store table. Due to low resolution and subtle visual cues, PANDA fails to detect the anomaly — even after 3 reflection rounds, the clip remains “Insufficient” status and is skipped.

This case shows that PANDA may struggle with fine-grained, low-visibility anomalies. We will explore stronger visual enhancement tools and finer-grained reasoning to address such cases in future work.

12. Keeping in view that user has to set a query containing the domain knowledge and type of anomaly he is expecting to see, your claim becomes null and void: "Therefore, we aim to achieve generalist VAD, i.e., automatically handle any scene and any anomaly types without training data or human involvement." There is human involvement therefore it is not all automatic. Please explain.

13. However, you can overcome this objection by using a fixed user query for all datasets and all videos. I was not able to find this result in the paper or supplementary document or in the rebuttal. If you were optimising user input at video level as in CSAD dataset, then performance improvement is expected.

14. Please disclose all 100 video names in CSAD dataset and the corresponding user input for each video.

15. A theoretical explanation how PANDA can handle long term dependencies is required.