Tri-MARF: A Tri-Modal Multi-Agent Responsive Framework for Comprehensive 3D Object Annotation

W1. Poor presentation

A1: We have thoroughly revised the format and layout of the entire paper in the revised version. Specifically, we have proofread and corrected all mathematical equations, especially the formatting errors in Section 3.2.2, to ensure that they are clear and accurate. We have restructured the content starting from Page 6, splitting the original paragraphs that mixed multiple topics into independent sub-paragraphs with clearer logic. In addition, all performance charts in the Appendix (such as Figures 7-13, etc.) have been regenerated to provide higher resolution and better readability. To make evaluation indicators clear enough, we have added a dedicated section in the revised appendix to provide clear mathematical definitions and equations for all key evaluation indicators. We believe that these corrections have significantly improved the overall quality of our paper.

W2. Limited technical contribution

A2: The core innovation of our work is to design a multi-agent collaborative architecture. To highlight our contribution, we conduct a dedicated ablation study in Section 7 (Pages 15-16). Table 4 directly compares our Tri-MARF with a strong baseline that uses the exact same VLM (Qwen-2.5 VL) but relies on only a single agent. The results show that our multi-agent collaborative architecture brings a +7.3 percentage point increase in CLIPScore. This decisively proves that the superior performance of our Tri-MARF stems from the tight and effective synergy between the agents.

W3. Unfair experiment settings

A3: No, our experimental settings are fair.

• To compare against published SOTAs (like Cap3D, ScoreAgg), we follow the original configurations reported in their papers or use their released codes. Forcibly integrating the latest VLMs into these SOTAs is a non-trivial task and might introduce other variables.
• To demonstrate that our Tri-MARF's performance is not dependent on a specific VLM, we conduct a comprehensive performance and cost evaluation in Figure 6, Section 11.1 (Page 21), where we replace the VLM in our Tri-MARF with several industry-leading models (including GPT-4.5, Claude-3.7, Gemini-Flash-2.0, etc.). The results show that Qwen2.5-VL represents the optimal balance between performance and cost, not that its performance far exceeds other models.

W4: Concrete math representations.

A4: We have addressed this in the revised paper by adding a new subsection adjacent to Appendix C.1 dedicated to providing rigorous mathematical definitions for our core evaluation metrics. For Uncertainty Accuracy ($UA$), we have introduced a classification-based equation, such as $UA = \frac{TP_{\text{uncertain}}}{TP_{\text{uncertain}} + FP_{\text{uncertain}}}$, to clearly define its calculation. For Calibration Error ($CE$), we have explicitly included the standard mathematical equation for the Expected Calibration Error (ECE) variant, namely $\sum_{m=1}^{M} \frac{|B_m|}{n} |\text{acc}(B_m) - \text{conf}(B_m)|$, detailing the process of binning and the calculation of the difference between confidence and accuracy. Finally, for Dynamic Adaptability ($DA$), we have elevated it from a descriptive proxy ("correlation between... signals and... mechanisms") to a formal statistical measure via the Pearson correlation coefficient, to precisely quantify the relationship between system uncertainty and the activation of deeper reasoning mechanisms like debate triggering.

Q5. In Conf(C) Eq. (1), why do lower values represent higher confidence?

A5: This score is mathematically defined as the Average Negative Log-Likelihood. Our paper explicitly states on Page 5, Line 160, that this confidence score "quantifies the semantic reliability through the average token log-likelihood." Its calculation method is detailed in Eq. (1) (Page 5). Since the generation probability P(ti) of a token is between 0 and 1, its logarithm, log(P), is negative. Therefore, taking the absolute value is mathematically equivalent to calculating its Negative Log-Likelihood (NLL), i.e., -log(P). The higher the probability of an event (i.e., the more "confident" the model is), the lower its negative log-likelihood value. Thus, as our paper directly states on Page 5, Line 166: "a lower Conf(C) value indicates higher confidence (i.e., higher token probabilities)." To resolve this clarity issue, we have renamed it in the revised paper to "Avg. NLL Score"**.

Q2. Inconsistent symbols.

A6: We sincerely apologize for this confusion. S_conf,i (Line 214) does refer to the confidence score calculated for the i-th candidate description using Eq. (1). Similarly, w_i (Line 216) is indeed the same variable as w_v,i defined in Eq. (3), which is the weight calculated based on CLIP. In the revised paper, we have conducted a thorough proofreading of the entire paper to unify all mathematical symbols, e.g., by consistently using Conf(C_{v,i}) and w_{v,i}, to ensure the rigor and clarity of our paper.

Q3. Does the balanced score's alpha change across datasets?

A7: NO! We have conducted a detailed sensitivity analysis of this hyperparameter in Section 11.5 (Figure 8, Page 24) and show that our Tri-MARF performs optimally on the large and diverse Objaverse-XL dataset when alpha is set to 0.2. In all the experiments, we keep alpha fixed at 0.2 without any re-tuning.

Q4: Line 250 is confusing.

A8: This is indeed a heuristic design, but it is supported by our unique prompting strategy. Our VLM Annotation Agent (Agent 1) employs a structured, multi-turn dialogue strategy, as detailed in Section 3.1 (Page 4). Agent 1 does NOT generate a long description all at once. Instead, Agent 1 responds to "What is this object? What is its specific name?" and further follows up with questions about attributes like color and material. This general-to-specific questioning manner naturally guides the VLM to first summarize the object's core identity in the initial sentence of its response, with subsequent sentences providing detailed elaboration. Therefore, extracting the first sentence as the "core description" is to ensure that the final global description has a clear and concise topic sentence, which is then enriched with details provided by other viewpoints.

Q5. Use similar/comparable pre-trained backbones/VLM/LLMs.

A9: We must clarify that some of the core baseline methods do not use a VLM in their setup. Despite the inherent architectural differences with some baselines, our experimental design is rigorously justified from two perspectives to ensure fairness, and we have added comparisons with other VLM-based methods. Firstly, to demonstrate the independent value of our Tri-MARF, we conduct a crucial ablation study (Table 4, Supplementary) comparing the full framework against a single-agent baseline using the same VLM. The results show that our Tri-MARF provides a +7.3 percentage point increase in CLIPScore, proving that the performance gain comes primarily from the synergistic mechanism, not the VLM. Secondly, to demonstrate the fairness of our VLM selection, we have evaluated multiple industry-leading VLMs (Figure 6, Supplementary).

Q6. Why are human annotations worse?

A10: This is due to the two inputs being fundamentally different (Section 4.2), i.e., the human answers are "multiple-choice," while our Tri-MARF's output is "free-text." Specifically, this experiment is to evaluate the accuracy of type annotation, and annotaters choose from a pre-set, limited list of options. If the most accurate answer is not in the list, they have to select a semantically broader or not perfectly fitting approximate answer. In contrast, our Tri-MARF generates a rich and detailed natural language description. GPT-4o, acting as the judge, has powerful semantic understanding capabilities and can recognize synonyms or more precise hyponyms. For example, when the ground truth is "cup," the human's options might only include the general term "container," but our Tri-MARF might generate "a white ceramic coffee mug." GPT-4o can accurately determine that "coffee mug" is semantically much closer to "cup" than "container" is, thus giving our Tri-MARF a higher score. Therefore, this result precisely validates the precision and richness of the descriptions generated by our Tri-MARF.

We have revised Section 4.2 to more clearly explain the evaluation mechanism of this scoring task, explicitly pointing out the difference between "multiple-choice" and "free-text" input formats and their impact on the final score.

Q7: Table 3 is not consistent.

A11: This discrepancy arises from measuring two different performance metrics: Latency versus Throughput. Specifically, Table 3 (Page 15) reports the end-to-end latency of 7-19 seconds for processing a single object on a single GPU. This time is predominantly driven by the serial network latency of remote API calls to the VLM. In contrast, the 12k objects/hour claimed in Section 4.1 (Page 7) is the aggregate throughput of our entire annotation process. As mentioned on Page 15, we achieve extremely high processing efficiency at a macro level by parallelizing thousands of independent API call requests across multiple machines and GPUs in our large-scale annotation process. This throughput metric is also the standard for efficiency comparison with baseline methods, as shown in the "obj/PGCHour" column of Table 1. Ours (12k/hour) is significantly superior to methods like Cap3D (8k/hour) and ScoreAgg (9k/hour).

Q12: Limitation Section.

A12: We have expanded it in the revised paper by adding a more in-depth discussion on potential negative social impacts. Specifically, we explicitly state that our Tri-MARF is designed to provide high-quality annotations for beneficial fields like autonomous driving and robotics. Thus, it could potentially be misused to generate misleading or biased descriptions of real-world scenes or objects.

Q1: 1) Are standardized/canonical poses assumed for the 3D models? 2) Are there mechanisms to handle symmetric or cylindrical objects?

A1: Regarding viewpoint standardization, our processing pipeline assumes a standardized object pose. During the data preparation stage, we normalize the 3D models by aligning their principal axes with the coordinate system. This is a standard preprocessing step to ensure we consistently render from six standardized orthogonal viewpoints.

The issue of potential information redundancy from symmetric objects highlights the advantage of our Information Aggregation Agent (Agent 2), particularly its adaptive selection mechanism based on the Multi-Armed Bandit (MAB). Unlike existing works use fixed rules (like simple concatenation or averaging), our MAB agent learns to dynamically evaluate the information value provided by each viewpoint. For a symmetric object, if multiple views generate similar or redundant descriptions, our MAB learns during its training process to lower the weight of these redundant "arms" (i.e., descriptions) and prioritize those that offer unique information. Thus, our Tri-MARF has an intrinsic robustness to handle such cases rather than treating them as experimental outliers. We have made the preprocessing step more explicit with more details in the revised paper.

Q2. Are LLMs predisposed to producing the first sentence of an output sequence? Is this an induced behaviour? via in-context learning?

A2: This is indeed a heuristic design, but it is an effective choice based on observations of LLM's behavior and our unique prompting strategy. Our VLM Annotation Agent (Agent 1) employs a multi-turn, structured dialogue strategy. We first ask the model to identify the object ("What is this?") and then follow up with questions to obtain specific attributes like color and material (Refer to Figure 2 for more details). This general-to-specific questioning approach, guided by prompt engineering, naturally leads the LLM to summarize the object's core identity in the first sentence of its response, with subsequent sentences providing detailed elaborations. This is a stable behavior of the LLM that we have observed and guided through prompt engineering, not a blind trust in the model.

In our Tri-MARF, this "core sentence" is primarily used to construct the beginning of the final description, ensuring it has a clear topic sentence. The rich details provided by other viewpoints are then appended to form a complete and comprehensive global description.

Q3. Justify the use of CLIPScore to measure performance

A3: Our Tri-MARF does not rely solely on CLIPScore, neither in its internal mechanism nor in its external evaluation. Firstly, in the core relevance weighting module of our Tri-MARF, we use a hyperparameter α to balance the VLM's own confidence with CLIP's visual alignment, which jointly form the final description score $s_i$. This ensures the model is not designed merely to optimize for CLIP similarity. More importantly, we employ a diverse set of complementary metrics in the final performance evaluation. As shown in Table 1, we also include ViLT R@5 retrieval accuracy and human A/B test scores. These results strongly prove that its performance advantage is comprehensive and genuine, not a case of "overfitting" to a single metric.

Secondly, we chose to include CLIPScore since it has become the de facto standard in the field for evaluating image-text semantic alignment, as used by Cap3D, ScoreAgg, etc.

Q4. The doubts about the A/B test scores (Table 1).

A4: This is a misunderstanding of how the results are reported; the correct interpretation is the opposite. In our A/B tests, Tri-MARF is set as the fixed reference baseline. The A/B score listed under another method (e.g., for Cap3D) represents the average preference score given to our method by human evaluators in a pairwise comparison against that method (based on a 1-5 scale, where 3 is a tie).

The correct way to read the score is as follows: the 3.3 score under Cap3D means that in a direct comparison with Cap3D, human evaluators preferred our Tri-MARF, giving it an average score of 3.3. Since all baseline methods have an A/B score greater than 3, this indicates that our Tri-MARF is preferred in all pairwise comparisons.

To completely eliminate any ambiguity, we have followed your suggestion and redesigned the A/B test results table in the revised paper to explicitly show the win/loss/tie ratios, and we have rewritten the caption in detail to make the results clear at a glance.

Q5. Why does the gated agent only perform filtering and not try to self-correct to reduce manual labeling?

A5: Our current design is primarily based on considerations of efficiency and modularity. The Gating Agent (Agent 3) plays a lightweight final verification role in our Tri-MARF, whose main task is to quickly filter out hallucinated descriptions generated by the VLM that are clearly inconsistent with the 3D geometry, thereby forwarding only the most questionable samples to human reviewers. This ensures the high throughput of the entire process (12,000 objects/hour). Introducing an automatic self-correction loop would significantly increase system complexity and processing latency. This would be contrary to our design goal of achieving high efficiency.

Q6. Separate ablations for each agent and display the A/B score or VilT retrieval score for each newly added agent.

A6: To highlight the contribution of our Tri-MARF, we have designed and added a step-wise ablation study in the revised paper to demonstrate the performance evolution from a single agent to our Tri-MARF. This experiment is conducted on the Objaverse-LVIS dataset (with a random sample of 1k objects, the same subset used in the original paper). We compare four key configurations: (1) Single VLM Annotation Agent, using only a single front view for description; (2) VLM Annotation Agent + IAA, a combination of the first two agents to evaluate multi-view fusion capabilities; (3) Single Uni3D Gating Agent, to separately assess the ability to describe based solely on 3D geometric information; and (4) Tri-MARF (Ours).

The results above demonstrate the necessity of the holistic and synergistic design of our Tri-MARF. The strong performance of the single VLM agent (CLIPScore 81.4), as an effective base model, significantly outperforms the geometry-only Uni3D agent (58.3). However, there is a counterintuitive phenomenon that combining the first two agents leads to a performance drop to 63.2. We attribute this to the fact that, without final geometric verification, the blind aggregation of multiple (and potentially noisy) views "pollutes" the high-quality description from the best single view. This phenomenon precisely highlights the indispensable role of the Gating Agent (Agent 3). After introducing the Gating Agent, our Tri-MARF's performance leaps from a CLIPScore of 63.2 to 88.7, a stunning 25.5-point increase. This decisively proves that the Gating Agent acts as a critical "arbiter", leveraging 3D point cloud information to resolve multi-view conflicts and VLM hallucinations.

Q7. The setup for measuring A/B testing quality is not clear.

A7: We have already provided a comprehensive and standardized explanation of the detailed settings for human evaluation in Section 12, which clarifies the recruitment criteria for evaluators, including their required annotation experience and language proficiency. It then details the specific procedures for each experiment (e.g., 3D caption quality evaluation, type annotation validation, etc.), covering task design, the number of samples and evaluators, and explicit scoring criteria (e.g., a 1-5 Likert scale). To ensure the reliability of the evaluation results, we have also implemented strict quality control measures, including a pre-task training process for the evaluators, and quantified the reliability by calculating inter-annotator agreement (Cohen's Kappa reached 0.76), thereby guaranteeing the fairness and scientific validity of the entire evaluation process.

Q8. Typos in formulas and grammar.

A8: We have corrected these and proofread the whole paper. Specifically, regarding the mathematical expression on Line 122, we have removed the extraneous dollar signs surrounding the formula to present it as a standard inline equation, $D_{v}={C_{v,i}}{i=1}^{M}$. Regarding the grammatical error on Line 128, we have corrected the verb "frames" to "frame"; for the inline mathematical expression on Line 202, we have rewritten it using clearer, standard notation, changing the original format to $C{\text{canonical}}^{(k)} := \underset{C_{v,i} \in \mathcal{C}k}{\arg\max} s{v,i}$; regarding the cluttered layout in Section 3.2.2, we have thoroughly re-typeset the subsection for clarity, which includes correcting the jumbled symbols in formula (Line 227) to a standard reinforcement learning objective function, fixing the extraneous semicolon and using the standard '\arg\max' command in the UCB1 algorithm expression (Line 231) to be $\underset{a \in \mathcal{A}}{\arg\max} \left( \hat{r}_a + c \sqrt{\frac{2 \ln t}{n_a}} \right)$, correcting the erroneous comma in the empirical mean update formula (Line 235), and adjusting the paragraph structure throughout the subsection to ensure a logical and coherent flow of the argument.

Thank you very much for your insightful and valuable comments! We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about our work, please feel free to contact us during the discussion period.

Q1. Is there any other approach for candidate selection other than RL? Can the authors compare with other candidate-selection methods in other datasets?

A1: Yes. To address your concerns regarding technical contribution and demonstrate the superiority of the Multi-Armed Bandit (MAB)-based Reinforcement Learning aggregation strategy over traditional methods, we have designed and conducted an additional ablation study, which is conducted on the Objaverse-XL dataset, comprising 10,000 randomly sampled objects, with the goal of quantifying the marginal benefits introduced by the MAB strategy. To ensure a fair comparison, we have strictly controlled the experimental variables. Specifically, we keep all other modules of our Tri-MARF framework identical (such as the initial VLM annotation, semantic clustering, comprehensive scoring mechanism, and final description synthesis logic), and replace the aggregation strategy being evaluated within the Information Aggregation Agent (Agent 2). The compared strategies include our MAB (UCB) method and four heuristic baselines: Maximum VLM Confidence, Maximum Combined Score, Weighted Voting, and Simple Prioritized Concatenation. The performance of all strategies is assessed using a comprehensive suite of metrics covering both quality and efficiency, including a human-evaluated Likert scale (1-10), automated CLIPScore and ViLT R@5 retrieval accuracy, and the inference time of the aggregation module measured on a single NVIDIA A100 GPU.

Table S1: Performance Comparison of Different Aggregation Strategies on the Objaverse-XL Dataset

Table S1 indicates that while simple heuristic aggregation strategies can achieve decent performance, our MAB (UCB) strategy employed in Tri-MARF demonstrates a distinct advantage across all core quality metrics. The simplest strategies, "Maximum VLM Confidence" and "Simple Concatenation," though fastest in inference, perform the worst on the Likert scale, CLIPScore, and ViLT retrieval accuracy, proving that relying solely on the VLM's initial judgment or fixed concatenation rules is sub-optimal. The better-performing "Maximum Combined Score" heuristic, which combines VLM confidence with CLIP visual alignment, achieves a Likert score of 8.9 and a CLIPScore of 82.03%, demonstrating the effectiveness of a robust, information-rich heuristic rule. In contrast, our MAB (UCB) strategy consistently outperforms all other alternatives in annotation quality, achieving the highest Likert score (9.3), CLIPScore (82.72%), and ViLT R@5 scores (I2T 38.82%, T2I 36.72%). Compared to the best-performing heuristic baseline, MAB improves the CLIPScore by approximately 0.7 percentage points and also shows a significant increase in the Likert human evaluation, indicating that its generated descriptions are perceived as more accurate, complete, and fluent by human assessors.

The advantage of the MAB strategy lies in its ability to dynamically learn and adapt its selection policy by balancing exploration and exploitation. This adaptive capability is particularly crucial when dealing with diverse object categories and initial VLM descriptions of varying quality, enabling it to consistently select optimal descriptions that fixed heuristic methods might miss. Although the MAB (UCB) has a slightly higher inference time (9.8ms), this minor increase (e.g., only 1.7ms slower than "Maximum Combined Score") is perfectly acceptable in practical applications, considering its throughput performance. In summary, this experiment proves that the MAB (UCB) aggregation strategy, despite adding a degree of complexity, delivers tangible and quantifiable improvements in annotation quality. This enhancement is vital for achieving state-of-the-art performance, thereby justifying its necessity and rationale as a core component of the framework.

Q2. References to traditional approaches that rely on a limited perspective?

A2: We have added related references in the revised paper on traditional methods for 3D understanding that rely on a limited perspective, such as Sparse Multi-View [s1], Voxel-Based [s2], and Monocular [s3] approaches.

[s1] H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller, ‘Multi-view Convolutional Neural Networks for 3D Shape Recognition’, in Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 945–953.

[s2] B. Zhang, J. Tang, M. Nießner, and P. Wonka, ‘3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models’, ACM Trans. Graph., vol. 42, no. 4, Jul. 2023.

[s3] X. Zhao, Z. Liu, R. Hu, and K. Huang, ‘3D object detection using scale invariant and feature reweighting networks’, in Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, 2019.

Q3. Are all large models used in this study frozen? Did the authors attempt to fine-tune some of the VLM/LLMs used?

A3: Yes, all large models used in our study (e.g., Qwen2.5-VL, BERT, CLIP) are kept frozen, and we do not perform any fine-tuning. This decision is based on two core design considerations:

• Generality and Scalability: Our objective is to create a universal 3D annotation framework. By leveraging the powerful zero-shot capabilities of pre-trained models, our Tri-MARF can be directly applied to different domains without the need for expensive and time-consuming fine-tuning on new datasets. This point is substantiated by our excellent cross-dataset generalization results (Section 4.4, Table 2).

• Efficiency and Cost-Effectiveness: Fine-tuning large models requires immense computational resources, which contradicts our goals of achieving high throughput (12,000 objects/hour on a single A100) and low cost. Our Tri-MARF framework ensures high efficiency and cost-effectiveness by using API calls for the VLM and integrating lightweight local aggregation and gating modules, all while maintaining high-quality outputs.

Therefore, keeping the models frozen is a key design decision made to strike an optimal balance between performance, efficiency, generality, and cost.

Thank you very much for your insightful and valuable comments! We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about our work, please feel free to contact us during the discussion period.

Q1. Limited Contribution and Novelty.

A1: While our Tri-MARF framework indeed utilizes established pre-trained models, which is standard practice in the current field, we emphasize that the core innovation lies in the system-level architectural design, i.e., a collaborative network composed of three agents for VLM Annotation, Information Aggregation, and Gating Verification. This "team of experts" architecture is specifically designed to address key challenges that single models struggle with, such as cross-view inconsistency and missing geometric information. Its design philosophy is fundamentally different from existing methods that aggregate multimodal inputs.

The value of our architecture is not merely conceptual but has been experimentally validated. For instance, the Information Aggregation Agent employs a Multi-Armed Bandit (MAB) dynamic strategy customized for this task, rather than a simple off-the-shelf rule. The ablation study in Section 8 (Table 5) confirms that our MAB approach significantly outperforms four heuristic baseline methods across all metrics. Therefore, our contribution is not a mere stacking of modules, but a breakthrough in performance and efficiency achieved through sophisticated agent "orchestration" and synergy, which is in itself a significant system-level innovation.

Q2. Insufficient Experiments. Comparison against a baseline that uses a single agent with the same inputs and architecture.

A2: To more precisely dissect the contribution of our multi-agent framework, we have designed and added a step-wise ablation study to the revised paper. This study aims to demonstrate the performance evolution from a single agent to our complete three-agent collaborative architecture. The experiment is conducted on the Objaverse-LVIS dataset (with a random sample of 1k objects, the same subset used in the original paper), using CLIPScore and ViLT R@5 as the core evaluation metrics. We compare four key configurations: (1) Single VLM Annotation Agent, using only a single front view for description; (2) VLM Annotation Agent + Information Aggregation Agent (IAA), a combination of the first two agents to evaluate multi-view fusion capabilities; (3) Single Uni3D Gating Agent, to separately assess the ability to describe based solely on 3D geometric information; and (4) Tri-MARF (Ours), our proposed complete three-agent collaborative framework.

The results above demonstrate the necessity of the holistic and synergistic design of our three-agent framework. The strong performance of the single VLM agent (CLIPScore 81.4) proves that we have selected an effective base model, which significantly outperforms the geometry-only Uni3D agent (58.3). However, a key and counterintuitive phenomenon is that combining the first two agents leads to a performance drop to 63.2. We attribute this to the fact that, without final geometric verification, the blind aggregation of multiple (and potentially noisy) views "pollutes" the high-quality description from the best single view. This phenomenon precisely highlights the indispensable role of the Gating Agent (Agent 3).

After introducing the Gating Agent, the complete Tri-MARF framework's performance leaps from a CLIPScore of 63.2 to 88.7, a stunning 25.5-point increase. This decisively proves that the Gating Agent acts as a critical "arbiter," leveraging 3D point cloud information to resolve multi-view conflicts and VLM hallucinations, thereby unlocking and amplifying the full potential of the collaborative framework. Therefore, this experiment strongly indicates that our Tri-MARF framework's superior performance stems from the tightly-coupled, synergistic design where all three agents are indispensable, rather than from a simple stacking of any single component.

Q3. Lack of detailed qualitative analysis, especially discussion of failure cases or illusions.

A3: In fact, we have already provided several examples of qualitative analysis in our paper. Figure 1 offers a direct qualitative comparison of our Tri-MARF against SOTA methods like Cap3D and ScoreAgg. Figure 1 shows that our Tri-MARF can generate richer and more accurate details (e.g., "scissor doors," "carbon fiber") and even recognize proper nouns (e.g., "Golden State Warriors"), which other methods fail to do. Figure 21 (Page 33) demonstrates our Tri-MARF's robustness in handling partially occluded objects, a typical scenario prone to failure. The descriptions generated in the figure remain accurate and detailed, proving our Tri-MARF's effectiveness.

We agree that the discussion of failure cases can be further strengthened. In the revised paper, we have added a dedicated subsection to discuss typical failure cases, such as challenges that may arise when processing objects with extreme geometric ambiguity, objects with distracting textures, or scene-level 3D models, to provide a more comprehensive perspective.

Q4. Reproducibility and computational cost brought by pre-trained models.

A4: In fact, we provide a very detailed analysis of this in Section 6 of the supplementary materials, titled "Analysis of GPU Memory Usage and Computing Efficiency" (Pages: 14-15). Table 3 in that section (Page 15) lists the GPU memory usage (GB) and processing time (seconds) for each module in our framework (Data Preparation, VLM Annotation, Information Aggregation, Gating) on a single NVIDIA A100 GPU. We explicitly state that the VLM Annotation Agent is called via an API and therefore consumes no local GPU resources. We also report the framework's peak memory usage (approximately 4.6GB) and the total time for a single run. This detailed data provides a transparent and concrete reference for the reproducibility of our method, while the affordable computational resource requirements and high throughput simultaneously demonstrate our architecture's scalability.

Q5. The performance of the gating module is heavily dependent on the quality of the visual and textual encoders.

A5: We argue that while the performance of any multi-modal system is indeed correlated with encoder quality, this is precisely why we design our three-stage, progressive multi-agent framework to distribute and mitigate over-reliance on any single module. The Gating Module (Agent 3) is the last line of defense in our pipeline, not the sole guarantor of quality. Before it, the VLM Annotation Agent (Agent 1) and the MAB-based Information Aggregation Agent (Agent 2) have already performed initial generation and multiple rounds of optimization on the content.

The comparison experiment in Figure 20 of the supplementary materials (Page 32) also substantiates this. Our Tri-MARF without the Gating Module achieves a CLIPScore of 85.7. Although lower than the full Tri-MARF's 88.7, this is still a very strong performance, proving the robustness of the first two agents. The Gating Module provides a further performance boost on top of this foundation. Its role is more of a final filtering function rather than providing the core annotation content, so it is unfair to discuss its performance separately from that of the first two agents.

Q6. The paper has several formatting issues, including small text in figures, unreadable symbols in equations, and inconsistent notation. These problems hinder the clarity of the presentation.

A6: We are very grateful to the reviewer for the valuable feedback on the paper's formatting and clarity. We fully agree that a clear and professional presentation is crucial for communicating our research findings. Following your suggestions, we have conducted a comprehensive proofreading and revision of the formatting issues in the revised paper to enhance overall readability. Specifically, we have made the following changes:

• Enhanced Figure Clarity: We have redrawn all core figures, including the architecture diagram (Fig. 2), performance comparison charts (Fig. 4), and the VLM cost-benefit analysis graph (Fig. 6). We have significantly increased the font size of legends, axis labels, and internal annotations to ensure all details are clearly legible.
• Standardized Mathematical Formulas: We have carefully checked and corrected the typesetting of all equations. For example, we have fixed rendering errors in the original paper caused by extraneous symbols (like semicolons) and ensured that mathematical symbols such as $\text{Conf}(C)$ and $\hat{r}_{a}$ are standard and consistent throughout the text.
• Unified Key Notation: We have performed a consistency check on the use of symbols throughout the paper. For instance, we found that the symbol "$\alpha$" is used to represent two different concepts in the original paper (a weighting coefficient on Page 6 and a gating threshold on Page 7), which could cause confusion. In the revised paper, we have renamed the gating threshold to a new symbol, $\tau$, to eliminate this ambiguity.

We believe these detailed revisions have effectively addressed the issues you raised and significantly enhanced the professionalism and readability of the paper.