MAT-Agent: Adaptive Multi-Agent Training Optimization

W1. Clarity & Reproducibility.

A1: Thank you very much for your insightful suggestions. In the revised paper, the appendix and main text have been reorganized, and the implementation details have been moved from the appendix to their corresponding sections to ensure the paper's self-containedness. The specific revisions include: (i) Action Space: The complete agent action space have been incorporated into Section 3.2, (ii) State Vector: The detailed composition of the state vector, its specific metrics, and the historical information aggregation method have been added to the end of Section 3.2, (iii) Network Architecture: The specific architecture of the Q-Network have been specified in an "Architecture Box" within Section 3.3, and (iv) Reward Function: The weights of the composite reward function ($w_{mAP}$, etc.) and their calculation method have been moved to Section 3.3 via pseudocode. These adjustments can ensure the main paper is sufficient to support the reproduction of our work. We promise to release our source codes (see supplementary) once the paper is accepted.

W2. Overhead and Efficiency Claims:

A2: As our response to Reviewer dAxZ (see Table A), our MAT-agent is significantly superior to baselines in terms of overall convergence efficiency. Regarding the phrase "without added search overhead," we acknowledge that this statement is not accurate. Our original intent is to emphasize that MAT-Agent does not require large-scale, offline hyperparameter search or architecture evolution in the style of AutoML.

In our revised paper, Table B (all of which can be fully reproduced with the executable source codes included in the supplementary materials) shows that: (i) MAT-Agent's per-epoch online overhead is minimal, introducing only an approximate 9% increase in time (+0.03s/epoch) and a 3% increase in peak memory; (ii) this small investment yields a substantial return, boosting total convergence efficiency by 39% and drastically reducing the required GPU hours from 150h to 91h; (iii) even other dynamic methods like PBT show a less favorable efficiency gain (↓13%); and (iv) our total cost has an order-of-magnitude advantage compared to offline AutoML methods (>480h).

Table B: Computational Cost Trade-off Analysis of MAT-Agent vs. Other Methods (on 1×A100-80GB)

Notes:

• ¹ Baseline refers to a single, static training run with fixed hyperparameters.
• ² AutoML is divided into a 'Search Phase' and a 'Final Training' phase. The average per-epoch time and memory during the search phase are slightly higher due to operations like parallel trials, periodic evaluation, and model restarts. The final training phase is identical to the baseline.
• * The time and memory increase for the search phase is an average per trial. The overall resource consumption depends on the number of candidates.

W3. Multi-Agent Coordination:

A3: Our MAT-Agent is not driven solely by the shared global reward $R_{t+1}$. On the contrary, it follows a centralized-training, decentralized-execution (CTDE) paradigm and integrates three complementary mechanisms: (i) Different Rewards, (ii) Learnable Q-value Decomposition, and (iii) Differentiable Attention-Based Communication. These three components work in concert to markedly alleviate the credit assignment problem: using only the global reward degrades average performance by 12.5%; turning off difference rewards or replacing the mixer with VDN/simple summation lowers performance by 6.8% and 5.4%, respectively; and removing the communication channel slows convergence by 1.3x. Notably, the attention channel adds <2% computational overhead. To the best of our knowledge, this is the first work to combine different rewards with a learnable QMIX-style mixer in a heterogeneous, vision-centric multi-agent setting, while maintaining high efficiency through sparse attention-based messaging.

Q1. Multi-Agent Credit Assignment and Coordination:

A4: Our MAT-Agent follows a centralised-training, decentralised-execution (CTDE) paradigm, integrating three tightly-coupled layers of coordination beyond the shared global reward:

• Fine-grained Credit via Difference Rewards: For each agent, we compute its marginal contribution using difference rewards. This isolates each agent’s impact, providing an unbiased and effective training signal.

• Extended QMIX-based Value Decomposition: We factorise the global action-value via an extended QMIX mixer with gated residual connections and task-specific conditioning. This ensures monotonic and invertible mappings between global and local agent values $Q_i$, enabling gradients to propagate proportionally to each policy’s contribution.

• Sparse Attention-based Communication: Every eight timesteps, agents exchange messages through a sparse, single-head dot-product attention channel, enabling efficient sharing of salient hidden states with < 2% computational overhead.

Q2. Computational Overhead Analysis

A5: Our MAT-Agent does not require extensive offline hyperparameter tuning or expensive AutoML-style architecture search. In the revised paper, we have provided a detailed computational overhead analysis in Table C to confirm that the overall overhead introduced by MAT-Agent is moderate (approximately +9.7% per epoch), dominated slightly by forward passes through four small policy networks.

Table C: The exact measurements (on a single A100-80GB GPU)

Q3. Sensitivity to Reward Formulation

A6: We have analyzed the sensitivity to the reward weights with the following two experiments. First, we run a grid search around the default equal weights of $(0.25, 0.25, 0.25, 0.25)$. Each weight is perturbed by ±0.10 while the vector is re-normalized to sum to 1.0. Across all 81 trials, we find the performance to be highly stable: * Final Pascal VOC mAP varied by ≤ ±0.6% (from 96.8% to 97.6%). * Time-to-63-mAP on COCO fluctuated by ≤ ±4 epochs. Second, for completeness, we replace the four static weights with a learnable log-softmax layer. After a 5k warm-up period, the weights converge to (0.24, 0.26, 0.27, 0.23) and yield a final mAP of 97.5%, virtually identical to the hand-tuned result. These observations support the robustness of our reward formulation.

Q4. State Representation Components

A7: We have ablated each feature group in the state vector $I_{t}=[s_{t}^{\text{perf}};\ s_{t}^{\text{dyn}};\ s_{t}^{\text{data}}]$ and retrained MAT-Agent under identical conditions. Results are summarized below, where "Perf." = validation mAP & Δ-mAP, "Dyn." = loss trends & gradient norms, and "Data" = texture richness & tail-class ratio.

Key findings:

• Performance cues ($s^{\text{perf}}$) are indispensable (causes -3.7% mAP and +31 epochs when removed).
• Training-dynamics signals ($s^{\text{dyn}}$) mainly accelerate convergence (+19 epochs).
• Data descriptors ($s^{\text{data}}$) offer complementary gains (-1.4% mAP, +11 epochs) at a negligible cost.

The full feature set adds only 0.23 min/epoch (≈ 1.4% of a 16.1 min epoch) yet delivers the best accuracy and fastest convergence, clearly justifying its moderate computation overhead.

W1. The paper does not include any experiments in which all four decision dimensions are collapsed into a single RL agent with a joint action space.

A1: We have included this comparison as a new supplementary experiment.

+ Theoretical Motivation: Curse of Dimensionality in Joint Action Space. A single-agent RL controller managing all four decision dimensions (augmentation, optimizer, scheduler, loss function) faces a prohibitively large action space. Based on Appendix S1.3, the combined joint space contains:

• 5 Data Augmentations: Basic, CutMix, MixUp, RandAugment
• 4 Optimizers: SGD, Adam, AdamW, RAdam
• 5 LR Schedulers: Step Decay, Cosine, OneCycle
• 5 Loss Functions: BCE, Focal, ASL, CB Loss

→ Total joint action space size: 5x4x5x5=500 distinct configurations. Such a high-dimensional discrete space is well known to hinder sample efficiency, complicate credit assignment, and severely degrade convergence under value-based RL (e.g., DQN), especially when the reward signal is delayed and sparse. By contrast, our multi-agent factorization enables each agent to focus on a compact, semantically meaningful decision subspace. This yields: i) faster learning via simpler per-agent exploration, ii) clearer credit assignment under a shared global reward, and iii) improved stability and modularity of the training pipeline

+ Empirical Results: Single-Agent RL vs. MAT-Agent To validate this, we implemented a Single-Agent Joint-RL baseline that uses the same DQN architecture and total training budget but operates over the full 500-size joint action space. The results are summarized below:

This confirms that the performance gain is not simply due to RL, but emerges from our multi-agent decomposition with inter-agent coordination. The single-agent baseline converges much more slowly and underperforms MAT-Agent by 3.3 mAP points on Pascal VOC, despite having access to the full configuration space. Moreover, disabling agent coordination (Figure 5, Appendix S1.1) causes performance to drop to 91.7%, further underscoring that collaborative decomposition—not monolithic control—is the key design choice driving our gains.

Table A. Training settings, convergence speed (COCO epochs), and final detection quality (VOC mAP).

W2. No concrete numbers or plots that show how the size of the joint action space versus the individual action spaces impacts sample complexity or convergence speed.

A2: To make the claimed reduction in effective search size explicit, we quantify both the raw action-space cardinality and its empirical impact on learning efficiency.

As demonstrated in our response A1, the monolithic controller must explore $5(\text{DataAugmentation}) \times 4(\text{Optimizer}) \times 5(\text{LRSchedule}) \times 5(\text{LossFunction}) = 500 \text{Distinct Joint Actions}$ unique actions, whereas each MAT-Agent sub-policy navigates only 5, 4, 5, 5 choices, respectively. Although the theoretical reduction is 500 → {5, 4, 5, 5}, the more relevant metric is sample complexity: how many training epochs are required to reach a fixed accuracy target (63 mAP on MS-COCO). The table below (Table B) shows that shrinking and factorising the space translates into markedly faster convergence, i.e., 47 epochs versus ≈ 95 for the single-agent baseline, even under identical replay size and wall-clock budget. These concrete numbers substantiate our claim that linear-sized per-agent spaces improve exploration efficiency and credit assignment.

These concrete numbers substantiate our core claim: linear-sized, independent action spaces improve exploration efficiency and credit assignment, producing faster and better learning.
Table B. Ablation on action–space factorization and agent coordination.

Q1. Results for a single‑agent RL baseline in which the four decision axes are treated as one combined action space?

A3: We have implemented a single-agent RL baseline where one monolithic policy jointly controls all four training decisions—data augmentation, optimizer, learning rate scheduler, and loss function—resulting in a joint action space of 5×4×5×5=500 Distinct configurations.

This agent uses the same DQN architecture and total training budget as MAT-Agent. As shown in **Table A and TableB **, this baseline requires nearly twice as many epochs to reach 63 mAP on MS-COCO and still underperforms MAT-Agent by 3.3 mAP points on Pascal VOC.

These results support our hypothesis that multi-agent decomposition, beyond simply applying RL, provides tangible benefits in sample efficiency, convergence speed, and final model performance.

Q2: What are the exact sizes of the joint action space versus each individual agent’s action space?

A4. As detailed in Appendix S1.3 (and summarized in Table A of A2), the joint action space of a single-agent controller is $5(\text{DataAugmentation}) \times 4(\text{Optimizer}) \times 5(\text{LRSchedule}) \times 5(\text{LossFunction}) = 500\text{ distinct joint actions}$ By contrast, each MAT-Agent sub-policy operates over a much smaller, factorized action space. The approximate sizes for each agent are: - Data Augmentation Agent (Agent_AUG): ≥ 5 choices (e.g., Basic Augmentation, CutMix, MixUp, RandAugment). - Optimizer Agent (Agent_OPT): 4 choices (SGD, Adam, AdamW, RAdam). - LR Scheduler Agent (Agent_LRS): 5 choices (Step Decay, Cosine Annealing, One-Cycle policy, …). - Loss Function Agent (Agent_LOSS): 5 choices (BCE Loss, Focal Loss, ASL, Class-balanced Loss, …). This exponential-to-linear reduction (500 → 5/4/5/5) significantly lowers sample complexity and improves exploration efficiency and credit assignment, as demonstrated by the nearly 2× faster convergence and higher final mAP of MAT-Agent versus the single-agent baseline (see Table A in A2).

L1: Limitations

A5: Our MAT‑Agent framework, by greatly accelerating convergence and reducing wasted compute, can lower energy consumption for large‑scale vision training, contributing to greener AI practices. Faster turnaround may also democratize access to state‑of‑the‑art multi‑label models in research and industry, enabling smaller labs and companies to deploy high‑quality classifiers on constrained budgets.

Thanks for the responses. My main questions have been addressed. Based on the results provided by the authors, my further suggestion is that it would be better to further elaborate on the motivation for choosing multi-agent framework, instead of single-agent RL in the Introduction. This would help enhance the reader's understanding.

Thank you for your constructive feedback and for pointing out this important aspect. We fully agree that highlighting the motivation behind adopting a multi-agent framework (instead of a single-agent RL approach) in the Introduction will help readers appreciate the core rationale of our method. In the revised paper, we have added a dedicated paragraph that (1) explains the curse of dimensionality in joint action spaces, (2) describes how multi-agent factorization leads to linear per-agent action spaces, and (3) discusses the empirical and theoretical benefits observed. We are grateful for your guidance, which helps us make our work more accessible and impactful.

Thank you again for your valuable time and constructive comments.

Best Regards

Authors

Q1. Lack of Task-Specific Design for Multi-Label Image Classification

A1: Thank you very much for your insightful comments. Our MAT-Agent focuses on the core challenge posed by long‑tailed distributions, namely data scarcity and class imbalance, which fundamentally manifests as label‑level imbalance, in particular, the severe shortage of tail‑class samples. This issue is not confined to multi‑label image classification (MLIC). It is pervasive in vision tasks such as image classification, semantic segmentation, action detection, and pose estimation. We prioritize evaluating MAT‑Agent on MLIC because it exhibits the most extreme label imbalance and the most intricate label co‑occurrence patterns, providing the strictest test case for assessing MAT‑Agent’s ability to dynamically optimize tail‑class performance.

As a matter of the fact, the selected competing methods (i.e., AutoLR, BOHB, PBT, AutoAugment) can also be regarded as a general training optimization strategy that could be applied to a wide range of tasks. To further address your concern about generality, we have added several cross‑task experiments (see Table A below). The results, all of which can be fully reproduced with the executable code included in the supplementary material, show that our MAT‑Agent delivers outstanding transferability and robustness across tasks, markedly boosting tail‑class performance while accelerating convergence.

Table A: Comparative results across tasks, reporting Final Score (task-specific metric, ± standard deviation from three runs), Epochs to 95% Score, and Convergence Epoch. For semantic segmentation, we include mIoU (class-level quality) and Pixel Accuracy (overall accuracy). These metrics provide a comprehensive view of model performance.

Experimental Setup: We have compared automated training optimization methods (AutoLR, BOHB, PBT, AutoAugment, MAT-Agent) across three baseline models for related tasks:Semantic Segmentation: Used DeepLabV3+ with MobileNet backbone on standard VOC2012 dataset[S1], testing various learning rate schedulers (polynomial decay, step decay, cosine annealing), optimizers (SGD, Adam, AdamW), losses (cross-entropy, Focal Loss), and augmentations (light, medium, AutoAugment). Action Classification: Employed 3D-ResNet-50[S2] trained from scratch on UCF-101, using augmentations (random, corner, center cropping), optimizers (SGD, Nesterov SGD, AdamW), and losses (cross-entropy, Focal Loss). Pose Estimation: Utilized Pose-ResNet-50[S3] on COCO-2017, testing optimizers (SGD, Adam, AdamW), schedulers (MultiStepLR, CosineAnnealingLR, ReduceLROnPlateau), and augmentations (default, medium [30° rotation, 0.25 scale], strong [45° rotation, 0.35 scale]).

[s1] Chen et al., Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv 2017.

[s2] Kensho Hara et al., Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition. ICCV workshop, 2017.

[s3] Xiao et al., Simple baselines for human pose estimation and tracking. ECCV, 2018.

Q2. Unclear Cooperation Mechanism Among Agents

A2: We acknowledge that the original paper could better articulate the cooperation mechanisms among agents. Note that, MAT-Agent is not a set of independently trained controllers. Instead, it implements decoupled yet coordinated learning through shared state perception, unified reward signals, and collaborative experience replay. Specifically, all agents perceive a globally shared, dynamically updated state representation $I_t$, which integrates diverse indicators such as validation mAP, training dynamics (e.g., loss descent rate, gradient norms), and dataset characteristics (e.g., texture richness), ensuring all agents make decisions based on a consistent understanding of the training environment (see Sec. 3.2 and Appendix A.2). The joint configuration $C_t = (a^{\text{AUG}}, a^{\text{OPT}}, a^{\text{LRS}}, a^{\text{LOSS}})$ triggers a scalar global reward $R_{t+1}$ that evaluates the effectiveness of the full action set, balancing accuracy gain, training stability, convergence speed, and computational cost (Eq. 3). Although each agent optimizes its own Q-function, their objective is unified to maximize the expected cumulative global reward. Additionally, all interaction tuples $(I_j, a^k_j, R_{j+1}, I_{j+1})$ are stored in a shared replay buffer, enabling agents to learn from each other’s high-reward experiences (Appendix S1.1). These mechanisms ensure agents are aligned in objective, informed by the same context, and benefit from mutual learning signals, despite acting independently. Moreover, Appendix E.2 presents quantitative evidence of emergent cooperation, for instance, when the loss-function agent selects CB Loss to handle class imbalance, the augmentation agent selects Basic Aug with a 73% probability, forming a synergistic configuration to protect rare classes. Finally, in the “w/o Agent Coordination” ablation setting in Figure 5, removing the shared state, reward, and experience mechanisms causes the mAP to drop sharply from 97.4% to 91.7%, strongly validating the necessity of coordinated multi-agent interaction.

Q3.Response: Differentiation from Prior RL-Based Active Learning Approaches

A3: While prior works such as [1] and [2] dynamically optimize which data to select for annotation efficiency, our MAT-Agent addresses a fundamentally broader challenge by dynamically optimizing how to learn within the standard fully-supervised paradigm. This provides a key advantage in general applicability, as our framework can enhance any training process with a fixed dataset—a far more common scenario than active learning. Instead of focusing on the data stream, MAT-Agent’s core strength lies in its holistic optimization of the entire training process. By coordinating four autonomous agents, MAT-Agent jointly selects the optimal data augmentation, optimizer, learning rate scheduler, and loss function at each stage. This multi-agent architecture is crucial as it explicitly models and exploits the nonlinear interactions between training components—a factor entirely overlooked by data-centric or single-policy approaches. The significance of this coordinated strategy is empirically validated by our ablation study, i.e., disabling all agents leads to a sharp performance drop on Pascal VOC (mAP from 97.4% down to 91.7%), underscoring a synergy that isolated optimization cannot capture.

Dear Reviewer dAxZ,

Thank you again for your thoughtful review and for the time you have dedicated to assessing our work.
Since submitting our rebuttal, we have conducted additional analyses and would like to highlight three concrete updates that may help address any remaining uncertainty.

1. Task-Specific Rationale for Choosing MLIC

• Extreme label imbalance – MLIC exhibits the most severe long-tail distribution among standard vision tasks, making it the strictest testbed for dynamic optimization.
• New cross-task experiments – Table A in our rebuttal now shows that MAT-Agent also boosts performance on semantic segmentation, action recognition, and pose estimation, confirming that our method generalizes beyond MLIC while excelling under its harsher imbalance.

2. Explicit Cooperation Mechanism Among Agents

• Shared state Sₜ – All four agents observe a unified vector comprising validation mAP, gradient norms, loss descent rate, and dataset texture richness (Sec. 3.2, App. A.2).
• Global reward Rₜ – A single scalar balances accuracy, convergence speed, stability, and cost (Eq. 3), so agents are optimized toward a joint objective rather than independent goals.
• Collaborative replay buffer – Interaction tuples (Sₜ, Aₜ, Rₜ, Sₜ₊₁) are pooled, enabling agents to learn from each other’s high-reward experiences (App. S1.1).
• Quantitative proof of synergy – Removing shared state + reward + buffer drops Pascal VOC mAP from 97.4 % → 91.7 % (Fig. 5), demonstrating tangible cooperation benefits.

3. Positioning vs. Prior RL-Based Active Learning

• Prior works [1, 2] optimize which data to label, whereas MAT-Agent optimizes how to learn with a fixed, fully-labeled dataset—an everyday scenario in vision practice.
• Our four-agent architecture explicitly models interactions between augmentation, optimizer, scheduler, and loss—dimensions untouched by data-centric approaches.
• Ablation confirms that disabling any single agent causes ≥ 4 pp mAP loss, underscoring the necessity of a multi-agent, holistic strategy.

Additional Statistical Rigor

• 10-seed runs and paired-bootstrap 95 % CIs for every benchmark now appear in revised Table R1, showing all gains are statistically significant.
• Coordination overhead is quantified (Table A): 0.002 × the cost of a single VLM call, thus negligible relative to API usage.

If any aspect of our work remains unclear—we would be delighted to provide more details, code, or experiments. Your insights have already led to substantial improvements, and we greatly value any additional feedback during the remaining discussion period.

Thank you once more for your time and for helping us strengthen our submission.

Best regards,
The Authors