TRiCo: Triadic Game-Theoretic Co-Training for Robust Semi-Supervised Learning

Q1: Method Complexity, Modularity, and Practicality

We thank the reviewer for raising important concerns regarding the complexity and modularity of TRiCo. While our method integrates several components, it remains a principled, end-to-end, and fully differentiable framework—distinct from multi-stage pipelines or ad-hoc ensembles. All components are trained jointly, and their interactions are synergistic rather than additive.

To assess modularity, we conduct an ablation study to examine the contribution of each component. Results (Table: Ablation on TRiCo Components) show that removing any major module degrades performance, with the full TRiCo achieving 96.3% on CIFAR-10 (10% labeled), and drops of 1.1–2.2% observed when disabling MI filtering, the meta-teacher, generator, or co-training structure.

Table: Ablation on TRiCo Components (CIFAR-10, 10% labeled) All ablations are repeated over 5 random splits to ensure stability.

Despite integrating multiple components, TRiCo remains efficient and simple to deploy: frozen backbones (no encoder training), lightweight embedding-level adversarial updates (no input gradients), and meta-learning with only first-order updates.

To quantify computational cost, we provide a component-wise breakdown in Appendix B and summarize below:

Table: Component-wise Complexity Breakdown of TRiCo

Overall, TRiCo offers a practical trade-off between performance and complexity. It does not rely on strong augmentations, learnable view encoders, or auxiliary networks, and operates efficiently on a single NVIDIA RTX A6000 (48GB) or equivalent 24GB-class GPU (e.g., RTX 3090, RTX 4090).

---

Q2: Theoretical Justification and Stackelberg Formulation

We sincerely thank the reviewer for their insightful question regarding the distinction between Nash and Stackelberg equilibria in our triadic game formulation. You are absolutely correct—our framework is designed as a Stackelberg game, where the teacher $\pi_T$ acts as the leader, while the student classifiers $f_1$, $f_2$, and the generator $G$ act as followers responding to the teacher’s strategy.

While our original statement focused on the existence of a Nash equilibrium, we emphasize that what we actually prove is a Stackelberg-Nash equilibrium. That is, the equilibrium point is derived from a Stackelberg setting, but satisfies the equilibrium conditions of all agents, as clarified below.

In particular, our proof (Appendix A, Theorem 1) proceeds in two stages: In Equations (9–18), we establish that the strategy spaces of the teacher, students, and generator are all compact subsets of Euclidean space, and the payoff functions are jointly continuous. Therefore, by Glicksberg’s Theorem, a pure-strategy Nash equilibrium exists. Notably, Glicksberg’s theorem is agnostic to role asymmetry and is applicable to Stackelberg games as long as continuity and compactness hold. In Equations (19–32), we construct the solution explicitly under the Stackelberg game formulation, where the teacher optimizes a meta-objective based on the best responses of the students and generator. This construction satisfies the Stackelberg equilibrium condition by solving the bilevel optimization problem where the followers’ reactions are uniquely defined.

Although the generator $G$ is non-parametric, it is defined via a deterministic one-step projected gradient ascent (PGD) update in the embedding space, which depends on the current student parameters $\theta_S$. This structure allows us to treat $G$ as an implicit function $G(\theta_S)$, which is continuous with respect to $\theta_S$.

This design ensures that the generator’s response satisfies the continuity and stability conditions required for the teacher's optimization in the Stackelberg game to be well-posed. Consequently, we do NOT require any additional structural assumptions on $G$ beyond those already assumed in our setting. Moreover, as shown in Equations(29)-(31) in Appendix A, the joint strategy of the students and generator admits a well-defined best-response mapping under a fixed teacher strategy $\pi_T$. By applying a fixed-point theorem over this composite response, we construct the Stackelberg equilibrium even in the presence of a non-parametric generator. This validates the existence of a triadic Stackelberg equilibrium under the assumptions already established.

To avoid ambiguity, we will revise the manuscript to explicitly state that our framework admits a Stackelberg equilibrium, and we will introduce this result formally as Theorem 2 in the revised version. We believe this clarification strengthens the theoretical foundation and removes any residual confusion about equilibrium definitions.

Theorem 2 (Existence of Stackelberg Equilibrium). In the Stackelberg formulation, we assume that one party (the leader) commits to a strategy first, and the remaining parties (the followers) best-respond. In our case, the teacher $\pi_T \in \Pi_T$ is the leader, while the students $(f_1, f_2) \in \Pi_S$ and generator $G \in \Pi_G$ are the followers. We use the same assumptions on compactness and continuity as in Theorem 1.

Given a fixed teacher strategy $\pi_T \in \Pi_T$, the followers play a simultaneous game. Their equilibrium is defined by the following best-response conditions:

$\forall f_i \in \Pi_S: \quad f_i^* (\pi_T) = \arg\min_{f_i \in \Pi_S} R_S(f_i, \pi_T, G^* (\pi_T)), \quad i = 1,2,$

$G^* (\pi_T) = \arg\max_{G \in \Pi_G} R_G(G, \pi_T, f_i^* (\pi_T)).$

The teacher then selects her strategy to maximize her own payoff given the best responses of the followers:

$$\pi_T^* = \arg\max_{\pi_T \in \Pi_T} R_T\left(\pi_T, f_1^* (\pi_T), f_2^* (\pi_T), G^* (\pi_T)\right).$$

Then, the tuple $\left(\pi_T^* , f_1^* (\pi_T^* ), f_2^* (\pi_T^* ), G^* (\pi_T^* )\right)$ is a Stackelberg equilibrium if the following conditions hold:

(Leader’s optimality)

$$R_T\left(\pi_T^* , f_1^* (\pi_T^* ), f_2^* (\pi_T^* ), G^* (\pi_T^* )\right) \geq R_T\left(\pi_T, f_1^* (\pi_T), f_2^* (\pi_T), G^* (\pi_T)\right), \quad \forall \pi_T \in \Pi_T,$$

(Followers’ best responses)

$$\forall f_i \in \Pi_S:\quad R_S\left(f_i^* (\pi_T^* ), \pi_T^* , G^* (\pi_T^* )\right) \leq R_S\left(f_i, \pi_T^* , G^* (\pi_T^* )\right), \quad i=1,2,$$ $$\forall G \in \Pi_G:\quad R_G\left(G^* (\pi_T^* ), \pi_T^* , f_i^* (\pi_T^* )\right) \geq R_G\left(G, \pi_T^* , f_i^* (\pi_T^* )\right).$$

Proof Sketch. We reuse the compactness and continuity assumptions established in Theorem 1 (Eqs. 9–18 in Appendix A). For each fixed $\pi_T$, the follower subgame among $f_1, f_2, G$ admits a Nash equilibrium. The teacher’s objective is continuous in $\pi_T$ given continuous best-response mappings. Therefore, the leader’s optimization has a maximizer $\pi_T^* \in \Pi_T$, which implies the existence of a Stackelberg equilibrium. The specific equlibria is provided in (Eqs. 19–32 in Appendix A)

We greatly appreciate the reviewer’s suggestion and are confident that this update will improve both precision and clarity for readers and practitioners.

---

Q3: Clarity and Presentation Improvements

We thank the reviewer for the suggestions on figure quality. We will increase curve thickness and relocate Figure 1 to the experiments section in the final version to improve clarity. Rather than a loose combination of techniques, TRiCo’s design is guided by a unified game-theoretic principle: the teacher regulates pseudo-label quality and loss dynamics to shape stable, complementary student learning trajectories.

---

Summary

We clarify that TRiCo is a principled, end-to-end differentiable framework—not a multi-stage ensemble—where all components are co-trained for synergy. Through ablations, we show that each module (e.g., MI filtering, generator, meta-teacher) contributes significantly to performance. Despite its triadic structure, TRiCo incurs only modest computational overhead and runs efficiently on a 24GB-class GPU.

Theoretically, we explicitly formalize TRiCo as a triadic Stackelberg game, not just a heuristic co-training strategy. We prove the existence of a Stackelberg equilibrium using compactness and continuity assumptions, even with a non-parametric generator. This resolves the ambiguity around equilibrium type and strengthens the theoretical foundation.

We also appreciate the reviewer’s suggestions on figure clarity and will revise visuals accordingly. Overall, TRiCo is both theoretically grounded and practically viable for semi-supervised learning.

Q1: Clarification of Notation and Mathematical Precision

We thank the reviewer for highlighting the importance of notation clarity and agree that the current manuscript can be improved in this regard. We will revise the final version to explicitly define key symbols and clarify equations.

In Equation (1), we compute mutual information (MI) between model predictions $\hat{y}$ and model posterior induced by dropout. Specifically, we use:

$\mathcal{I}[\hat{y}; \omega] = \mathbb{H}\left[\frac{1}{K} \sum_{k=1}^{K} p_{\theta}(y|x;\omega_k)\right] - \frac{1}{K} \sum_{k=1}^{K} \mathbb{H}\left[p_{\theta}(y|x;\omega_k)\right]$

where $\omega_k$ denotes the $k$-th Monte Carlo dropout mask. We will clarify that $\bar{p}(y|x) := \frac{1}{K} \sum_{k=1}^{K} p_{\theta}(y|x;\omega_k)$ is the ensemble distribution, and $\mathbb{H}[p] = -\sum_{c=1}^{C} p_c \log p_c$ is the entropy of a categorical distribution. Additionally, we will specify that $\tau_{\mathrm{MI}}$ is a scalar threshold used to select pseudo-labels when $\mathcal{I}[y;\omega] \geq \tau_{\mathrm{MI}}$, and that its range lies in $[0, \log C]$, where $C$ is the number of classes.

Theorem 1 will also be revised to include the appropriate quantifiers. The corrected statement reads:
Theorem 1. Under assumptions (A1)–(A3), for all teacher strategies $\pi_T \in \Pi$, there exists a Stackelberg equilibrium $(f_1^\star, f_2^\star, G^\star)$ such that $(f_1^\star, f_2^\star, G^\star)$ form a joint best-response to $\pi_T$, and $\pi_T$ minimizes validation loss given these optimal responses.

Moreover, while the generator $G$ is non-parametric (via 1-step PGD), its updates are deterministic given student parameters, and can be treated as an implicit, structured response in the Stackelberg formulation. We will update Section 4.4 to clarify these points and explicitly distinguish Nash and Stackelberg equilibrium.

---

Q2. Clarification on Equilibrium Type and Theoretical Guarantees

We sincerely thank the reviewer for their insightful question regarding the distinction between Nash and Stackelberg equilibria in our triadic game formulation. You are absolutely correct—our framework is designed as a Stackelberg game, where the teacher $\pi_T$ acts as the leader, while the student classifiers $f_1$, $f_2$, and the generator $G$ act as followers responding to the teacher’s strategy.

While our original statement focused on the existence of a Nash equilibrium, we emphasize that what we actually prove is a Stackelberg-Nash equilibrium. That is, the equilibrium point is derived from a Stackelberg setting, but satisfies the equilibrium conditions of all agents, as clarified below.

In particular, our proof (Appendix A, Theorem 1) proceeds in two stages: In Equations (9–18), we establish that the strategy spaces of the teacher, students, and generator are all compact subsets of Euclidean space, and the payoff functions are jointly continuous. Therefore, by Glicksberg’s Theorem, a pure-strategy Nash equilibrium exists. Notably, Glicksberg’s theorem is agnostic to role asymmetry and is applicable to Stackelberg games as long as continuity and compactness hold. In Equations (19–32), we construct the solution explicitly under the Stackelberg game formulation, where the teacher optimizes a meta-objective based on the best responses of the students and generator. This construction satisfies the Stackelberg equilibrium condition by solving the bilevel optimization problem where the followers’ reactions are uniquely defined.

Although the generator $G$ is non-parametric, it is defined via a deterministic one-step projected gradient ascent (PGD) update in the embedding space, which depends on the current student parameters $\theta_S$. This structure allows us to treat $G$ as an implicit function $G(\theta_S)$, which is continuous with respect to $\theta_S$.

This design ensures that the generator’s response satisfies the continuity and stability conditions required for the teacher's optimization in the Stackelberg game to be well-posed. Consequently, we do NOT require any additional structural assumptions on $G$ beyond those already assumed in our setting. Moreover, as shown in Equations(29)-(31) in Appendix A, the joint strategy of the students and generator admits a well-defined best-response mapping under a fixed teacher strategy $\pi_T$. By applying a fixed-point theorem over this composite response, we construct the Stackelberg equilibrium even in the presence of a non-parametric generator. This validates the existence of a triadic Stackelberg equilibrium under the assumptions already established.

To avoid ambiguity, we will revise the manuscript to explicitly state that our framework admits a Stackelberg equilibrium, and we will introduce this result formally as Theorem 2 in the revised version. We believe this clarification strengthens the theoretical foundation and removes any residual confusion about equilibrium definitions.

Theorem 2 (Existence of Stackelberg Equilibrium). In the Stackelberg formulation, we assume that one party (the leader) commits to a strategy first, and the remaining parties (the followers) best-respond. In our case, the teacher $\pi_T \in \Pi_T$ is the leader, while the students $(f_1, f_2) \in \Pi_S$ and generator $G \in \Pi_G$ are the followers. We use the same assumptions on compactness and continuity as in Theorem 1.

Given a fixed teacher strategy $\pi_T \in \Pi_T$, the followers play a simultaneous game. Their equilibrium is defined by the following best-response conditions:

$\forall f_i \in \Pi_S: \quad f_i^* (\pi_T) = \arg\min_{f_i \in \Pi_S} R_S(f_i, \pi_T, G^* (\pi_T)), \quad i = 1,2,$

$G^* (\pi_T) = \arg\max_{G \in \Pi_G} R_G(G, \pi_T, f_i^* (\pi_T)).$

The teacher then selects her strategy to maximize her own payoff given the best responses of the followers:

$$\pi_T^* = \arg\max_{\pi_T \in \Pi_T} R_T\left(\pi_T, f_1^* (\pi_T), f_2^* (\pi_T), G^* (\pi_T)\right).$$

Then, the tuple $\left(\pi_T^* , f_1^* (\pi_T^* ), f_2^* (\pi_T^* ), G^* (\pi_T^* )\right)$ is a Stackelberg equilibrium if the following conditions hold:

(Leader’s optimality)

$$R_T\left(\pi_T^* , f_1^* (\pi_T^* ), f_2^* (\pi_T^* ), G^* (\pi_T^* )\right) \geq R_T\left(\pi_T, f_1^* (\pi_T), f_2^* (\pi_T), G^* (\pi_T)\right), \quad \forall \pi_T \in \Pi_T,$$

(Followers’ best responses)

$$\forall f_i \in \Pi_S:\quad R_S\left(f_i^* (\pi_T^* ), \pi_T^* , G^* (\pi_T^* )\right) \leq R_S\left(f_i, \pi_T^* , G^* (\pi_T^* )\right), \quad i=1,2,$$ $$\forall G \in \Pi_G:\quad R_G\left(G^* (\pi_T^* ), \pi_T^* , f_i^* (\pi_T^* )\right) \geq R_G\left(G, \pi_T^* , f_i^* (\pi_T^* )\right).$$

Proof Sketch. We reuse the compactness and continuity assumptions established in Theorem 1 (Eqs. 9–18 in Appendix A). For each fixed $\pi_T$, the follower subgame among $f_1, f_2, G$ admits a Nash equilibrium. The teacher’s objective is continuous in $\pi_T$ given continuous best-response mappings. Therefore, the leader’s optimization has a maximizer $\pi_T^* \in \Pi_T$, which implies the existence of a Stackelberg equilibrium. The specific equlibria is provided in (Eqs. 19–32 in Appendix A)

We greatly appreciate the reviewer’s suggestion and are confident that this update will improve both precision and clarity for readers and practitioners.

---

Q3. Stability of Multi-Agent Training and Sensitivity to Hyperparameters

We thank the reviewer for raising the important question of stability and sensitivity in training multi-agent systems. Despite TRiCo’s triadic design, our training remains highly stable in practice, as evidenced by the low standard deviation across all benchmarks (see Tables 1, 2, and Appendix C). This is due to three core design choices:

• Structured Modularity
  The roles of the teacher, students, and generator are cleanly separated via frozen encoders and cross-view pseudo-labeling, avoiding entangled feedback loops that commonly destabilize multi-agent learning.

• Smooth Meta-Scheduling
  The teacher $\pi_T$ is warm-started and trained with conservative updates (first-order meta-gradients) using stop-gradient feedback, which helps avoid oscillation in threshold/loss scheduling during early training.

• Implicit Generator
  The generator performs a single-step PGD perturbation in embedding space, without trainable parameters or gradients, keeping its behavior interpretable and consistent.

In addition to stability, we conducted a systematic sensitivity analysis of TRiCo’s key hyperparameters—MI threshold $\tau_{\mathrm{MI}}$, unsupervised loss weight $\lambda_u$, and perturbation budget $\epsilon$. As shown below, TRiCo exhibits strong robustness across a wide range of values:

Table: Sensitivity of TRiCo on CIFAR-10 w.r.t. hyperparameters $\tau_{\mathrm{MI}}$, $\lambda_u$, and $\epsilon$. Other settings fixed as default.

---

Q4: Explicit Limitations

We thank the reviewer and will clarify TRiCo’s limitations in Section 6. TRiCo relies on a small labeled validation set for meta-optimization, which may limit use in fully unsupervised settings—future work may explore self-supervised proxies. The non-parametric generator lacks memory and could benefit from learnable dynamics (e.g., conditional SSMs). TRiCo does not yet address extreme shifts like open-set scenarios, which could be tackled via OOD-aware extensions. These are trade-offs that offer clear paths for future work.

Q1 TRiCo Framework

To clarify our theoretical contribution, we summarize the key differences between TRiCo and standard co-training paradigms in the table below.

TRiCo formulates semi-supervised learning as a triadic Stackelberg game, where the teacher dynamically adapts its strategy based on student feedback—contrasting with conventional symmetric or heuristic co-training schemes. This explicit leader-follower hierarchy and embedding-space regularization form the foundation for TRiCo’s robustness and generalization.

Table: Key distinctions between TRiCo and conventional co-training.

---

Q2 Training Cost and Efficiency

We thank the reviewer for raising the importance of computational overhead. To assess whether TRiCo’s triadic structure introduces prohibitive cost, we compare against the most related baseline, Meta Co-Training (MCT), under identical settings: CIFAR-10 with 4k labels, ViT-B encoder, and batch size 64.

As detailed in Appendix B and summarized below, TRiCo adds only +7.1% FLOPs and +9.8% peak GPU memory per iteration. The additional cost stems from:

1. Mutual information estimation via $K{=}5$ stochastic forward passes (no backward),
2. Single-step PGD in embedding space (lightweight),
3. First-order meta-gradient updates.

Table: Training compute and memory cost comparison between TRiCo and Meta Co-Training (MCT)

Note: All costs are reported relative to MCT ($1.00 \times$) under identical hardware and training settings. For completeness, we also provide a component-wise cost breakdown in Appendix B. These modest overheads are justified by TRiCo’s significant performance gains.

Furthermore, since we do not use gradient checkpointing or mixed-precision optimization, additional acceleration is achievable in deployment settings.

---

Q3 Robustness

We thank the reviewer for this insightful observation.

While mutual information (MI) estimation via Monte Carlo dropout can be noisy early in training, TRiCo is specifically designed to mitigate such instability through three integrated mechanisms:

1. Warm-started teacher scheduling
   The meta-learned teacher $\pi_T$ begins with conservative parameters and gradually adjusts thresholds only after a stabilization phase (first 5 epochs), preventing premature pseudo-label filtering.

2. EMA smoothing of student predictions
   To reduce variance in MI estimates, we apply exponential moving average (EMA) over the output logits of student classifiers, enhancing temporal consistency.

3. Stochastic averaging over $K=5$ passes
   MI is computed via the difference between ensemble entropy and averaged predictive entropy across 5 stochastic forward passes, aligning with standard epistemic uncertainty estimation [Gal & Ghahramani, 2016].

We will clarify this explicitly in the final version.

---

Q4: Stability and Hyperparameter Sensitivity

We thank the reviewer for raising the important question of stability and sensitivity in multi-agent training. Despite TRiCo’s triadic design, training remains highly stable, as evidenced by low standard deviations across all benchmarks (see Tables 1–2, Appendix C). This robustness is attributed to three key design choices:

• Structured Modularity: The teacher, students, and generator operate on frozen encoders with cross-view pseudo-labeling, avoiding entangled feedback loops.
• Smooth Meta-Scheduling: The meta-teacher $\pi_T$ is warm-started and updated via first-order meta-gradients with stop-gradient signals, mitigating early oscillation.
• Implicit Generator: A deterministic, single-step PGD in embedding space ensures stability without learnable parameters or gradient feedback.

To further assess robustness, we conduct sensitivity analysis on key hyperparameters—MI threshold $\tau_{\mathrm{MI}}$, unsupervised loss weight $\lambda_u$, and perturbation budget $\epsilon$. As shown below, TRiCo maintains stable performance across wide value ranges without fine-tuning:

Table: Sensitivity of TRiCo on CIFAR-10 w.r.t. key hyperparameters
Other settings fixed to default.

Unlike prior SSL methods that require manual tuning of confidence thresholds per dataset, our meta-teacher adjusts these hyperparameters online during training. This reduces human intervention and enhances TRiCo’s usability across diverse datasets.

Summary

We thank the reviewers for their thoughtful feedback. In this rebuttal, we clarified TRiCo’s game-theoretic formulation, showing it departs from prior heuristic co-training by explicitly modeling a triadic Stackelberg game, where a meta-learned teacher dynamically adapts based on student feedback. We demonstrated that TRiCo remains efficient—adding only +7.1% FLOPs over MCT—and stable across benchmarks, supported by modular design, warm-started meta-scheduling, and an implicit generator. Our sensitivity analysis shows robustness to hyperparameters (≤1.1% variation), with no reliance on dataset-specific thresholds. TRiCo also achieves consistent gains across diverse frozen encoders, validating that its benefits are orthogonal to backbone strength. We hope these clarifications address all concerns and reinforce TRiCo’s theoretical soundness, empirical effectiveness, and practical viability.

We hope these revisions and clarifications address your concerns and strengthen the case for TRiCo’s significance and practical viability.

Q1. ImageNet Gains are Marginal

We thank the reviewer for this observation. While the absolute gain on ImageNet-10% is moderate (+0.1%), our improvement is more pronounced under the challenging 1% label setting, where TRiCo narrows the gap to full supervision by +2.4% over prior SOTA.

We also emphasize that TRiCo achieves superior performance with fewer training epochs and smaller backbones compared to ViT-H/14 used in some strong baselines. Moreover, as shown in our Few-Shot and Imbalanced (Appendix D) settings, TRiCo shows greater improvements in low-resource regimes—its primary focus.

---

Q2. Role of Mutual Information vs. Confidence Thresholding

We appreciate the reviewer’s skepticism regarding the effectiveness of MI filtering. To clarify, our ablation (Table 7) compares confidence filtering with fixed $\tau$ versus meta-learned MI thresholding, not confidence with meta-learning.

As shown in the table below, mutual information filtering consistently outperforms a meta-learned confidence threshold across all datasets and label regimes. These results highlight the advantage of epistemic uncertainty estimation for robust pseudo-label selection, especially under low-label and high-variance scenarios.

Table: Comparison of pseudo-label filtering strategies under meta-learned control across multiple datasets and label budgets (in % labeled data).
MI Filtering consistently outperforms a meta-learned confidence threshold (Meta-Conf), with lower variance and stronger robustness. Results are averaged over 5 independent runs.

---

Q3. Alternate Uncertainty Estimation Methods

Thank you for the insightful suggestion. While our current implementation adopts Monte Carlo dropout (MC-Dropout) for simplicity and efficiency, we agree that other techniques (e.g., ensemble-based variance, Dirichlet output modeling [Malinin et al., 2018]) may provide more calibrated uncertainty estimates.

We plan to explore these theory directions in future work and briefly mention this in the updated limitations section. That said, MC-Dropout offers a good trade-off between compute and performance: it requires no architectural change and adds only ~4.5% FLOPs (see Table below), while enabling robust pseudo-label filtering.

Table: Estimated FLOPs Overhead for Uncertainty Estimation Methods
Estimates based on ViT-B encoder, CIFAR-10 (4k labels), batch size 64.

---

Q4. Training Cost and Practicality

To assess whether TRiCo’s triadic structure introduces prohibitive cost, we compare against the most related baseline, Meta Co-Training (MCT), under identical settings: CIFAR-10 with 4k labels, ViT-B encoder, and batch size 64.

As detailed in Appendix B and summarized below, TRiCo adds only +7.1% FLOPs and +9.8% peak GPU memory per iteration. The additional cost stems from:

1. Mutual information estimation via $K{=}5$ stochastic forward passes (no backward),
2. Single-step PGD in embedding space (lightweight),
3. First-order meta-gradient updates.

Table: Training compute and memory cost comparison between TRiCo and Meta Co-Training (MCT)

Note: All costs are reported relative to MCT ($1.00 \times$) under identical hardware and training settings.

For completeness, we also provide a component-wise cost breakdown in Appendix B. These modest overheads are justified by TRiCo’s significant performance gains.

Furthermore, since we do not use gradient checkpointing or mixed-precision optimization, additional acceleration is achievable in deployment settings.

Summary

In summary, we reaffirm that TRiCo is designed for robustness in low-label and imbalanced regimes, where its benefits are most prominent. While gains on ImageNet-10% are modest, TRiCo delivers significant improvements under the more challenging ImageNet-1% setting, few-shot, and long-tailed benchmarks—highlighting its effectiveness where conventional SSL methods often struggle.

We clarified that mutual information-based filtering consistently outperforms confidence-based thresholds, especially under high uncertainty, and that our epistemic uncertainty estimation via MC-Dropout achieves a strong balance between accuracy and computational cost (~4.5% overhead) without architectural modifications.

Finally, we demonstrate that TRiCo’s triadic design adds only moderate computational overhead (+7.1% FLOPs, +9.8% memory) relative to Meta Co-Training, and fits within a single GPU setup. Its design avoids fragile components such as fine-tuned augmentations or auxiliary networks, making it both practically viable and broadly applicable.

We hope this response addresses all concerns and highlights TRiCo’s theoretical soundness, empirical rigor, and practical efficiency.

Q1: Class Imbalance

We thank the reviewer for pointing this out. We have conducted additional experiments on the CIFAR-10-LT benchmark (imbalance ratio 100), following standard protocols~[Wei et al., 2021; Kim et al., 2022]. As shown in Appendix D, Table 15, TRiCo achieves +4.2% higher tail-class accuracy than FixMatch and +2.8% over FreeMatch, confirming its robustness under class imbalance.

To further strengthen our claim, we also conducted evaluation on the more challenging ImageNet-LT benchmark under 1% label setting, comparing TRiCo against recent semi-supervised long-tailed learning methods such as ACR and SimPro.

Table: Top-1 Accuracy (%) on ImageNet-LT under Class Imbalance (1% labeled, 3 runs)
We report mean ± std deviation.

TRiCo achieves +2.6% improvement in tail-class accuracy over SimPro and +6.3% over ACR, verifying its robustness to long-tail label distributions. In addition to stronger average accuracy, TRiCo demonstrates lower variance, suggesting stable generalization under limited-label, imbalanced scenarios.

We also benchmark on ImageNet-127 and ImageNet-1k under varying resolution and imbalance settings. As shown below, TRiCo outperforms all existing methods in most configurations.

Table: Top-1 Accuracy (%) on ImageNet-127 and ImageNet-1k under Varying Test Imbalance Ratios and Resolutions (1 run) All models trained under $\gamma_l = \gamma_u \approx 286$ for ImageNet-127 and $\gamma_l = \gamma_u = 256$ for ImageNet-1k. † indicates ACR reproduction without anchor distributions.

These results demonstrate TRiCo’s superior generalization across diverse class imbalance levels and input resolutions.

---

Q2: Standard Deviation Values

We agree with the reviewer on the importance of reporting variance in semi-supervised settings. We mention it in Appendix C , we have included mean ± standard deviation across 5 independent runs in appendix tables, using different labeled splits (see Appendix C). In the final version , we will included mean ± standard deviation across 5 independent runs in all key tables.

While prior SSL works often omit variance reporting, we appreciate this suggestion and now incorporate it for improved reproducibility and transparency.

---

Q3: Representation Ability

TRiCo is not dependent on DINOv2 alone. As shown in Figure 4(b), we benchmark across multiple frozen encoders, including MAE, CLIP, and SwAV.

Notably, our performance gains persist across weaker encoders (e.g., MAE and SwAV), demonstrating that our improvements stem from the triadic co-training and MI-driven regularization, not from backbone strength.

To clarify further:
Even with MAE+SwAV—two encoders with lower standalone performance—TRiCo achieves a +3.7% gain over Meta Co-Training (MCT), reinforcing that our gains are orthogonal to encoder selection.

Table: Top-1 Accuracy (%) across different encoder combinations on CIFAR-100 (10% labels).
Results are averaged over 5 random splits. DINOv2 and CLIP are stronger encoders; MAE and SwAV are weaker baselines.

---

Q4: Method Complexity, Modularity, and Practicality

We thank the reviewer for raising important concerns regarding the complexity and modularity of TRiCo. While our method integrates several components, it remains a principled, end-to-end, and fully differentiable framework—distinct from multi-stage pipelines or ad-hoc ensembles. All components are trained jointly, and their interactions are synergistic rather than additive.

To assess modularity, we conduct an ablation in Appendix D to examine the contribution of each component. Results (Table: Ablation on TRiCo Components) show that removing any major module degrades performance, with the full TRiCo achieving 96.3% on CIFAR-10 (10% labeled), and drops of 1.1–2.2% observed when disabling MI filtering, the meta-teacher, generator, or co-training structure.

Table: Ablation on TRiCo Components (CIFAR-10, 10% labeled) All ablations are repeated over 5 random splits to ensure stability.

Despite integrating multiple components, TRiCo remains efficient and simple to deploy: frozen backbones (no encoder training), lightweight embedding-level adversarial updates (no input gradients), and meta-learning with only first-order updates.

To quantify computational cost, we provide a component-wise breakdown in Appendix B and summarize below:

Table: Component-wise Complexity Breakdown of TRiCo

Overall, TRiCo offers a practical trade-off between performance and complexity. It does not rely on strong augmentations, learnable view encoders, or auxiliary networks, and operates efficiently on a single NVIDIA RTX A6000 (48GB) or equivalent 24GB-class GPU (e.g., RTX 3090, RTX 4090).

---

Summary

We appreciate the reviewer’s constructive feedback. Our response has clarified that TRiCo’s performance stems not from encoder strength but from its principled triadic co-training framework and mutual information–based regularization. Through new experiments under class imbalance and across diverse encoders, we demonstrate that TRiCo is robust, modular, and generalizes well beyond the settings seen during training. We have also addressed concerns regarding complexity by quantifying training overheads, which remain modest, and showing that each component contributes meaningfully to the overall gain. Furthermore, we clarified our use of statistical reporting and theoretical foundations to support reproducibility and rigor. We believe these clarifications reinforce TRiCo’s originality, practicality, and relevance to the SSL community.

We also plan to reorganize and enrich the experimental section in the final version to enhance transparency.

We hope these updates address the concerns and strengthen our submission.