Margin Discrepancy-based Adversarial Training for Multi-Domain Text Classification

Complexity: The theoretical nature of the paper might make it less accessible to readers without a strong background in machine learning and domain adaptation. A more intuitive explanation of key concepts could be helpful.

Robustness Analysis: Perform more robustness and sensitivity analyses to explore the behavior of the MDAT method under various conditions or data scenarios.

Empirical performance. The performance improvement brought by the new method is not substantial and requires a detailed explanation of why this is the case. Additionally, please clarify the strengths of their approach.

In scientific writing, balancing theoretical depth with readability and clarity is crucial to make the paper accessible and informative to a wide range of readers, including reviewers. This feedback aims to guide the authors in achieving that balance. The theoretical part is detailed, and while theoretical underpinnings are important, they should be presented in a way that does not overwhelm the reader.

1. The theoretical contribution appears to be trivial. Techniques such as "calculating the centroid of the M domains" have already been employed in MAN [1]. The concept of the marginal discrepancy bound was previously introduced by MDD [2]. Merely combining these techniques doesn't seem sufficiently innovative to be deemed non-trivial.

2. There seems to be a flaw in the model design. Specifically, the private feature extractor does not leverage the power of domain-specific feature. This contradicts the authors' claim that it can "captures domain-specific features". I will explain it step by step.

   The issue arises when the private feature is concatenated with the shared feature and then inputted into both classifiers. As a result, any private feature becomes domain-invariant. This is enforced by the min-max game played between the two classifiers, even though each private feature extractor only has access to one specific domain.

   I want to emphasize that margin discrepancy simply acts as another measure of domain difference. If the model reaches its optimal state, the optimization of margin discrepancy across various domains ensures that the extractor aligns each domain’s distribution to the same latent space, leading to domain-invariant encoding. For a detailed explanation on how two classifiers still enforce domain-invariant features, even without a domain discriminator, the authors can refer to paper MCD [3].

   The authors might counter-argue that the inclusion of a private feature extractor enhances performance compared to using only the shared feature extractor. However, there could be two reasons for this: (1) the model might not be in its optimal state, as domain differences persist for both private and shared features. This allows the private feature to retain some domain-specific information. (2) Introducing a private feature extractor might simply increase the model’s parameters, leading to better performance.

   I strongly recommend that the model should not feed the private feature into both classifiers, mirroring the approach taken by MAN [1]. This modification could further enhance the model's performance.

[1] Chen, Xilun, and Claire Cardie. "Multinomial adversarial networks for multi-domain text classification." arXiv preprint arXiv:1802.05694 (2018).

[2] Yuchen Zhang, et al. Bridging theory and algorithm for domain adaptation. International Conference on Machine Learning, pages 7404–7413. PMLR, 2019.

[3] Saito, Kuniaki, et al. "Maximum classifier discrepancy for unsupervised domain adaptation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

1. As shown in Appendix A, the theoretic conclusion is the direct corollary of some previous works. The contribution is limited.
2. The experiment performance of the proposed method differs only fractionally from some of the other methods, and proves inferior to some of the latest state-of-the-art ones, as indicated in Appendix F (instead of the main text).
3. The discussion on the necessity of multi-domain text classification is not sufficient, specifically in the age of LLMs (large language models).

The theoretical development closely follows [Zhang et al, 2019]. In fact, to this reviewer, all theoretical results are merely re-stating the results of [Zhang et al, 2019] in the context of MDTC. In other words, the novelty in theoretical analysis is very thin. -- This is in fact not a weakness; I bring it up here just to mean that the theoretical analysis in this paper should not be regarded as a "strength" of any significant magnitude, despite the theoretical results presented. I suggest the authors clearly state in Section 4.1 that the theoretical analysis closely follows that in [Zhang et al, 2019], so as not look over-claiming.

I think the paper is above the acceptance threshold, although marginally.

In below, I declare a confidence level of 5, which I need to explain. I did not check the proofs. But since the application of [Zhang et al, 2019] is so natural in this context, I think the presented results must be correct (that is also why I do not consider the theoretical development novel). I have not read all literature of MDTC. So my confidence level "5" only refers to the correctness of the paper. If there were another work that has already exploited [Zhang et al, 2019], then my rating of the paper would drop below the acceptance threshold.