FreeLM: Fine-Tuning-Free Language Model

• Need more in-depth analysis regarding the influence on NLG tasks: As the author mentioned that FreeLM has a higher perplexity than GPT-2, it would be more informative to provide a numerical comparison on NLG benchmarks instead of only showing a few cases.
• Missing comparison with multitask fine-tuning: Table 4 does not include the results of the multitask fine-tuning (https://openreview.net/forum?id=gEZrGCozdqR) baseline, which is also learning-free when applied to other tasks. It would be important to compare with multitask fine-tuning results and analyze the generalization ability compared with FreeLM.
• Scalability: In-context learning is another learning-free method when performing downstream tasks and the performance of which gains a lot after scaling. However, there is a lack of comparison with in-context learning when model scales up.

1. While the "teacher iterative" method is an interesting take on prompt-based learning, merging pretraining and prompt-finetuning into a single step may not be as novel as it seems. A clear distinction and justification for this fusion could strengthen the paper.

2. The choice of the GLUE benchmark for evaluation does not make sense to me for a few reasons:

   1. GLUE is typically used to assess single-task, supervised learning models, mainly encoder-only (BERT-like) or sometimes encoder-decoder LMs (T5-like). However, decoder-only LMs, like FreeLM, may not perform as well on these tasks. For instance, decoder-only LMs generally score around 80 on MNLI, while state-of-the-art encoder-only models can achieve scores over 90. Even BERT-Base can achieve an accuracy of 84.

   2. The paper's approach to evaluating "generalization to unseen datasets" does not follow standard practices. A more established benchmark for zero-shot generalization of LMs on classification tasks is T0-Eval (https://arxiv.org/pdf/2110.08207.pdf), which includes 62 training datasets and 12 evaluation datasets.

   3. It is more appropriate to assess decoder-only LLMs on generative tasks, such as MathQA, HumanEval, MBPP, or OpenBookQA, rather than GLUE.

3. The comparison between FreeLM, GPT-3, and InstructGPT may not be entirely fair. A better approach would be to create a pretraining-finetuning baseline or to fine-tune an open-source LM like LLaMa on the GLUE benchmark for comparison.

4. Lastly, the paper's writing could be improved for clarity and readability. The language should be fluent and succinct, rather than coming across as if it was translated from another language.

Minor Weakness:

• I take issue with the term "fine-tuning-free" for the language model, as it suggests a misconception that the proposed method requires no training. I acknowledge that the community lacks a clear definition of "fine-tuning," commonly understood as "continuous training" on a pre-trained model. Nonetheless, the proposed language model also undergoes continuous training based on GPT-2, contradicting the "fine-tuning-free" label. Furthermore, even if you are starting training from scratch, calling it a "fine-tuning-free" language model is misleading if it indeed involves parameter updates.

Major Weaknesses:

• The paper uses many terms to embellish the method, such as "teacher signal" and "language signal," but in reality, the approach is a "co-training" scheme that trains on both target tasks and unsupervised English data. The paper argues that "pre-training and fine-tuning is not an optimal paradigm" and suggests "training them together." I do not recognize the benefits here. Is it to prevent catastrophic forgetting? If that's the case, it requires verification within the paper. Overall, this paper lacks novelty and fails to demonstrate the effectiveness of the proposed method.
• I assume one of the motivations behind "co-training" is to enhance generation ability. However, the authors did not substantiate this with perplexity scores. Some examples are unconvincing. More evidence is needed to support it, and if other methods are considered unreliable, human evaluation is required.
• The paper asserts that their proposed LM achieved SOTA, surpassing GPT-3, which is not accurate. Even straightforward fine-tuning on these tasks with GPT-2 outperforms Free-LM. Would it then be plausible for the authors to assert that "fine-tuning GPT-2 on these target data achieves SOTA and is superior to GPT-3"? The authors should explicitly state the limitations in this section. By the way, the authors also need to show details how they design the prompt for GPT-3, an improper prompt also has big impact on the performance.

Although the paper has many good motivations, it also has serious flaws.

1. The central claim is miss leading. They claims that their method is "finetuning free" and they compare their results to zero-shot and few-shot results. However, this is not the case. In particular, they still require gradient steps on the training split data of their eval tasks (or of a similar eval task). This is effectively finetuning, albeit with a slightly different loss function. In particular, using their method does not alleviate the user from collecting the finetuning set, which is importantly not the case for typical zero-shot and few-shot settings with a foundation LLM. Thus the claims involving comparison to GPT-3 and InstructGPT are unfair because those models did not explicitly have access to those finetuning training sets.

2. The proposed method lacks novelty. It is essentially doing multi-task finetuning with the pretraining task mixed in. They did not show whether doing the same scheduling mixing pretraining tasks with vanilla finetuning tasks would have a similar effect, or whether multi-task parallel training with both the pretraining task and the vanilla finetuning tasks would have a similar effect. This is a very important baseline that is completely missing. It is possible that a much simpler approach - simply mixing in some pretraining data/task at finetuning stage would have all the same effect.

3. The proposed formulation of the "unified data" seems narrow and classification heavy. It may not capture well generation heavy LLM use cases such as "write me a poem" that are very important and popular these days.

4. The evals are too narrow. For comparison with GPT3 and InstructGPT, the author should test few-shot and zero-shot performances on unseen tasks.