Can Information-Theoretic Generalization Bound Explain the Generalization of Pre-trained Language Model?

Thank you very much for your careful review. We appreciate your feedback and will do our best to answer your questions.

Q: Limited verification: this paper focuses on pre-trained language models, but they only used encoder-style models. Most commonly used pre-trained models today are decoder-only, and this has not been validated in this paper. This issue also extends to tasks such as language modeling and other sequence-to-sequence tasks.

A: We have added the experimental results for GPT-2 (Medium). The training performance, test performance, generalization error, and IIW of the pre-trained GPT-2 model and non-pre-trained model are shown in the table below. It is evident that pre-trained models consistently exhibit lower IIW while achieving better test performance.

We observe that during fine-tuning, a lower IIW does not always correspond to the best test performance, which aligns with the conclusions of our paper. The following table shows the training performance, test performance, and generalization error of GPT-2 during the fine-tuning process on the RTE dataset. Notably, GPT-2 achieves its best test performance at epoch 2, but this does not correspond to the lowest IIW.

We also have added the experimental results of GPT-2 on the XSum and CNN/DailyMail summarization tasks. Perplexity (PPL) is used as the evaluation metric. We observe that pre-trained models achieve lower PPL while also exhibiting lower IIW.

The following are the results of GPT-2 during the fine-tuning process on the CNN/DailyMail dataset. It can be observed that the model achieves the smallest generalization error at Epoch 1, but the corresponding IIW is not the smallest. This is consistent with the conclusions of our previous experiments.

Thank you very much for your careful review. We appreciate your feedback and will do our best to answer your questions.

Q: In my opinion the main weakness of this study is about one of the main findings that shows IIW does not reliably serve as a proxy for generalization performance during fine-tuning. ...

The authors explore different fine-tuning methods (e.g., full fine-tuning vs linear probing followed by full fine-tuning) but find that lower IIW does not consistently correlate with better test performance across these methods. The lack of consistent results across fine-tuning methods makes it difficult to draw actionable conclusions for practitioners who might want to use IIW as part of their model evaluation process.

Without offering practical solutions or next steps for improving IIW estimation, the paper leaves readers with more questions than answers regarding how to apply its findings in real-world scenarios. This limits its usefulness for both researchers and practitioners looking for concrete takeaways.

A: Since these questions are quite similar, we will provide a unified answer. We believe that the effectiveness of using information theory to explain pre-trained models is a promising direction. However, there has been no prior work that combines the information-theoretic generalization bounds with pre-trained language models. Additionally, there has been no work that provides a theoretical explanation of the limitations of the current IIW approximation. While we have not solved the limitation problem of the IIW approximation, our work promotes the application of information theory generalization boundaries in pre-trained models.

Q: The paper mostly focuses on presenting empirical results without clearly testing specific hypotheses about why certain phenomena occur (e.g., why lower IIW does not always correlate with better test performance). A more hypothesis-driven approach could have strengthened the paper by providing clearer explanations or predictions about when and why certain patterns would emerge. I think more explanation is needed for the inconsistency between IIW and test performance.

A: In fact, our experiments have shown that for models with significant differences in generalization error, such as pre trained and non pre trained models, IIW can serve as a proxy for generalization error. However, when the performance between models is relatively close, IIW cannot stably act as a proxy for generalization error. In Section 4, we provide relevant theories and experiments for verification. Our theory suggests that existing IIW approximation methods experience an increase in approximation error as the weight gap increases.

Thank you very much for your careful review. We appreciate your feedback and will do our best to answer your questions.

Q: a) When is IIW computed? 1)In table 2, when comparing between FT and LP-FT, when is IIW computed for each training method? Is it computed at the end of training for each method? Furthermore, please consider the following questions: 2)Will IIW at initialization be a measure that can different between the two methods? How would IIW for LP-FT after LP (first epoch) phase compare to IIW for FT at initialization? 3)Can IIW present better correlation for regularized fine-tuning, where the weights are restricted to stay close to the pre-trained values? A plot comparing IIW, regularization strength, and generalization behavior can further strengthen the statement in proposition 1. 4)Furthermore, Kumar et al.'22 proposed to use OOD generalization performance to differentiate between FT and LP-FT? Can IIW relate better to OOD generalization?

A: 1)During the experiments, we calculate IIW at each epoch. For all experiments, we consistently report the IIW and other performance metrics from the final epoch.

2)For a given dataset and model, the initial IIW is fixed. Therefore, for the same model, the initial stage IIW cannot be used to distinguish different training methods. The following are the performance metrics of BERT using LP-FT and FT on QQP. We can see that the first Epoch having a lower IIW does not necessarily result in the final Epoch having the best performance metrics.

3)In fact, if the weights are forced to remain unchanged, the model will not be able to fit the training set well, even if the IIW at this time has a better approximation effect. Since IIW depends on weight gap and Fisher matrix, we will consider how to combine weight gap and Fisher matrix to design regularization methods in the future.

4)From the perspective of information-theoretic generalization bounds, the purpose of IIW is not related to OOD. We believe that the relationship between IIW and OOD can be explored theoretically first, and this is worth exploring as a direction for future research.

I thank the authors for their detailed responses. I believe that this is a good paper. However, I keep my score as-is as I agree with reviewer aaJ2's comments. In the next version, please include discussions on how IIW can be useful to predict generalization trends. GFT is an important contribution, please include the basis of design choices behind SFT as well.

Thank you very much for your careful review. We appreciate your feedback and will do our best to answer your questions.

Q1: It is hard for me to pin down exactly what the contributions of the paper are. For example, I could see the paper trying to answer any one of the following questions:...

A: In this paper, we primarily investigate whether information-theoretic generalization bounds can be used to explain the effectiveness of pre-trained language models. Specifically, we explore whether these bounds can directly measure the generalization performance of pre-trained language models. To validate this, we experimentally compare the IIW (Information in Weights) of pre-trained and non-pre-trained language models and find that pre-trained models exhibit tighter generalization bounds. Since IIW depends solely on the training dataset, we further examine whether IIW can be directly used during the fine-tuning process as a criterion for model selection without relying on the test set. However, experimental results show that models with lower IIW do not always exhibit better performance, creating an inconsistency with the information-theoretic generalization bounds. We analyze this phenomenon and demonstrate, both theoretically and empirically, that current methods for estimating IIW involve certain inaccuracies, which may explain why IIW cannot consistently serve as a reliable proxy for model generalization.

Q2: All of the experiments in the paper focus on encoder-only language models. This makes the scope of the paper quite limited. It makes me question whether or not these results would generalize to decoder-only language models.

A: We have added the experimental results for GPT-2 (Medium). The training performance, test performance, generalization error, and IIW of the pre-trained GPT-2 model and non-pre-trained model are shown in the table below. It is evident that pre-trained models consistently exhibit lower IIW while achieving better test performance.

We observe that during fine-tuning, a lower IIW does not always correspond to the best test performance, which aligns with the conclusions of our paper. The following table shows the training performance, test performance, and generalization error of GPT-2 during the fine-tuning process on the RTE dataset. Notably, GPT-2 achieves its best test performance at epoch 2, but this does not correspond to the lowest IIW.

Q3: The tasks being explored are only binary (or ternary) classification tasks. What about more complex tasks of interest to the NLP community like summarization or generative benchmarks?

A: We have added the experimental results of GPT-2 on the XSum and CNN/DailyMail summarization tasks. Perplexity (PPL) is used as the evaluation metric. We observe that pre-trained models achieve lower PPL while also exhibiting lower IIW.

The following are the results of GPT-2 during the fine-tuning process on the CNN/DailyMail dataset. It can be observed that the model achieves the smallest generalization error at Epoch 1, but the corresponding IIW is not the smallest. This is consistent with the conclusions of our previous experiments.

Thank you for the detailed response and additional experiments. I understand the scope and intended contribution of the paper much better now. However, it is still difficult for me to see the causal relationship (or even sufficiency) relationship between IIW and generalizability, in practice (I understand that it leads to tighter bounds on generalizability, but loose bounds do not necessarily imply good generalization; and it seems that the authors' own summary of their work agree with this statement). As a result, I keep my score as is.