Rethinking the Bias of Foundation Model under Long-tailed Distribution

Thank you for your insightful feedback. Below, we summarize your points in quotes, followed by our corresponding replies.

How to formulate incomplete semantic factors and the question of their maximum number.

The incomplete semantic factor ($C$) represents the semantic region in the image that the foundation model prefers (the foundation model relies on it to make the final prediction). For example, in the first row of Fig. 4, OpenCLIP predominantly attends to the head ($C=0$), whereas MetaCLIP primarily focuses on the body ($C=1$), which is further supported by the experimental evidence in Fig. 8. Actually, different values of $C$ correspond to specific semantic regions, and the possible values of $C$ are infinite and are not limited to 0 (head) or 1 (body). In this way, the granularity of $C$ can be further refined, depending on the number of models. However, we set the maximum number limited to three as a trade-off between performance and costs. More details are in Sec.E.5.

The selection of foundation model and what model can be used and what model cannot?

As shown in Fig.5 in the paper, the path $D \rightarrow C$ represents the incomplete semantic factor that arises due to parameter imbalance, stemming from the imbalance in the pre-training data of the foundation model. Consequently, we have chosen CLIP, OpenCLIP, and MetaCLIP because they are pre-trained on distinct datasets, resulting in varying degrees of parameter imbalance and differing incomplete semantic factors.

In causal theory, Eq. 7 should cover the value space of $C$ as fully as possible for accurate causal effect estimation. Since $C$ has infinite values, we approximate it finitely, with broader coverage improving estimation accuracy. In this way, model selection should take into account the path $D \rightarrow C$. If two foundation models are trained on similar pre-training datasets, only one should be selected, as choosing both would not significantly increase the coverage of the value space of $C$.

Compare with MoE voting

In Eq. 7, $P(C)$ represents the prior distribution of the confounder. In large-scale datasets, this can be assumed uniform, making the final implementation resemble a voting process (like MoE) among fine-tuned CLIP models. However, unlike MoE, backdoor adjustment provides theoretical guidance for expert selection. For example, the parameter imbalance results in different experts exhibiting distinct tendencies in their prediction distributions: OpenCLIP exhibits a "head of the object" preference ($C=0$), while MetaCLIP exhibits a distinct "body of the object" preference ($C=1$), as shown in Fig.8. Guided by backdoor adjustment, if an additional foundation model that prioritizes the body is introduced, it should be combined with OpenCLIP rather than MetaCLIP. This is because pairing it with OpenCLIP broadens the range of confounders accounted for, whereas MetaCLIP does not provide such an expansion. To verify this, we introduce CLIP-CP, pre-trained on the CommonPool datasets, and combine it with OpenCLIP and MetaCLIP, respectively. The results are shown as follows:

This experiment shows that combining CLIP-CP with OpenCLIP is more effective. Additionally, following the experiments in Sec. E.6, we calculate the average confidence scores for the head images ($confidence$=0.2732) and body images ($confidence$=0.7122) based on CLIP-CP. These results confirm that CLIP-CP exhibits a "body of the object" preference, consistent with MetaCLIP, and demonstrate the effectiveness of backdoor adjustment in model selection.

Questions about imbalanced factors

Imbalanced factors (IF) measure dataset imbalance, particularly in downstream tasks. For example, the imbalanced factors for ImageNet-LT, Places365-LT, and iNaturalist2018 are 256, 996, and 500, respectively. Datasets with higher IFs tend to have larger performance disparities between head and tail classes, so addressing these imbalances is crucial for better performance.

For pre-training parameter imbalance, due to the inaccessibility of pre-training data, we can only measure this imbalance by giving a specific downstream dataset. For example, we estimate the label prior using Eq. 3 from the paper on the Places365-LT dataset for different foundation models and then calculate the IF. As shown below, different foundation models exhibit varying degrees of imbalance when evaluated on the same downstream dataset.

This metric can vary when samples are drawn from different distributions (domains), even if the category set is the same. For example, in a food classification task, dumplings and noodles may be more common in images from China, while hamburgers are more frequent in images from America. These cultural differences can lead to varying levels of imbalance.

We acknowledge that we will incorporate all the discussions in the paper.

We sincerely appreciate your thoughtful feedback. In the following, your questions are summarized in quotes, followed by our point-by-point responses.

Could the improvement be purely due to randomness?

To ensure that the observed performance improvement is attributable to the advantages of our method and not random variation, we conduct experiments with five different random seeds and report the mean and standard deviation for both LIFT and our method on Places365-LT and ImageNet-LT. As shown below, our method outperforms LIFT with a smaller standard deviation (reported in brackets), demonstrating that the performance gains are not due to randomness.

In addition, we also conduct a significance test (T-test) between the results of our method and the LIFT, obtaining a $p_{value1}=4.57\times10^{-9}$ and $p_{value2}=3.62\times10^{-10}$ for Places365-LT and ImageNet-LT, respectively. This indicates that our method is statistically significant, with a probability of making no more than a 5% error.

Please make all figure legends clearer.

After conducting a comprehensive review, we have carefully refined the captions and legends for each figure. Given that figures 3, 5, and 6 previously lacked sufficient detail, we present the revised results below.

Figure 3: The performance of different groups with (a) CE and (b) LA on the Places365-LT dataset. The three rows, from top to bottom, represent the performance of P-Many, P-Medium, and P-Few, respectively. The three columns, from left to right, represent the performance of D-Many, D-Medium, and D-Few, respectively.

Figure 5: The framework of our proposed method. (a) In the confounded setting, $C$ influences both $X$ and $Y$, leading to the backdoor path $X \leftarrow C \rightarrow Y$, thereby introducing confounding bias. (b) After intervening on $X$, its parent nodes $C$ and $U$ are severed, eliminating the unstable backdoor path and leading to a more reliable estimation of the causal effect between $X$ and $Y$.

Figure 6: The performance of different groups with our method on Places365-LT. After applying backdoor adjustment, performance improves across different groups, particularly for samples in both P-Few and D-Few.

Due to word limits, additional revisions of figure captions are not listed here. We promise that we will carefully rewrite all figure legends and captions in the original paper.

Clearly outlining the entire fine-tuning pipeline or algorithm.

During the fine-tuning phase, our method can be decomposed into two stages. In the first stage, we apply logit adjustment to fine-tune various foundation models (CLIP, OpenCLIP, and MetaCLIP), addressing the data imbalance present in the downstream dataset. In the second stage, we input each test sample into the fine-tuned models, obtaining output logits without additional fine-tuning. As described in Eq. 7, we then ensemble these logits using the importance weight $P(c)$ to correct for parameter imbalance, ultimately producing the final prediction score.

We promise to give an entire algorithm pipeline in the original paper.

What do authors think the parameter imbalance in LLM?

Thank you for your insightful comments! We also believe that current LLMs suffer from parameter imbalance. We analyze this issue from two perspectives, the influence of parameter imbalance without fine-tuning and with fine-tuning under downstream data:

(1) Without Fine-Tuning: Since LLMs are trained on diverse corpora, some domains (e.g., news articles, Wikipedia, and general web content) dominate the pre-training process. As a result, the model parameters may encode more precise representations of frequent patterns while underrepresenting rare or specialized knowledge. This imbalance becomes apparent when applying LLMs to highly domain-specific tasks, such as specialized scientific research or low-resource languages, where the model struggles due to insufficient parameter representation.

(2) Fine-tuning: When we use LLMs for customized tasks, such as role-playing or simulating a specific individual's tone, parameter imbalance can also have an impact. Since pre-training data is often biased toward general linguistic patterns, the model may struggle to capture highly personalized or domain-specific styles. For example, if an LLM is fine-tuned to simulate the conversational style of a historical figure, but the pre-training corpus contains limited examples of their actual patterns, the model may default to generic language structures instead of faithfully mimicking the target style, making it difficult to fully adapt to the new task.

We promise that we will add this discussion to the original paper

Thank you for your valuable comments and acknowledgement of our work! In the following, we summarize a series of works on Invariant Risk Minimization (IRM) to reduce bias and briefly introduce the differences between our work and existing studies.

Invariant Risk Minimization (IRM) aims to enhance out-of-distribution generalization by identifying and optimizing invariant features, thereby reducing bias in deep learning models [1, 2, 3]. In linear systems, IRM has strong theoretical guarantees and a clear connection to causal theory. Building upon IRM, numerous variants have emerged [4, 5, 6, 7], aiming to address some of IRM’s challenges, such as its failure in nonlinear tasks [8], the requirement for extensive domain information [2], and optimization difficulties in deep neural networks [9].

While IRM aims to identify and optimize invariant features across domains to reduce bias, our method leverages a causal learning framework that identifies and mitigates biases introduced by spurious correlations, which are induced by the backdoor path $X \leftarrow C \rightarrow Y$. Rather than assuming the existence of invariant features, we treat the incomplete semantic factor as a confounder and apply a backdoor adjustment method to learn the true causal effect, offering a more flexible solution to both parameter and data imbalance.

[1] Arjovsky, Martin, et al. "Invariant risk minimization." arXiv preprint arXiv:1907.02893 (2019).

[2] Lin, Yong, et al. "ZIN: When and how to learn invariance without environment partition?." Advances in Neural Information Processing Systems 35 (2022): 24529-24542.

[3] Deng, Yihe, et al. "Robust learning with progressive data expansion against spurious correlation." Advances in neural information processing systems 36 (2023): 1390-1402.

[4] Ahuja, Kartik, et al. "Invariant risk minimization games." International Conference on Machine Learning. PMLR, 2020.

[5] Krueger, David, et al. "Out-of-distribution generalization via risk extrapolation (rex)." International conference on machine learning. PMLR, 2021.

[6] Robey, Alexander, George J. Pappas, and Hamed Hassani. "Model-based domain generalization." Advances in Neural Information Processing Systems 34 (2021): 20210-20229.

[7] Ahuja, Kartik, et al. "Invariance principle meets information bottleneck for out-of-distribution generalization." Advances in Neural Information Processing Systems 34 (2021): 3438-3450.

[8] Rosenfeld, Elan, Pradeep Ravikumar, and Andrej Risteski. "The risks of invariant risk minimization." International Conference on Learning Representations (2021).

[9] Chen, Yongqiang, et al. "Pareto invariant risk minimization: Towards mitigating the optimization dilemma in out-of-distribution generalization." The Eleventh International Conference on Learning Representations (2023).