The Boundaries of Fair AI in Medical Image Prognosis: A Causal Perspective

Thank you for your thoughtful review. We appreciate your recognition that our work deals with an important and timely topic, offers novel approach, and provides extensive empirical analysis. We would now like to address your remaining concern below:

Q1. Regarding the challenges of TTE prediction such as informative censoring and competing risks.

We completely agree that informative censoring and competing risks present important challenges in TTE prediction. However, we would like to clarify that these challenges arise primarily from the perspective of TTE modeling itself, rather than from the fairness perspective, which is the main focus of our study.

In this work—the first comprehensive benchmark of fairness in medical image prognosis—we investigate the fairness behavior of data and models under the standard TTE prediction setting, which assumes a single risk and non-informative right censoring. This setting already presents significant experimental challenges, especially given the constraints of publicly available medical image datasets. Constructing datasets suitable for studying informative censoring or competing risks would require substantial effort during data collection and annotation, which we believe is currently not feasible at scale.

That said, we greatly appreciate your valuable suggestion and will consider exploring these directions in future work. We also emphasize that our proposed framework FairTTE is flexible and can incorporate a wide range of TTE models and datasets, including those designed for settings involving competing risks or informative censoring, when such data become available. We will include a discussion of these future directions in the revised version of the manuscript.

Q2. Regarding causal perspective used in our study.

We would like to clarify that the causal graphs proposed in our study serve as conceptual tools to illustrate how sources of bias may arise in medical image data—specifically, when there are causal pathways between sensitive attributes and other variables in the graph. In addition, our study focuses on group fairness notions, which are quantified using observed data distributions, rather than on counterfactual fairness, which requires causal path identification. As such, the causal graph is intended to aid in interpreting why disparities exist across subgroups, not to serve as a target for complete causal identification.

Q3. Regarding decomposition formula.

As mentioned in our response to Q2, our study focuses on group fairness notions, which are quantified based on observed data distributions. The decomposition formula is designed to break down disparities between group-specific data distributions into quantifiable terms, enabling us to systematically analyze group fairness in our experiments. In other words, the decomposition formula is not related to causal or counterfactual fairness concepts explored in works such as [1], but rather provides a descriptive tool grounded in observed data.

Q4. Regarding the appropriateness of fairness notion in medical domain.

We would like to clarify that there is no universal fairness notion suitable for all scenarios. Selecting an appropriate fairness criterion in the medical domain requires careful consideration of clinical context, ethical principles, and statistical validity. An effective fairness notion should align with the intended use of the model, its potential impact on different patient subgroups, and real-world constraints.

In our study, we adopt group fairness metrics, such as performance gaps across subgroups, because they are widely used in medical imaging tasks and offer clinically meaningful interpretations [2,3,4]. However, we acknowledge that this does not imply such metrics are appropriate for all medical applications. In contexts where prediction is inherently more challenging for one group, as you mentioned, minimizing performance gaps may inadvertently reduce performance for the advantaged group, which can be counterproductive. In such cases, maximizing subgroup-wise performance may be a more suitable objective.

Furthermore, by examining fairness through the lens of the data generation process, as we do in our study, we offer a principled framework for understanding the sources of bias. This perspective can help guide the selection of fairness notions that are best suited to the specific characteristics and ethical goals of a given application.

References

[1] Zhang, Junzhe, and Elias Bareinboim. "Fairness in decision-making—the causal explanation formula." Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018.

[2] Seyyed-Kalantari, Laleh, et al. "Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations." Nature medicine 27.12 (2021): 2176-2182.

[3] Vaidya, Anurag, et al. "Demographic bias in misdiagnosis by computational pathology models." Nature Medicine 30.4 (2024): 1174-1190.

[4] Tian, Yu, et al. "FairSeg: A Large-Scale Medical Image Segmentation Dataset for Fairness Learning Using Segment Anything Model with Fair Error-Bound Scaling." The Twelfth International Conference on Learning Representations (2024)

Q3. Regarding the appropriateness of fairness notion in medical domain.

Thank you for your valuable feedback. Based on our discussion, we plan to include the following paragraph in the revised version of our manuscript to clarify our rationale for the fairness criterion used in this study:

Selecting an appropriate fairness criterion in the medical domain requires careful consideration of the clinical context, ethical principles, and statistical validity. An effective fairness notion should align with the model’s intended use, its potential impact on different patient subgroups, and real-world constraints. In our study, we focus on statistical group fairness (i.e., predictive performance gaps across subgroups) which is widely examined in the medical image analysis literature. This choice is grounded in the assumption that any causal pathway from sensitive attributes represents an unfair influence in the causal graph and should be mitigated through fairness constraints. However, we acknowledge that in practical clinical scenarios, such causal pathways may reflect fair and clinically meaningful relationships. Enforcing group fairness in these cases may inadvertently remove relevant information and degrade predictive performance.

Moving forward, we suggest several directions for fairness research in medical image analysis: (1) Identifying the causal nature of bias to distinguish between fair and unfair sources, enabling models to address specific pathways appropriately; (2) Developing fairness metrics and mitigation strategies that preserve clinically relevant (fair) pathways while minimizing the effect of unfair ones; and (3) Collaborating with clinicians and domain experts to define context-specific fairness objectives that are aligned with both clinical utility and ethical standards.

In addition to including this paragraph, we will also include the limitations section in the revised version to clearly acknowledge the scope of our fairness notion and to highlight the potential trade-offs in enforcing group fairness (see our response to Q1).

Thanks, the response is appreciated, I have increased the score from 3 to 4.

While I am still of the opinion that doing state-of-the-art medical survival analysis is almost impossible without considering informative censoring and competing risks, it is true that the landscape is complex and one has to start from somwhere, and your effort is moving significantly in the right direction.

Thank you for your thoughtful review. We appreciate your recognition that our work addresses an important problem in medical AI, presents a thorough evaluation, and offers insightful findings. We would now like to address your remaining concern below:

Q1. Regarding suggestions to develop new fairness algorithm for TTE prediction based on the causal graph.

Thank you for this valuable feedback. We offer several suggestions for future research on fairness in medical image prognosis:

• Identifying the causal nature of bias is essential for distinguishing between fair and unfair sources, enabling models to adjust for specific pathways rather than treating all disparities uniformly.

• Developing appropriate bias mitigation strategies and fairness metrics that leverage fair causal pathways while minimizing the influence of unfair ones.

• Collaborating with clinicians and domain experts to define application-specific fairness objectives ensures that fairness interventions align with clinical utility and ethical standards.

In our opinion, a holistic approach to fairness should span the entire AI development pipeline (i.e., from data collection and fairness metric selection to model architecture and evaluation) to effectively mitigate bias in real-world applications.

For example, if we identify an unfair causal pathway between a sensitive attribute and the underlying health condition during the data collection process, techniques such as fair representation learning may help remove spurious correlations. However, if the sensitive attribute is a clinically meaningful risk factor for the TTE outcome, the model should be encouraged to extract sensitive attribute-related features from the image to calibrate its predictions appropriately. In such cases, applying fairness methods that obscure this information like fair representation learning could inappropriately degrade performance by forcing the model to ignore clinically relevant signals.

We will incorporate this discussion in the revised manuscript to highlight the potential impact of our framework beyond benchmarking.

Q2. Regarding comparison with medical image diagnosis scenario.

We clarify that there are both shared and distinct aspects in the causal analysis of bias between diagnosis and prognosis tasks. While causal analysis has been applied to study biases in diagnostic settings, prognostic tasks introduce unique challenges that require specialized consideration. Specifically, TTE prediction involves longitudinal outcomes and censoring, which fundamentally differ from the static labels used in diagnosis. These temporal aspects introduce additional causal pathways through which bias can arise, such as disparities in event timing or censoring mechanisms across subgroups.

Among the five sources of bias identified in our framework, disparities in image feature distributions have been explored in prior works on medical image diagnosis [1,2]. However, the remaining sources, particularly those related to TTE outcome and censoring, are unique to prognosis.

We will make these distinctions more explicit in the revised manuscript to highlight how our framework extends fairness analysis in prognostic modeling beyond the conventional diagnostic setting.

Q3. Regarding image comparison.

We would like to clarify that the image comparisons discussed in Section 2.1 (lines 106-116) and Appendix F.4 (lines 1418-1425) are conducted in different contexts. Specifically, in Section 2.1, the goal is to define a fairness metric that requires measuring distances between images in a clinically meaningful way, which is inherently a challenging task. In contrast, the analysis in Appendix F.4 involves comparing image features from a statistical perspective to quantify sources of bias. In this latter case, using a statistical distance measure such as the Wasserstein distance is sufficient and appropriate for the intended analysis.

Q4. Regarding source of variability.

We would like to clarify that, in addition to calculating confidence intervals using test-set bootstrapping, we also performed statistical significance testing across results from all random seeds in our experiments to verify the robustness of our findings to training variability induced by random initialization.

Q5. Regarding the practical utility of FairTTE.

We would like to clarify that FairTTE can be used as a versatile tool that can support practitioners in multiple stages of the ML pipeline for medical prognosis. Specifically:

• FairTTE provides a standardized framework to evaluate fairness across subgroups under realistic clinical conditions, such as distribution shifts and censoring. Regulators could use FairTTE to assess whether a TTE prediction model exhibits consistent performance and equitable treatment across demographic groups, forming part of an evidence base for approval or deployment.

• FairTTE also enables practitioners to systematically identify sources of bias, whether from image features, outcome disparities, labeling process, or censoring mechanisms. This can guide unbiased dataset curation process.

• Developers can use FairTTE to benchmark the fairness behavior of candidate models or fairness-aware training strategies across multiple axes (datasets, attributes, metrics). This allows for more informed model selection and ensures that fairness-performance trade-offs are clearly understood before deployment.

Ultimately, we see FairTTE as a practical and extensible framework that can be integrated into the clinical ML development lifecycle.

References

[1] Jones, Charles, Mélanie Roschewitz, and Ben Glocker. "The role of subgroup separability in group-fair medical image classification." International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2023.

[2] Yang, Yuzhe, et al. "The limits of fair medical imaging AI in real-world generalization." Nature Medicine 30.10 (2024): 2838-2848.

Thank you for your thoughtful review. We appreciate your recognition of our work's solid theoretical framework, large-scale comprehensive evaluation, and clear presentation. We would like to address your remaining concern below:

Q1. Regarding experiments with pathology images.

Thank you for the suggestion. Pathology images indeed play a critical role in medicine by providing detailed visual information about tissue morphology, which is essential for accurate disease diagnosis, such as in cancer. However, to the best of our knowledge, existing publicly available pathology image datasets lack either sensitive attributes or longitudinal clinical outcomes—both of which are necessary to study fairness in TTE prediction. For example, the dataset referenced in [1] does not include longitudinal outcomes, making it unsuitable for TTE analysis. Additionally, due to the time constraints of the rebuttal phase and the complexity involved in accessing (e.g., controlled access) and processing pathology images (e.g., whole-slide image handling, linking images with clinical outcomes) from sources such as TCGA [2], conducting a large-scale analysis using these data is currently not feasible. We will continue to monitor developments in this area and plan to incorporate pathology image datasets into our future studies as soon as appropriate resources become available.

References

[1] Vaidya, Anurag, et al. "Demographic bias in misdiagnosis by computational pathology models." Nature Medicine 30.4 (2024): 1174-1190.

[2] https://www.cancer.gov/ccg/research/genome-sequencing/tcga

Thank you for your thoughtful review. We appreciate your recognition that our work addresses an important yet underexplored problem and that our evaluation is thorough. We would now like to address your remaining concern below:

Q1. Regarding the proposal of a new fairness method for TTE prediction.

We would like to clarify that our primary contribution lies in providing a novel and systematic framework to analyze the fairness behavior of (fair) TTE prediction models in medical image prognosis—a critical yet underexplored area. To the best of our knowledge, this is the first comprehensive study that rigorously examines how existing (fair) TTE prediction methods perform across different datasets, sensitive attributes, and fairness metrics under clinically relevant distribution shifts. Our framework enables the identification of key limitations in current approaches, offering actionable insights for the development of more robust and equitable models in the future. Thus, our work fills an important methodological gap and lays the foundation for principled fairness evaluation in medical image prognosis.

Q2. Regarding identification of fair and unfair pathways in the causal graph.

We would like to clarify that the proposed causal graph serves as a conceptual tool to identify and explain potential sources of bias and to guide fairness analysis in a more structured manner. As stated in our main paper, our primary goal is to measure bias sources from medical images and TTE prediction models, which can be quantified using observed data; therefore, full identification of causal pathways is not required for our study. That said, we acknowledge your point that identifying causal pathways can be challenging in some medical contexts, and input from domain experts may be necessary to assess the fairness of specific causal pathways (as noted at the end of page 6 in our paper). Ultimately, fairness decisions should be informed by context-specific ethical and clinical considerations. Our framework is intended to support that reasoning—not to replace it.

Q3. Regarding distribution shift settings.

We would like to clarify that distribution shifts created by intervening causal graph in our study mimic real-world scenarios as follows.

• Adding noise to images mimics real-world scenarios such as patients in different geographic locations are scans with different equipment. This causes the medical image to appear systematically different for groups in each location.

• Adding noise to TTE labels mimics real-world scenarios such as delayed or inaccurate event recording in EHR system.

• Flipping censoring indices mimics real-world scenarios in which certain groups experience less consistent access to care due to financial or geographic. Thus, these groups are more likely to drop out of care, resulting in a higher censoring rate.

By leveraging these data intervention mechanisms, we can analyze how different sources of bias impact the fairness behavior of TTE prediction models. While studying real-world distribution shifts is important, it is often challenging to identify the exact sources of bias and distribution shift mechanisms which complicates the fairness analysis. Based on your suggestion, we will add this discussion into our revised version.

Q4. Regarding development of potential holistic approaches for fair TTE prediction.

We would like to clarify that a holistic approach to fairness should span the entire AI development pipeline (i.e., from data collection and fairness metric selection to model architecture and evaluation) to effectively mitigate bias in real-world applications. In particular, we provide some suggestions for future research on fairness in medical image prognosis as below:

• Identifying the causal nature of bias is essential for distinguishing between fair and unfair sources, enabling models to adjust for specific pathways rather than treating all disparities uniformly.

• Developing appropriate bias mitigation strategies and fairness metrics that leverage fair causal pathways while minimizing the influence of unfair ones.

• Collaborating with clinicians and domain experts to define application-specific fairness objectives ensures that fairness interventions align with clinical utility and ethical standards.

For example, if we identify an unfair causal pathway between a sensitive attribute and the underlying health condition during the data collection process, techniques such as fair representation learning may help remove spurious correlations. However, if the sensitive attribute is a clinically meaningful risk factor for the TTE outcome, the model should be encouraged to extract sensitive attribute-related features from the image to calibrate its predictions appropriately. In such cases, applying fairness methods that obscure this information like fair representation learning could inappropriately degrade performance by forcing the model to ignore clinically relevant signals.

Q5. Regarding the impact of discretization granularity on both fairness and accuracy in TTE prediction models.

We would like to clarify that the choice of discretization granularity in TTE prediction models can influence both predictive accuracy and fairness outcomes. However, based on our preliminary experiments, varying the discretization scheme did not alter the key empirical findings of our study. Therefore, we chose to fix the discretization scheme across all experiments to isolate the effects of fairness interventions and distribution shifts on both fairness and accuracy in TTE prediction.

To provide additional context, discretization granularity affects model behavior in several ways. Finer-grained discretization allows the model to capture more precise temporal information, potentially improving predictive accuracy. However, it can also increase model complexity and lead to data sparsity within individual bins, particularly for underrepresented subgroups, which may exacerbate disparities and fairness concerns. On the other hand, coarser discretization reduces data sparsity and simplifies the learning task, which can enhance model stability and, in some cases, mitigate fairness disparities. This benefit, however, comes at the cost of lower temporal resolution, which may limit the model’s ability to make accurate TTE predictions. We will expand this discussion in the revised manuscript to better clarify the trade-offs involved.

Thank you for your thoughtful review. We appreciate your recognition that our work addresses a critical gap in fair AI research. Below, we address your remaining concerns:

Q1. Regarding the limitations of existing fairness methods for TTE prediction.

We would like to clarify that the observed limitations are not weaknesses of our work. Instead, our study is the first systematic investigation into the fairness behavior of existing (fair) TTE prediction algorithms. Through extensive evaluation, we identified several key limitations of current methods:

• Fairness algorithms exhibit inconsistent performance across datasets, sensitive attributes, and fairness metrics.

• Enforcing fairness may harms predictive performance, especially in case sources of bias are small, causing fairness-accuracy trade-off.

• Pre-training improves model accuracy but has minimal impact on fairness.

In sum, these findings highlight shortcomings in existing methods that our framework helps to reveal, not deficiencies in our own contribution.

Q2. Regarding triangle inequality and symmetric assumptions for TTE performance metrics.

While some metrics such as the concordance index do not satisfy the triangle inequality or symmetry assumptions, they can be modified to do so. For example, prior work [1] proposed the symmetric discordance index, a variant of the concordance index that satisfies both assumptions. Moreover, these assumptions were introduced solely to facilitate the proof of Theorem 1 and do not affect our other theoretical results, including Proposition 2 and the decomposition formula in Equation (4), or our empirical findings.

Q3. Regarding causal pathway identification.

We would like to clarify that causal pathway identification is not required for our work. Our goal is to measure bias from medical images which can be quantified from observed data. The causal graph serves as a conceptual tool to explain why disparities exist in data distributions across subgroups, not as a target for full identification.

Q4. Regarding distribution shift settings.

We would like to clarify that distribution shifts created by intervening causal graph in our study mimic real-world scenarios as follows.

• Adding noise to images mimics real-world scenarios such as patients in different geographic locations are scans with different equipment. This causes the medical image to appear systematically different for groups in each location.

• Adding noise to TTE labels mimics real-world scenarios such as delayed or inaccurate event recording in EHR system.

• Flipping censoring indices mimics real-world scenarios in which certain groups experience less consistent access to care due to financial or geographic. Thus, these groups are more likely to drop out of care, resulting in a higher censoring rate.

By leveraging these data intervention mechanisms, we can analyze how different sources of bias and distribution shift types impact the fairness behavior of TTE prediction models. While studying real-world distribution shifts is important, it is often challenging to identify the exact sources of bias, which complicates the fairness analysis.

Q5. Regarding fairness metrics for TTE prediction.

We would like to clarify that performance gaps across subgroups are appropriate fairness metrics and not narrows the scope of our study. These metrics are widely used in medical imaging tasks [2,3,4] and are clinically meaningful. While other fairness metrics exist in the TTE literature, they may not be appropriate for the clinical domain. For example:

• Enforcing similar predicted TTE outcomes across subgroups may compromise clinical relevance and accuracy. A case study in Alzheimer’s disease illustrates this point well: requiring young and elderly individuals to have similar predicted times to disease onset would not make sense.
• Enforcing similarity between similar patients is impractical in medical imaging, where well-defined measures of semantic similarity between images are lacking. This could lead to forcing similarity between inherently different cases.

Based on your suggestion, we will expand the discussion of fairness metrics for medical image prognosis in the revised manuscript.

References

[1] Shaker, Ammar, and Carolin Lawrence. "Multi-source survival domain adaptation." Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 8. 2023.

[2] Seyyed-Kalantari, Laleh, et al. "Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations." Nature medicine 27.12 (2021): 2176-2182.

[3] Vaidya, Anurag, et al. "Demographic bias in misdiagnosis by computational pathology models." Nature Medicine 30.4 (2024): 1174-1190.

[4] Tian, Yu, et al. "FairSeg: A Large-Scale Medical Image Segmentation Dataset for Fairness Learning Using Segment Anything Model with Fair Error-Bound Scaling." The Twelfth International Conference on Learning Representations (2024).