Gradient-based Explanations for Deep Learning Survival Models

Thank you for your valuable and insightful feedback. Before addressing your concerns in detail, we want to clarify a few crucial aspects and potential misunderstandings:

• This is not a benchmark paper.
• We do provide the definition and explanation of local accuracy in the appendix.
• We do provide comprehensive results (as far as meaningful) for all experiments and all methods and for aggregated metrics.
• We do compare with SurvLIME (contrary to the point you raised).

R4A1) What is the objective of this paper?

The objective of our paper is explicitly stated to not be a benchmark paper comparing different gradient-based explanation methods for survival DNNs (see XAI benchmark papers by Liu et al., 2021 and Agarwal et al., 2022 for classic prediction models). Instead, we adapt these methods for survival DNNs, provide method-specific visualizations and interpretations (addressing the recent disagreement problem), and compare GradSHAP(t) as a flexible model-specific version to SurvSHAP(t), making the calculation of Shapley values for survival DNNs possible (see R1A3 and R3A1).

R4A2) Why no comprehensive (aggregated) evaluations for all experiments? Only synthetic data?

A comprehensive evaluation across all methods is not feasible because XAI methods pursue different goals and lack definitive "correct" explanations. For gradient-based methods, this disagreement problem (Sturmfels et al., 2020; Krishna et al., 2023) origins from varying implicit or explicit baselines, making direct comparisons on a local level unreliable (e.g., Grad(t) measures output-sensitivity, GradxInput(t) attributes implicitly against a zero baseline, etc.). We evaluate the methods as local explanation techniques and highlight their characteristics using the introduced visualizations (see Appendix A.1 and A.2 for our comprehensive results). A meaningful aggregation of results across all instances is only possible for Shapley-based methods (as we did for two measures in Sec. 5.2). However, aggregation compromises the local nature of the explanations as it leads to global measures.

Additionally, we mainly used synthetic data for the comprehensive evaluation since it is the only reliable way to verify – in a controlled environment – if the model identifies truly prediction-relevant local effects based on the method's goal. Instead, a benchmark would focus on comparing methods against each other.

R4A3) Marginal technical contribution?

Please refer to the detailed response for Reviewer 3 R3A1).

We acknowledge that we may not have effectively communicated our intended contributions (adoptions for survival including challenges, visualizations and package implementation). In the final paper, we will emphasize and explain these contributions better.

R4A4) Concerns about local accuracy and advantage of GradSHAP(t)

The local accuracy measure for survival outcomes is defined and explained in detail in Appendix A.3.1. We acknowledge the importance of this metric and will incorporate it into the main text. It measures the average decomposition quality of the local attributions of SHAP-based methods (i.e., decomposing $\hat{S}(t|x)-E(\hat{S}(t|x))$). The primary advantage of our approach (GradSHAP(t)) lies in its drastically improved runtime efficiency compared to SurvSHAP(t). In addition to the results provided in our paper in Sec. 5.2. Fig. 6 and Appendix A.3.2., we now conducted an additional runtime comparison between SurvSHAP(t) and GradSHAP(t) as discussed in responses to Reviewer 1: R1A1 and R1A3).

R4A5) Comparison to SurvLIME and other non time-varying methods.

We compared GradSHAP(t) with SurvLIME in terms of global feature importance ranking (see Sec. 5.2, Fig. 7, and A.3.3). While SurvLIME estimates local feature importance values, SurvSHAP(t) and GradSHAP(t) provide local attributions. A direct comparison is therefore only meaningful by evaluating the resulting feature ranking across all instances.

In the related work, we purposefully only explicitly mention survival XAI methods, which are comparable to the gradient-based explanations discussed in the paper. Other survival explainability methods which do not provide time-dependent, local feature attributions are beyond the scope of our paper, including Kovalev et al. (2021) and Utkin et al. (2022), but are discussed in great detail in the referenced comprehensive review by Langbein et al. (2024).

Thank you for your careful evaluation and suggestions. We acknowledge that our contributions may not have been communicated clearly enough in the original submission. To address this, we will revise the manuscript to better clarify these key contributions:

• Our work follows adaptations common in survival XAI but tackles key challenges beyond straightforward mathematical extensions. E.g., a naive application of these methods on CoxTime is not possible.
• We introduce tailored visualizations and post-hoc interpretations for gradient-based survival outputs and exemplify the impact of implicit vs. explicit baselines, addressing the recent debate on the disagreement of gradient-based explanations.
• As a novel contribution, we provide a software package implementing all described gradient-based XAI methods for DeepSurv, DeepHit, and CoxTime.

In the following, we provide a detailed explanation regarding your feedback:

R3A1) Marginal technical contribution?

Our primary contribution is extending six standard gradient-based explanation methods to time-dependent survival analysis, addressing a crucial gap in survival XAI research. The extensions are far from trivial from both a technical implementation and post-hoc interpretation perspective. For instance, in the CoxTime model time is an input feature, allowing for complex relationships between the artificially created time feature and other features. Thus, applying gradient-based methods naively to the survival function $S(t|x)$ separately at each time point $t$ is not feasible, as time-expanded instances are no longer independent, leading to accumulated gradients from earlier time points. This computational difficulty is not captured in the formal mathematical adaptions. Similar methodological adaptations are standard practice in survival interpretability research (see Kovalev et al., (2020), Krzyzinski et al., (2023); even the time-dependent local accuracy metric in Krzyzinski et al., (2023) follows a similar extension from the original Shapley value axiom.

Furthermore, another major contribution of our work are effective visualization and interpretation techniques for functional outputs tailored to different methods. This is particularly important given the ongoing debate and disagreement regarding gradient-based methods (Sturmfels et al., 2020; Krishna et al., 2023; Koenen et al., 2024). Our work contributes to this discussion by clarifying how implicit or explicit baselines in these methods influence survival explanations and provides practical guidance on selecting appropriate techniques based on their interpretability characteristics.

Finally, we highlight our R package, Survinng, as an additional contribution. Existing libraries like Captum and innsight do not natively support survival DNNs, necessitating custom implementations. Our package, which includes all described gradient-based XAI methods for DeepSurv, DeepHit, and CoxTime, will be made available with the final version of this paper.

We appreciate the suggestion to include semiparametric AFT-based survival deep learning models, such as Deep AFT Rank-regression for Time-to-event prediction model (DART). It is an interesting approach, which estimates the survival function in similar fashion to a non-Cox-based version of the DeepSurv model. However, the pre-trained baseline hazard function additionally depends on the output of the base neural network, as highlighted in Eq. 9 in the paper by Lee et al. (2023). Unlike Cox-based methods, such as DeepSurv and CoxTime, this formulation allows for gradient computation on the baseline hazard, which could provide further insights into the differences between Cox and non-Cox methodologies. Given its potential, we aim to explore its integration and explanations in future research.

Our current focus is on methods that are readily accessible to practitioners, particularly those implemented in widely used libraries such as Pycox (Python) and survivalmodels (R). Expanding these packages or developing more comprehensive survival analysis software to also include AFT models such as DART is an important direction, and we are actively working toward addressing this gap.

More broadly, extending gradient-based XAI methods to AFT-based deep learning models like DART is an exciting avenue for future work. We will highlight this in the paper's future work section.

We would also like to take the opportunity to highlight that we conducted additional comparisons of SurvSHAP(t) and GradSHAP(t) on real data. In the multi-modal model (Sec. 5.3), SurvSHAP(t) was aborted after 10 hours (256 threads, 700GB RAM) with only two reference samples, while GradSHAP(t) completed in ~8 minutes using 100 reference and 20 integration samples. See our response to Reviewer 1 (R1A3) for details.

Thank you very much for the positive feedback!

We are grateful for your constructive feedback. In response, we conducted additional experiments on the feasibility and computational efficiency of GradSHAP(t) and SurvSHAP(t) on the multi-modal real data example, which we will include in the final paper. Here, GradSHAP(t) took 5 minutes to compute, while we had to abort the computation of SurvSHAP(t) after 10 hours (details below). In light of these results, we argue that runtime is one of the most crucial aspects for post-hoc Shapley value approximation, which is supported by previous literature. Additionally, we want to highlight that local accuracy and importance ranking are already aggregated metrics. The following summarizes all your concerns in-depth:

R1A1) Limited gain of GradSHAP(t) relative to SurvSHAP(t)?

The objective behind GradSHAP(t) is not merely to propose a "better" method, but rather a practical one that balances flexibility and computational feasibility. Although differences in local accuracy are visible due to the log scale, they are practically negligible, whereas the runtime improvement of GradSHAP(t) is substantial. This becomes particularly apparent in our additional experiments, including image data, which demonstrate that SurvSHAP(t) quickly becomes computationally infeasible without substantial resources, whereas GradSHAP(t) remains efficient and can be computed on a standard laptop (for further details refer to R1A3). Computational runtime is a well-established criterion in post-hoc XAI methods, particularly for SHAP explanations. It is frequently used as a key selling point in the scientific literature when introducing new estimation methods, see e.g., TreeSHAP (Lundberg et al., 2020), FastSHAP (Jethani et al., 2021) or various other SHAP algorithms such as in Ancona et al. (2019) or Chen et al. (2018) and runtime is included as a comparison metric in their SHAP benchmarking suite (Python SHAP package, Lundberg and Lee, 2017).

R1A2) Limited number of examples/lack of aggregated metrics?

Our comparison of SurvSHAP(t) and GradSHAP(t) was performed for all three survival DNN classes (see Appendix A.3 for full plot) with varying numbers of input features, averaged over 20 trained models per case. In spite of its name, local accuracy is already a dataset-wide aggregated measure (see Eq. 10, Appendix A.3.1.), which we plot for $p=30$ over the survival time in Fig. 6. Additionally, Figure 7 presents a comparison of global feature importance, particularly importance rankings, which are also aggregated over survival time. Could you clarify if you were referring to something beyond these analyses?

R1A3) Real data?

We performed additional comparisons of SurvSHAP(t) and GradSHAP(t) on real data. On the multi-modal model (including images) for explaining a single instance (Sec. 5.3), we had to abort the computation of SurvSHAP(t) after 10 hours (using 256 threads and 700GB of RAM) with only two reference samples. In contrast, GradSHAP(t) completes in around 8 minutes with 100 reference and 20 integration samples. As an additional comparison, we conducted the same experiment (but ResNet18) on downscaled images (32×32) using a standard ML workstation (48 threads, 256GB RAM). As shown in the table below, SurvSHAP (with 50 samples) takes more than 41 times longer than GradSHAP (n = 25, samples = 50), which is nearly 25 minutes for just a single explanation compared to 36 seconds. GradSHAP achieves this not only faster but also with better instance-wise local accuracy (averaged over survival time $t$, i.e., no aggregation over all instances).

We agree that this is crucial information for the readers and we will include discussions and visualizations of the computational efficiency of both methods on real data in the final version of the paper.

We are also happy to provide further clarification on Figure 8, if you further specify which aspects remain unclear from the figure and/or text.