Learning Causal Alignment for Reliable Disease Diagnosis

Thank you for acknowledging the novelty and significance of our article. We have modified the manuscript as suggested.

Regarding the attributes. Yes, each of the binary variables $a_i$ indicates the presence or absence of an attribute.

Regarding the distance function. We adopt the L1 norm in Eq. (2), which is also suggested by Wachter et al.

In what follows, we conduct experiments to study the impact of different distance functions on counterfactual generation, with results presented in Fig. 7 in the updated manuscript. As we can see, the L1 norm encourages sparse modifications in critical features for malignancy assessment, such as spiculation and margin. In contrast, the squared Euclidean distance (L2 norm) tends to produce uniform changes across the whole nodule. This observation is consistent with the findings of Wachter et al. Moreover, we notice that the performance of L1/L2 norms and their scaled versions are similar, which can be attributed to the fact that we have already normalized the pixel values before training (see Sec. 5.1).

We then show the performance of our method under different distance functions in the following table. The results indicate that the L1 norm outperforms the L2 norm, which can be attributed to the sparser modifications made by the L1 norm, facilitating more accurate localization of causal decision areas.

Reference:

Wachter et al. Counterfactual explanation without opening the black box: automated decisions and the GDPR. https://arxiv.org/pdf/1711.00399

We thank the reviewer for the highly constructive feedback and invaluable suggestions. We address your concerns below.

1. "The main weakness in the paper, which, as I said, can be addressed textually, concerns the causal diagram in Figure 3. This diagram implies that the attributes are independent of each other conditional on the mass ... I don't believe the model needs to be changed, but I feel it needs to be presented as a heuristic choice or a choice about the types of feature sets the authors aim to discover."

We are sorry for the confusion. Indeed, we choose the causal diagram in Fig. 3 to characterize the annotation process of radiologists. According to McNitt-Gray et al. (page 4) and Lee et al. (page 3), radiologists annotate each attribute solely based on image features, which explains the causal edges from $X$ to each $A_i$ and the conditional independence among attributes. We have incorporated this discussion to the article as suggested.

2. "A second comment is that causal search algorithms are available for finding models in the form of Figure 3, and these have not been compared directly for efficacy. The authors could search for these in the literature and explicitly state that perhaps, while they may be effective, a different approach is being taken up and evaluated here, or at least to give reasons why they are not pursued."

We did not use causal discovery algorithms because our goal is to align the model's decision with that of radiologists. Therefore, we need to introduce the causal diagram that describes the decision process of radiologists (inspired by McNitt-Gray et al. and Lee et al.) ahead and use it as supervision for alignment.

We also try the PC algorithm to recover the causal graph from data (see Fig. 8 in the updated manuscript) under the Markov and faithfulness assumptions. We find the skeleton of the recovered graph is consistent with that of Fig. 3.

3. "Multiple masses might occur in a single image, and it is not clear what the method would say in cases like this."

Our method can be adapted for cases with multiple masses. Specifically, if the goal is to classify the malignancy of the patient, we require the experts to additionally provide annotations indicating which masses causally contribute to the diagnosis. We can then apply our method for alignment. In the other case where the objective is to classify the malignancy of each mass, we can utilize existing mass detection algorithms (e.g., Armato et al.) to crop each mass from the images and apply our method to these cropped images.

4. "A fourth comment is that what model class is being assumed here must be explicitly stated. It's a mixed linear Gaussian/discrete regime, though this should be stated, and if that's not the case, then that should especially be stated up front at the beginning of the article."

Our method does not rely on specific parametric models, as both counterfactual generation and the CCCE are model-agnostic. They only require the assumptions outlined in Sec. 4.1 and Appx. A. We have incorporated this clarification into the article as recommended.

Reference

McNitt-Gray et al. The lung image database consortium data collection process for nodule detection and annotation.

Lee et al. A curated mammography data set for use in computer-aided detection and diagnosis research.

Armato et al. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology.

Thanks again for your efforts in reviewing our paper. We hope our responses address your questions and can thus help you reassess our paper. If you have additional questions, we'd be more than happy to provide additional clarification. Thank you for your attention.

Dear reviewer

As the deadline for discussion is approaching, would you mind reading our rebuttal and informing us whether we have addressed your concerns? If you have remaining concerns, we are more than happy for further discussion and clarification. Thanks for your attention.

With regard, Authors

We thank the reviewer for the effort and positive feedback. Below, we address your concerns:

For multi-class scenarios, we can simply chose a $y^*\neq y_0$, without needing to consider different values for $y^*$. This is because the alignment loss always constrains the classifier from altering regions beyond the expert annotations, no matter which $y^*$ is chosen.

For cases with a larger number of attributes, we can first employ feature screening to select attributes that are dependent on $y$, and then compute the CCCE over the selected attributes. Nonetheless, we would like to point out that most medical diagnosis protocols, such as Lung-RADS and Bi-RADS, typically involve a limited number of attributes (usually fewer than ten).

Thanks again for your efforts in reviewing our paper. We hope our responses address your questions and can thus help you reassess our paper. If you have additional questions, we'd be more than happy to provide additional clarification. Thank you for your attention.

We thank the reviewer for the positive assessment. We have modified the manuscript as suggested.

1. "The selection of the set of attributes that are causally related to the label requires further justification ... However, since only six attributes are annotated in both the LIDC-IDRI and CBIS-DDSM datasets, the proposed algorithm may still work even if $r_i=(1,...,1)^\top$ for all $i$."

We attribute only three causes to ensure that the selected features are informative.

2. "The paper claims that previous methods, which regularize the model's input gradient to align with expert-annotated areas (lines 37–40), only capture associational factors. However, the paper later (lines 405–407) attributes the failure of these methods not to their inability to distinguish spurious correlations, but to the limited effectiveness of gradient-based approaches themselves."

We apologize for any confusion. By “limited effectiveness of gradient-based approaches” mentioned in lines 405–407, we refer exactly to the fact that these methods capture only associational factors and do not offer causal explanations. This aligns with the findings in the paper cited in line 407 (Grimsley et al.). We have modified the manuscript for clarity.

3. "In fact, even if the identified image regions are only associational factors, regularizing the gradient to expert annotations will rule out spurious correlations. Although the explanation in Figure 1(a) is conceptually reasonable, the simulation does not explicitly demonstrate the advantage of causal factors over associational factors when expert annotations are leveraged."

We have compared with gradient-based methods (Ross et al.) in Tab. 1, showing significant advantages over associational features when expert annotations are leveraged. Could you please further elaborate on why the simulation does not illustrate the benefits of causal factors?

4. "How the area marks $m$ fit into the causal diagram could be explained further."

We have $X=$ {$X_m, X_{m^c}$}, where $X_m$ denotes the mask area and $X_{m^c}$ denotes the background area. The causal graph is then $X_m \to A \to Y; \,\, X_{m^c}$, where there is no causal edge connected to $X_{m^c}$.

5. "Could the authors provide the computation time for the proposed method compared to the baseline methods?"

The following table shows the computation costs of our method and baselines. As shown, and also noted in the limitations section, our method requires additional training time due to the estimation of the implicit gradient. However, considering the significant improvements in alignment and accuracy that are critical for medical diagnosis, this extra cost can be acceptable for practical applications.

6. "Is the loss function in line 194 optimized directly? From Algorithm 1, it appears that this loss function is optimized in a two-step process, i.e. finding $x^*$ by minimizing $\mathcal{L}_{ce}$ and then finding $\theta$ by minimizing $\mathcal{L}_{align}$."

In Alg. 1, the last line should be “optimize $\theta$ with $\theta\leftarrow \eta \mathcal{L}$” instead of $\mathcal{L}_{align}$. We apologize for the typo. That means, the loss function in line 194 is optimized directly. Steps 2-4 calculate the gradient, and step 5 updates the parameter.