An Entropic Risk Measure for Robust Counterfactual Explanations

We thank the reviewer for their positive evaluation of our work and their comments and suggestions. We also thank the reviewer for acknowledging that our research problem is “relevant” and our approach is “novel and promising”.

(1 of 3) Regarding the gradient and computation complexity: The algorithm does not use the actual gradients but only the estimates. Our approach shares comparable computational complexity with T-Rex. As discussed in Section 5, it is important to note that the reliance of our method on sampling and gradient estimation introduces a downside, namely, an elevated level of computational complexity.

(2 of 3) Regarding the inclusion of additional metrics: We appreciate the reviewer's insight regarding the inclusion of additional metrics like actionability, plausibility, and fairness. This consideration is acknowledged in the final sentence of the first paragraph of Appendix C, where we recognize the importance of addressing these factors beyond explainability and robustness. Dealing with multi-criteria optimization involves techniques from multi-objective optimization, and we plan to delve into the necessary trade-off analysis using multi-criteria optimization in future research. Thank you for sharing this with us. We make sure to explain these thoroughly in an earlier part of the paper.

(3 of 3) Regarding readability and understandability: We acknowledge the importance of a readable and understandable manuscript. We are committed to incorporating your suggestions.

We thank the reviewer for their comments and questions.

(1 of 3) Regarding our assumptions: We have not assumed a known probability distribution. We recognize the inherent challenge in verifying the MGF-equivalence assumption. As acknowledged on page 7, we intentionally choose relatively strong assumptions on MGFs within this framework, primarily for illustrative purposes to showcase the potential for algorithmic advancements based on our theoretical insights. Relaxing this assumption, though demanding more intricate proofs, paves the way for less assertive yet still valuable probabilistic guarantees.

(2 of 3) Regarding the choice of the risk measure and its implications: Entropic risk measure enables the connection to worst-case robust approaches and offers a unifying perspective. Also, it is known the entropic risk measure works with the tail of the distribution. Analogous to the significance of Renyi mutual information over mutual information [Mironov (2017); Baharlouei (2020)] that arises in other domains such as fairness, privacy, communications, etc. Reyni measures introduce a tunable $\alpha$ parameter to bridge between worst case and average case. In the limit of $\alpha$ going to infinity, the measure converges to the worst-case expression (maximum), while in the limit of $\alpha$ going to 1, the measure converges to an average-case expression (mutual information).

Similarly, the significance of our contribution lies in the insights that our measure offers for the theoretical understanding of robust counterfactuals, along with the novel unification it brings to various approaches within this domain. Our metric has a ‘knob’, the risk parameter $\theta$, that can be adjusted to trade-off between risk-constrained and worst-case/adversarially robust approaches, depending on where the user wants to be. While risk-constrained accounts for general model changes in an expected sense, the adversarial robustness prioritizes the worst-case perturbations to the model, thus having a higher cost. Our approach enables one to tune “how much” a user wants to prioritize for the worst-case model changes, by trading off cost. The extreme value of the knob (risk parameter $\theta$) maps our measure back to a min-max/adversarial approach.

Notably, (i) Without such a tunable knob, it is not possible to have the flexibility to tradeoff between Risk-Constrained and Worst-Case. To the best of our knowledge, existing Risk-Constrained measures, e.g., T-Rex, SNS, etc. do not have such an analogous tunable parameter that enables them to mathematically converge to Adversarial (Worst-Case) Robustness. To clarify, they do have other hyperparameters but varying them would not lead to the mathematical equivalence to Adversarial (Worst-Case) Robustness. (ii) It also enables us to characterize the pareto-optimal front and balance cost and risk by varying the tunable risk parameter $\theta$ which is not present in Hamman et. al. [2023].

[Mironov (2017)] Mironov, Ilya. "Rényi differential privacy." 2017 IEEE 30th computer security foundations symposium (CSF). IEEE, 2017;

[Baharlouei(2019)] Sina Baharlouei, Maher Nouiehed, Ahmad Beirami, Meisam Razaviyayn: Rényi Fair Inference. ICLR 2020

(3 of 3) Regarding the distributionally robust optimization: The literature recognizes the equivalence between risk-optimization and distributionally robust optimization; for instance, the entropic risk measure is equivalent to a distributionally robust formulation with a KL-divergence ball as the ambiguity set. Different risk measures correspond to distinct ambiguity sets. Notably, risk measures prove more amenable to gradient descent-based techniques compared to distributionally robust optimization methods.

We thank the reviewer for their feedback and for acknowledging that the paper is “clearly written”, the problem is “relevant,” our idea is “innovative,” and our results are “detailed” and “non-trivial”.

(1 of 5) Regarding the choice of risk measure: Entropic risk measure enables a theoretical connection between our method (entropic-risk-based approach) and the worst-case robust counterfactuals (min-max optimization), e.g. ROAR, unifying these two classes of robust counterfactual methods under model changes. The novelty of our measure is analogous to the significance of Renyi mutual information over mutual information [Mironov (2017); Baharlouei (2020)] that arises in other domains such as fairness, privacy, communications, etc. Reyni measures introduce a tunable $\alpha$ parameter to bridge between worst case and average case. In the limit of $\alpha$ going to infinity, the measure converges to the worst-case expression (maximum), while in the limit of $\alpha$ going to 1, the measure converges to an average-case expression (mutual information).

Similarly, the significance of our contribution lies in the insights that our measure offers for the theoretical understanding of robust counterfactuals, along with the novel unification it brings to various approaches within this domain. Our metric has a ‘knob’, the risk parameter $\theta$, that can be adjusted to trade-off between risk-constrained and worst-case/adversarially robust approaches, depending on where the user wants to be. While risk-constrained accounts for general model changes in an expected sense, the adversarial robustness prioritizes the worst-case perturbations to the model, thus having a higher cost. Our approach enables one to tune “how much” a user wants to prioritize for the worst-case model changes, by trading off cost. The extreme value of the knob (risk parameter $\theta$) maps our measure back to a min-max/adversarial approach. [Mironov (2017)] Mironov, Ilya. "Rényi differential privacy." 2017 IEEE 30th computer security foundations symposium (CSF). IEEE, 2017;

[Baharlouei(2019)] Sina Baharlouei, Maher Nouiehed, Ahmad Beirami, Meisam Razaviyayn: Rényi Fair Inference. ICLR 2020

Notably, (i) Without such a tunable knob, it is not possible to have the flexibility to tradeoff between Risk-Constrained and Worst-Case. To the best of our knowledge, existing Risk-Constrained measures, e.g., T-Rex, SNS, etc. do not have such an analogous tunable parameter that enables them to mathematically converge to Adversarial (Worst-Case) Robustness. To clarify, they do have other hyperparameters but varying them would not lead to the mathematical equivalence to Adversarial (Worst-Case) Robustness. (ii) It also enables us to characterize the pareto-optimal front and balance cost and risk by varying the tunable risk parameter $\theta$.

(2 of 5) Regarding the benchmarking: Our main contribution is to demonstrate a theoretical connection between our method (entropic-risk-based approach) and the worst-case robust counterfactuals (min-max optimization), e.g. ROAR. Our experiments simply aim at showcasing that our method performs at par with the state-of-the-art, SNS algorithm, while being grounded in the solid theoretical foundation of large deviation theory, without relying on any postulations; enabling a unifying theory for robust counterfactuals. We acknowledge the existence of several intriguing techniques in robust counterfactuals that might surpass our strategy in specific scenarios. However, the practical implications of model change remain uncertain. Our fundamental innovation lies in making a theoretical contribution that facilitates a nuanced trade-off with robustness.

We thank the reviewer for their feedback. We appreciate their acknowledgment of our work being “relatively well-written” and the problem being “important in the literature”. We are committed to further refining it based on their constructive suggestions.

(1 of 6) Regarding novelty and significance of the proposed method: The novelty of our measure is analogous to the significance of Renyi mutual information over mutual information [Mironov (2017); Baharlouei (2020)] that arises in other domains such as fairness, privacy, communications, etc. Reyni measures introduce a tunable $\alpha$ parameter to bridge between worst case and average case. In the limit of $\alpha$ going to infinity, the measure converges to the worst-case expression (maximum), while in the limit of $\alpha$ going to 1, the measure converges to an average-case expression (mutual information).

Similarly, the significance of our contribution lies in the insights that our measure offers for the theoretical understanding of robust counterfactuals, along with the novel unification it brings to various approaches within this domain. Our metric has a ‘knob’, the risk parameter $\theta$, that can be adjusted to trade-off between risk-constrained and worst-case/adversarially robust approaches, depending on where the user wants to be. While risk-constrained accounts for general model changes in an expected sense, the adversarial robustness prioritizes the worst-case perturbations to the model, thus having a higher cost. Our approach enables one to tune “how much” a user wants to prioritize for the worst-case model changes, by trading off cost. The extreme value of the knob (risk parameter $\theta$) maps our measure back to a min-max/adversarial approach.

Notably, (i) Without such a tunable knob, it is not possible to have the flexibility to tradeoff between Risk-Constrained and Worst-Case. To the best of our knowledge, existing Risk-Constrained measures, e.g., T-Rex, SNS, etc. do not have such an analogous tunable parameter that enables them to mathematically converge to Adversarial (Worst-Case) Robustness. To clarify, they do have other hyperparameters but varying them would not lead to the mathematical equivalence to Adversarial (Worst-Case) Robustness. (ii) It also enables us to characterize the Pareto-Optimal front and balance cost and risk by varying the tunable risk parameter $\theta$ which is not present in T-Rex-like algorithms.

[Mironov (2017)] Mironov, Ilya. "Rényi differential privacy." 2017 IEEE 30th Computer Security Foundations Symposium (CSF). IEEE, 2017;

[Baharlouei(2019)] Sina Baharlouei, Maher Nouiehed, Ahmad Beirami, Meisam Razaviyayn: Rényi Fair Inference. ICLR 2020

(2 of 6) Regarding the experiments: Our main contribution is to demonstrate a theoretical connection between our method (entropic-risk-based approach) and the worst-case robust counterfactuals (min-max optimization), e.g. ROAR. Our experiments simply aim at showcasing that our method performs at par with the state-of-the-art while being grounded in the solid theoretical foundation of large deviation theory, without relying on any postulations; enabling a unifying theory for robust counterfactuals. We acknowledge the existence of several intriguing techniques in robust counterfactuals that might surpass our strategy in specific scenarios. However, the practical implications of model change remain uncertain. Our fundamental innovation lies in making a theoretical contribution that facilitates a nuanced trade-off with robustness.