Everyone Deserves Recourse: Feasible Recourse Paths Using Data Augmentation

• Formulation: The paper doesn’t motivate why searching for distance functions helps, in particular because initial $h^*$ is obtained by assuming distance function $ell_1$ as described in lines 381-388
• Writing: In line 389, “”learning $d,\tau$”, there is no mention of $\tau$.Crucial Details on how transition weights are created is missing(line 395).
• Limited Experimental Results: Authors do not compare their methods to any single step algorithm other than nearest neighbours, while several papers on path based recourse are mentioned in related works sections.

• Privacy Concerns with FACE as a Basis: Using FACE as the foundational algorithm raises significant privacy concerns, as the method inherently relies on traversing data points from other users, which exposes sensitive information of other unsuspecting users. This risk, highlighted in recent studies (e.g., [1]), casts doubt on the real-world applicability of FACE-based recourse due to its clear privacy violations.

• Limited Evaluation Scope: The empirical evaluation lacks breadth and depth. The comparison is limited to two other recourse algorithms, neglecting the diverse set of algorithms in the CARLA library [2] that could provide a more thorough assessment of the proposed method's effectiveness. For example, any gradient based algorithm can be return a recourse path. Hence, the algorithms in [2] serve as natural baselines for comparison. Additionally, the core evaluation is restricted to a logistic regression model. Since the authors claim the method is model-agnostic, it would be valuable to test all datasets on other model types, such as neural networks and tree-based classifiers, to validate its general applicability.

• Inadequate Handling of Categorical Variables: The paper does not sufficiently address how categorical variables are managed, which is crucial in tabular datasets where they are common. Without clarification, it remains unclear whether the proposed approach is suited for real-world applications that often involve categorical features.

• Not Useful and Disconnected Theoretical Contributions: The theoretical results in Section 3 are not well-integrated with the empirical findings. While they cover basic insights, such as bounding the VC dimension of a specific hypothesis class, these insights don’t directly support the method's empirical performance. Additionally, the feasibility learning appears overly complex without practical benefits, especially given the lack of efficient algorithms for managing a large set of parameterized distance functions. Fruther, the paper does not provide theoretical guarantees that the synthetic data augmentation will remain within the data manifold. Such guarantees are necessary to ensure that the generated samples represent realistic and plausible data points, enhancing the method's reliability and applicability.

References

[1] Pawelczyk et al (2023), On the Privacy Risks of Algorithmic Recourse, AISTATS 2023, https://arxiv.org/abs/2211.05427

[2] Pawelczyk et al (2021), Carla: a python library to benchmark algorithmic recourse and counterfactual explanation algorithms, NeurIPS 2021, https://arxiv.org/abs/2108.00783

1. Recourse problem is a human-centric problem. The paper lacks the human-understanding of why the graph is not connected. Simply augmenting points to make the graph connected could not solve the root cause of the problem.

2. The augmented nodes lacks interpretability. There is no argument to support that the new nodes correspond to feasible features in the real world. Thus, one could not guarantee that the proposed path is realistic.