Characterizing Exceptional Distributions with Neural Rule Extraction

We appreciate the detailed feedback and the kind suggestions.

I. Density notation

In our work we do introduce the used quantities as densities, but leave implicit the measure-theoretic formalisation as it is not necessary for the needs of this work. As for the divergences, indeed, previously we slightly abused notation to avoid the introduction of the laws, again, as these are not used elsewhere in the manuscript. To avoid confusion, however, we now introduce this notation in the updated manuscript at the end of the preliminary section (Sec 2), so that we can now adopt the usual notation.

II. Defining S

The latent r.v. S was introduced in the manuscript in the last part of sec. 2. as a (latent) binary r.v. that represents the membership in a subgroup. However, we agree that it could have been missed or been too terse. To rectify this problem we clarify its role in our model, as well as the underlying model in our inference in the beginning of section 3.1, in addition to the preliminaries.

III. Typos

We thank the reviewer for the notes. We have incorporated them all in the revised manuscript. 4. 5. We believe our original derivation is correct. However, we acknowledge that the original writeup could be improved. We therefore moved the derivation to the appendix, and, given the added space, we explain the two mechanisms that need different bounds, and also demonstrate why the former bound is not depending on $C_X$.

IV. Experimental Results

1. Number of features in synthetic data
   There are two main reasons behind the maximum of 10 features which define the subgroup. Firstly, the premise of our work (and subgroup discovery in general) is that the results have to be announced to a human, and therefore fewer features lead to more succinct descriptions and are therefore considered superior, from the interpretability perspective. The number of 10 was chosen as, intuitively, we considered it as a round number that was already past the limit of what could reasonably be interpreted by a human expert (e.g., doctor).
   Secondly, increasing the amount of features for thresholding also comes with a data generation problem. Imposing a rule with 100 conditions on independent features would lead to an interval that covers the entire feature domain ($0.2^(1/100)≈1$), or if the features are correlated, extreme redundancy in which feature to actually threshold. This would allow a different process to start becoming apparent, in which arbitrary features would simply become relevant simply by random chance, and necessitate additional mechanisms for testing (e.g., statistical corrections for random correlation).
2. KL vs other metrics
   We agree with the reviewer that comparing to methods that optimise different objectives is inherently difficult. However, KL divergence is a more general measure of distribution dissimilarity than other widely used criteria; for instance, when assuming Gaussian target distributions, then there is a monotone bijection between KL and difference of means, while in other cases we could easily craft a case where the KL divergence exceeds an arbitrary bound, while the mean shift remains close to zero. In other words, a low mean difference does not imply that the resulting subgroup is not interesting, as in the exponential wage distribution example from Fig. 3b.
3. Stability of results The issue came primarily from a small number of iterations on similar datasets. We increased the number of iterations and used a newly generated randomised dataset for each of them to now improve the smoothness of the experimental measurements.
4. Description of baselines We fixed this in the revised manuscript.

V. References

1. , 2. We have added the suggested references to the introduction.
2. That would be the Proença et al. We rephrased the relevant sentence to make clear that we were still referring to the last mentioned method.

VI. Questions

1. Please see II. This has been incorporated in the revised manuscript in preliminaries (last sentence) and Sec. 3.1 (first sentence).

2. Algorithms Briefly, we first fit a normalising flow to the entire data, then initialise $p_{Y|S=1}$ to the former model, and perform SGD steps that jointly update both the conditional target model and the membership rule. To avoid the gradient of the rules from vanishing, we follow a simulated annealing scheme, by starting with a very moderate temperature parameter.

3. Although our model is primarily aimed at real features, ordinal and binary features can also easily be incorporated, by simply treating the values as real. In our experiments we treat the features as real numbers, resulting in logistic-like regression. For discrete data we resort to one-hot encoding, where we create a binary variable that indicates the presence/absence of each discrete value. This allows SYFLOW to easily select individual values for its rules and matches the ability of standard subgroup discovery methods for dealing with discrete data.

Thanks to the authors for their response. Some improvements have been made in the revised manuscript, but I will maintain my score for the following reasons:

Math: There are still many technical typos which make the results difficult to understand. Some examples:

1. (First paragraph of Section 3.1) "That is, from the statistical inference perspective, we assume that $S|\mathbf{X} = \mathbf{x} \sim \textrm{Bernoulli}(\sigma(\mathbf{x}; \alpha, \beta))$. The Bernoulli parameter should be $s$, not $\sigma$, since $\sigma$ is the 0-1 (hard) subgroup rule. (If $\sigma$ is correct, then $S = \sigma(\mathbf{x}; \alpha, \beta)$ with probability 1.)
2. (Theorem 2) The $d\mathbf{x}$ was placed on the wrong side of the inequality, instead of with the integral.
3. $S$ is not a single random variable, but rather a random function that depends on $\mathbf{x}$.

It's possible that these problems can be resolved with just notational fixes and not by changing the fundamental content, but it is not the reader's job to do so. The paper should be carefully edited so that all mathematical statements are precise and correct before it is published.

Experiments: There are still no error bars on most of the results, so the significance is still unclear. Regarding the choice to report KL divergence vs difference in some common statistics, I agree with the authors that one can construct examples where these statistics will not differ even though the distributions are quite different. I would encourage you to report these results (or perhaps some other metrics quantifying difference between probability distributions, e.g. Wasserstein distance) in addition to the KL divergence to show that your method has some amount of robustness to the measure of dissimilarity.

I believe the method is promising and I encourage the authors to resubmit after making these changes.

Thank you for your detailed feedback.

1. The latent random variable S was introduced in the manuscript in the last part of Sec. 2. as a (latent) binary random variable representing the membership of a sample to the subgroup. We see that this is both terse and easy to miss. We therefore now discuss its role in the model and during inference in more detail in the beginning of Section 3.1 of the updated manuscript.
2. We now give an introduction to describe the baselines in the beginning of the experiment section.
3. We now describe the hyperparameters, data generation and training procedure in the Appendix of the revised manuscript.

We thank the reviewer for their kind feedback. In regards to the posed questions:

1. Currently, a subset is limited to one continuous interval per feature, so that we would need two subsets to capture two ranges right now. Our newly introduced regularizer helps find such contrasting subsets.
2. We have added a description of the baselines to the revised manuscript and a discussion on failure modes/limitations in the respective Appendix.

Further writing such as making the abstract more concise and updating the captions are underway and will be incorporated as soon as possible. Thank you very much for your helpful feedback.

Thank you for your detailed feedback.

1. Although it may be interesting to find multiple subgroups, we note that it is not our main objective. We here focus on developing a method that can: 1) deal with arbitrary target distributions, 2) find symbolic and highly interpretable descriptions of numeric features in a differentiable end-to-end fashion and 3) significantly reduce the runtime compared to state-of-the-art baselines.
   Nonetheless, our method can be extended in a straightforward manner to find a diverse set of subgroups. The main idea is to iteratively search for subgroups and extend the optimization objective with a regularizer of the form $KL(P_{Y|S} || P_{Y|S_{i-1}})$, described in Appendix C. This allows us to find subgroups that differ from the overall distribution and simultaneously differ from previously found subgroups.
2. The trade-off between explainability and expressivity is a general problem in the area of interpretability. The rules/descriptions we are interested in, i.e. boolean predicates over binned features, have clear and actionable meaning. The subgroup in Fig 3b, for example, is characterized by the rule “education < 16 AND I(Female)”, permits us to say for any current or future data point whether it is part of the subgroup or not. Rules of the suggested form, e.g. $0.5*$education + I(Female), have a vastly different meaning. This includes the inclarity of what “$0.5*$education” would mean in terms of which samples are in or outside the subgroup, and, the inclarity of what the semantics of adding a real-valued literal to a boolean-valued variable would be. While for classification these aspects perhaps make sense, it does not for descriptive purposes.
   Nonetheless, if such linear rules are of interest, SyFlow can be easily adapted by replacing the rule layer with a linear layer. In general, the rule layer can be replaced by any function that maps features to a probability in a differentiable manner.
3. We show in the covid experiment the ability of SyFlow to adjust for factors which may be not clinically relevant, by means of searching the joint distribution of target quantity (e.g. survivial) and factor to adjust for (e.g. age). This advantage is unique to SyFlow, as it is the only method that is designed for non-gaussian multi-variate data.
4. They are not. Our goal is description, rather than prediction, and hence cross-validation does not make much sense. After all, the point in description (unsupervised) is to provide insight into the available data, rather than achieving the best possible generalization to unseen data, as is the point in predictive (supervised) settings. To show that our method discovers subgroups with exceptionally distributed target variables, we report the size-corrected KL-divergence of the top subgroup per method
5. This is a fair point. We will include the discovered rules in the Appendix. Note, however, that these results only make sense when the discovered subgroups have meaningfully high divergences; for wine and wages, for example, the competing methods all fail to discover a subgroup with an exceptional distribution of the target variable, so it makes little sense to inspect these rules.
6. We added a Limitations section to the Appendix, and propose to prune some of the figures in the experiments to make space for this in the main manuscript.

For your questions:

1. We test SyFlow across a variety of data distribution types as well as on a diverse set on real world datasets, where it is consistently amongst the top methods.
2. We are not exactly sure what you mean by this, but in our experiments SyFlow recovers a planted ground truth subgroup (interpretable as class) accurately.

Thank you for your detailed feedback.

1. Eq (2) does not involve the subgroup rule function s, while Eq (7) does and therewith gives us gradients to optimize the rule s.
2. We have added a paragraph describing the baseline methods to the updated manuscript.
3. Although it may be interesting to find multiple subgroups, we note that it is not our main objective. We here focus on developing a method that can: 1) deal with arbitrary target distributions, 2) find symbolic and highly interpretable descriptions of numeric features in a differentiable end-to-end fashion and 3) significantly reduce the runtime compared to state-of-the-art baselines. Nonetheless, our method can be extended in a straightforward manner to find diverse sets of subgroups! The main idea is to iteratively search for subgroups and extend the optimization objective with a regularizer of the form $KL(P_{Y|S} || P_{Y|S_{i-1}})$, described in Appendix C. This allows us to find subgroups that differ from the overall distribution and simultaneously differ from previously found subgroups.
4. The effect of quantisation is shown in the first experiment of the synthetic experiments (Feature Thresholding), where we vary the number of cut points for subgroup discovery (SD-$\mu$). The results are displayed in Figure 1.b. For a small number of cutpoints (coarse quantization), the performance is significantly worse than for a finer quantization. To match the performance of SyFlow, SD-$\mu$ requires 40 cutpoints, while the runtime increases by three orders of magnitude.