Interpretability-driven active feature acquisition in learning systems

Response to W1 & Q1: We thank the reviewer for this insightful comment. As noted, our conclusions are primarily based on empirical observations derived from the Oracle results, which demonstrate that our approach outperforms other AFA methods. Intuitively, we assumed that a model with reasonable task performance would inherently emphasize the most important features, and this hypothesis was empirically validated through our Oracle experiments. However, we acknowledge the limitations of SHAP as a feature ranking method, including its dependency on the underlying model and the concerns highlighted by other reviewers. To further validate our assumptions, we conducted an ablation experiment on the CIFAR-10 dataset. Specifically, we used ResNet-10, a smaller model compared to ResNet-18, as the pre-trained model for determining the SHAP ranking order, while retaining ResNet-18 as the classification network. We observed that the performance of our method decreased from 78.61% to 78.22% on the test set, and from 79.12% to 78.42% on the validation set. These results indicate that the model’s capacity to emphasize important features impacts the SHAP-based ranking order and, consequently, the performance of our method. This supports our hypothesis that models with strong task performance inherently prioritize important features, even when relying on SHAP-based rankings.

Response to W2: Recalculating feature attribution values dynamically as features are acquired could capture changing dependencies and potentially improve the feature acquisition order. However, recalculating SHAP values dynamically is computationally prohibitive due to the exponential number of subsets involved. Methods such as Kernel-SHAP and FastSHAP approximate SHAP values by sampling a subset of these differences or using deep learning models for efficient estimation. Due to these constraints, we opted to keep the SHAP ordering fixed during training. Despite this limitation, our empirical results demonstrate that the fixed SHAP ordering outperforms state-of-the-art AFA techniques in terms of accuracy for a given number of features (see Oracle performance comparisons in the results section). This suggests that even with a fixed SHAP ordering, our method effectively captures meaningful feature importance rankings for AFA. We acknowledge that exploring dynamic recalculations of feature importance remains an open area for future research, and we appreciate the reviewer for highlighting this potential avenue for improvement. We incorporated this point as a potential direction for future research in the revised manuscript.

Response to W6: We did not perform detailed parameter search on the experiments. We selected the context length $\ell$ parameter by comparing the validation scores on CIFAR-10 and the other parameters were selected heuristically by hand. Subsequently, all these parameters were fixed for all the experiments. About the $\ell$ parameter selection, we found that our model achieved accuracies of 78.41%, 78.76%, 79.12%, and 78.12% on the CIFAR-10 validation dataset with $\ell=1$, $\ell=2$, $\ell=4$, and $\ell=8$, respectively. Based on these observations, we assigned $\ell=4$. We added these details and pseudocodes for our training strategies (in the Appendix) in the revised manuscript.

Response to W3, W4 & W5: Please see the common response.

Response to Q2: The static global baselines often fail compared to the dynamic baselines because static global feature rankings do not inherently capture the interactions between the features at the instance level. This was our motivation behind creating a new AFA framework that acquires features based on instance-specific ranking. While we agree that we can improve our framework by recalculating feature attributions after each acquisition considering the interaction between the features, this approach could be computationally expensive. Our experimental results, especially that of the oracle network, show the effectiveness of how acquiring features based on instance-specific ranking, even without recalculations, can largely improve the performance of the AFA framework. Please check the common response for more information.

Response to Q3: We thank the reviewer for asking this question seeking clarification on how we handle re-acquisition. To select features only from the unacquired feature set, during both training and inference, we subtracted $m \cdot \text{inf}$, where $m$ is the mask and $\text{inf}$, from the output logits of the policy network before applying the softmax layer. This was done to suppress the logits of the acquired features. In the paper, we mentioned that the subset of features within the acquisition pool is taken by a set difference of the features and the mask:

$\arg\max q_\pi(\mathbf{x}_M) \in [d] \backslash M$

Due to this, the acquired index $\arg\max q_\pi(\mathbf{x}_M)$ will not be one that has been previously acquired.

Response to W1: We thank the reviewer for raising this important point. While SHAP may not be the only available method for determining instance-wise feature importance, we empirically observed that an ideal (oracle) policy network that sequentially selects features based on their SHAP values, ranked from highest to lowest, outperforms current state-of-the-art active feature acquisition (AFA) techniques in terms of accuracy for any fixed number of features (see oracle performance comparisons in our results). In the AFA domain, determining the optimal feature acquisition order for each instance remains a significant challenge due to the lack of ground truth (or reference standard) for feature importance rankings. To the best of our knowledge, no prior work provides theoretical guarantees for the optimality of feature acquisition orders. For example, the GDFS method provides theoretical justification for recovering a greedy policy based on its objective functions but does not prove the optimality of the greedy policy itself in the AFA problem. In addition, intuitively, we assumed that a model with reasonable task performance would inherently emphasize the instance-wise most important features, and explanation methods can identify those emphasized features. Through our oracle experiments, we showed that SHAP provides a relatively better ranking order compared to current state-of-the-art methods, making it a practical and effective choice for our approach despite its limitations. These Oracle experiments further support our assumption that models with strong task performance inherently prioritize important features, which can be effectively identified using explanation methods.

Response to W2 & Q1: During training on the image datasets, we applied random augmentations. Because of the superior performance of our method on these datasets, we believe our method is robust to input perturbations. Also, in the experiments, we used three different types of architectures: MLP, CNN, and ResNets (both ResNet18 and ResNet50), which also show that our method can be integrated with various architectures. Additionally, we performed an ablation study to evaluate the effectiveness of using decision transformer and the robustness of our method with different architectures on the policy network. Also, on our new tabular dataset results, we show the performance of our method tested on other feature ranking methods, such as LIME, and different SHAP calculation methods (TreeSHAP vs KernelSHAP). For example, the results with varying approaches of explanation and SHAP calculations show that our model is dependent on the quality of the ranking order provided by the explanation methods.

Response to W1 & Q2 : We appreciate reviewer's suggestion to explore medical datasets. Please see the common response.

Response to W2 & Q1: We appreciate the reviewer’s thoughtful feedback regarding the use of SHAP and the suggestion to consider metrics like mean decrease in accuracy. Our approach is motivated by the need to balance both global and instance-specific perspectives of feature importance. While mean decrease in accuracy is effective for assessing a feature's global impact on predictive performance, it does not capture instance-level variations, which can be crucial in certain contexts. For example, a feature that is consistently important across all instances might not provide significant additional information for certain model updates, whereas instance-specific variations can highlight features that are critical for refining predictions in localized regions of the feature space. Our acquisition policy leverages an instance-specific ranking of feature importance, which inherently accommodates both scenarios: (a) For features with consistent importance across the dataset, our policy prioritizes them naturally based on their contribution to overall performance. (b) For features with instance-specific importance, our policy effectively adapts to their variability and ensures targeted acquisition that maximizes information gain. To validate this, we implemented an “Oracle” network as a baseline to demonstrate how acquiring features based on instance-specific rankings can substantially enhance the performance of the AFA framework. Our results indicate that instance-specific ranking offers significant benefits over static global approaches. We will provide a detailed discussion of these results in the revised manuscript, emphasizing how our approach balances global and local perspectives while maintaining robustness to the reviewer's highlighted concern.

Response to W1: We appreciate the reviewer’s insightful comment and agree that SHAP values may have some limitations, as highlighted in the referenced works. However, we would like to emphasize the following points: (a) Our proposed method is agnostic to the specific ranking mechanism used. Any improvement in feature importance ranking, whether through SHAP or alternative methods, directly enhances the performance of our framework. Thus, our approach can easily adapt to advancements in local explanation techniques. (b) While local explanation methods like SHAP are one of the many approaches to determine feature importance, our experiments demonstrate that they can effectively generate instance-specific feature ranking orders. This is particularly useful in guiding feature acquisition policies. (c) We selected SHAP due to its widespread adoption and robust theoretical foundation, making it a natural baseline for comparison. Furthermore, approximation techniques like FastSHAP significantly enhance its computational feasibility, making it practical for our use case. (d) We acknowledge the limitations of SHAP highlighted in the cited works. However, our method is designed to seamlessly incorporate more accurate or specialized ranking techniques as they become available. This ensures that the framework can continuously improve alongside advancements in feature importance estimation. To further support our framework, we performed additional experiments using other explainability methods and showed encouraging results. Please see the common response. In summary, while SHAP has inherent limitations, its practicality and instance-specific utility make it a reasonable choice for our work. As better local explanation methods are developed, our approach will readily integrate these improvements to deliver enhanced results.

Response to W2: We appreciate the reviewer’s suggestion and agree that a comparison to a baseline using global feature importance rankings provides valuable insights. To address this, we included CAE (concrete autoencoders) as a baseline in our evaluation. CAE identifies discrete features that are representative of the training data, effectively providing a global feature ranking. Prior work has also established CAE as a standard baseline for evaluating acquisition policies based on global feature rankings. In our experiments, we compared our instance-specific ranking approach against CAE to demonstrate the added value of tailoring feature acquisition to individual test instances. Our results indicated that acquisition based on instance-specific rankings achieved superior performance compared to CAE. We believe this comparison addresses the concern, as it evaluates the effectiveness of global rankings while highlighting the advantages of our instance-specific approach.

Q3: We compared our method against the GDFS and DIME methods, which are CMI-based. Please see the common response and submission for the results.

Response to W3, Q1, & Q2: As per the reviewer's suggestion, we evaluated the model performance on other tabular medical datasets as well as explored other feature ranking methods. Please see the common response.

The following table presents our model’s performance compared to other methods on the Metabric, CPS, AIDS, and CKD datasets. Performance metrics were calculated by averaging scores across 20 features, except on the CPS dataset, where the scores were averaged over all 8 features. Please see Figure 2 in the revised manuscript for non-averaged performance comparisons across different numbers of features.

We also tested our method performance using various feature ranking approaches on the medical tabular datasets. Specifically we used four local explanation methods: TreeSHAP [4], LIME [5], KernelSHAP [6], and Sampling (IME) [7]. For the tabular datasets, our default SHAP calculation method is TreeSHAP. The following table presents the averaged results as the previous table. For the details, please see the revised manuscript.

References:

[1] Covert et. al.,, “Learning to maximize mutual information for dynamic feature selection,” vol. 202 of Proceedings of Machine Learning Research, pp. 6424–6447, PMLR, 23–29 Jul 2023.

[2] Gadgil et. al., “Estimating conditional mutual information for dynamic feature selection,” in The Twelfth International Conference on Learning Representations, DOI: 10.48550/arXiv.2306.03301, 2024.

[3] Balın et. al., “Concrete autoencoders: Differentiable feature selection and reconstruction,” in vol. 97 of Proceedings of Machine Learning Research, pp. 444–453, PMLR, 09–15 Jun 2019.

[4] Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable ai for trees. Nature machine intelligence, 2(1):56–67, 2020.

[5] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should i trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pp. 1135–1144, New York, NY, USA, 2016.

[6] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.

[7] Erik Štrumbelj and Igor Kononenko. An efficient explanation of individual classifications using game theory. Journal of Machine Learning Research, 11(1):1–18, 2010.