Latent Feature Mining for Predictive Model Enhancement with Large Language Models

We sincerely thank the reviewer for their time and effort. We greatly appreciate Reviewer JQW7's recognition of the novelty of our framework and its practical value. Here are our point-by-point responses to the questions raised by the reviewer:

1. Concern on scalability to adapt to new domains.

Reviewer JQW7 concerned about the requirement for domain expertise in step 1 of our proposed framework, FLAME, will limit the scalability to new domains. From our experience in the two domains (criminal justice and healthcare), the workload for domain experts are not unreasonable because the tasks do not involve technical programming. Instead, experts contribute by writing rationales based on their professional experience, a process that is both intuitive and efficient for those familiar with the domain. As noted in the paper, the inclusion of domain expertise and contextual information leads to huge improvements in both interpretability and performance. We believe this trade-off is justified, as the required human efforts and tailoring to new domains is worthwhile compared to the significant value it adds to the prediction.

2. Comparison of the computational requirements between FLAME and traditional approaches.

We have elaborate the computational requirements of FLAME in Appendix B. Open-source LLMs (such as LLaMA) indeed require more computational resources ( such as GPU) than the traditional approach, however, advancements in black-box pre-trained LLMs have made many models (such as GPT4) accessible through simple API calls. This significantly reduces the barrier to adoption and makes FLAME more practical for a wider range of users and applications.

3. How to adapt FLAME in domains when expert knowledge is ambiguous?

Our framework is designed for scenarios where expert knowledge is available and meaningful. In cases where expert knowledge is ambiguous or unavailable, FLAME may not be the most suitable approach, as its strength lies in leveraging clear and domain-specific insights for latent feature extraction and reasoning.

4. Can FLAME be adapted for online learning scenarios, such as streaming data?

FLAME is not specifically designed for online learning or streaming data scenarios. Its current focus is on leveraging static datasets (batched processing), where domain expertise can be effectively applied. Extending FLAME to handle streaming data would require significant modifications to its structure and workflow, which is beyond the motivation and scope of our current framework, but could be an interesting future research direction.

---

We sincerely thank you for your feedback and comments, and hope this response addresses your question. We are looking forward to further discussion. If our responses have addressed your concerns, we kindly request a reconsideration of the rating score. Thank you again for your valuable input!

We sincerely thank the reviewer for their time and thoughtful feedback. Our innovation is the proposal of transforming latent feature mining task into propositional logic reasoning task, which is provided to be working in two different domains. Here are our response to questions and concerns from reviewer U2yA:

1. What is the necessity of components of the proposed method ?

We conducted the ablation study to demonstrate that each component of our method is important. The results are shown in Appendix D.

2. Why is fine-tuning needed?

The fine-tuning process incorporates feedback loops with domain experts to adjust and correct the LLM’s reasoning pathways, ensuring that the latent features inferred, such as the need for various programs, are aligned with nuanced, causal relationships rather than superficial statistical correlations (see our response to Reviewer cb5e). We also elaborate the importance of finetune in the main paper and in the ablation study (Please see appendix D for more details).

3. Why is reasoning needed?

The reviewer proposes a simple baseline to use the LLM to directly generate an analysis of the question, and then use a text model (such as BERT) to classify both the question and the LLM analysis. Thank the reviewer for this interesting suggestion. While we appreciate this suggestion, we would like to clarify the motivation behind our work and how it differs: Our primary goal is to leverage LLMs to generate human-like reasoning processes, especially in low-resource settings where there is insufficient data to train text models for classification. The design of a step-by-step reasoning process offers the following advantages:

(1). Challenges with Direct Long-Content Analysis: As shown in Appendix D, Figure 10(a), the performance of LLMs in zero-shot and few-shot scenarios is poor and unstable, highlighting the necessity of fine-tuning to guide LLMs toward human-like reasoning. The proposed baseline would require substantial effort from human annotators to produce the necessary training data for fine-tuning, which is infeasible in low-resource settings.

(2). Simplification through Step-by-Step Reasoning: Our reasoning approach decomposes complex questions into multiple steps, making it significantly easier than directly generating a long-content analysis. Transparency and Interpretability: The structured reasoning process provides a clear, step-by-step logical flow, which is crucial for maintaining the transparency and interpretability of the COT process. It also helps retaining the causal relationship as we respond to Reviewer cb5e. This transparency and interpretability are especially desirable in settings where human oversight and understanding of AI decisions are critical.

4. Concern on inconsistent results of LLMs from multiple prompts.

Thank you to the reviewer for raising the concern about inconsistent results of LLMs across multiple prompts. We acknowledge that inconsistency poses a potential risk to the robustness of LLMs. This has been an important consideration in our implementation, and we have taken steps to guide LLMs to respond more robustly and reliably: First, we fine-tune the model to ensure it generates text strictly adhering to the predefined format. This reduces the likelihood of variability in responses caused by different formatting. Second, we carefully design and test prompts to minimize ambiguity and guide the model toward consistent interpretations and outputs.

For further mitigation, we can aggregate results from multiple prompts and apply filtering or voting mechanisms to enhance robustness and reduce outlier responses.

---

We sincerely thank you for your feedback and comments, and hope this response addresses your question. We are looking forward to further discussion. If our responses have addressed your concerns, we kindly request a reconsideration of the rating score. Thank you again for your valuable input!

We sincerely thank the reviewer for their time and thoughtful feedback. We particularly appreciate reviewer HEk5’s comment that our approach, FLAME, is both clever and demonstrates practical importance. Here are our response to questions and concerns from reviewer HEk5:

1. Clarification on the definition of latent feature.

Thank the reviewer HEk5 for pointing out the inconsistency in the visual illustration of the latent features. The latent feature (Z) is correlated with both input feature (X), and prediction target (Y). We will revise the DAG figure to more clearly illustrate the mutual correlation between X and Z during the final revision. Thank the reviewer HEk5 for this valuable suggestion.

2. Why is interpretability important for prediction?

Interpretability is crucial for prediction, especially in high-stakes areas like healthcare and criminal justice, because it ensures we understand the role and impact of each feature in the decision-making process. Moreover, our FLAME approach produces latent features that are directly interpretable, which allow users to see how these features are deducted from step-by-step rationale. This process builds trust, enables accountability, and helps identify potential biases or errors during the rationing process. Without interpretability, using predictions in critical scenarios could lead to unintended consequences, as we wouldn't be able to justify or validate the outcomes.

3. Model scale difference (million-parameters LLMs V.S. smaller neural networks) makes evaluation difficult.

The reviewer may have misunderstood the key comparison, which pertains to the experiment in Section 6.2 rather than Section 6.1. Specifically, in Section 6.2, we extract a single latent feature using FLAME and evaluate its impact on prediction performance by comparing models of comparable size (LR, MLP, and GBT) with and without the latent feature. Your question is more relevant to the experiment discussed in Section 6.1, which we address below.

4. It's not a fair comparison in experiment 1, since the LLMs are extremely larger than neural networks.

We acknowledge the misunderstanding regarding the purpose of Experiment 1. The experiment was designed to (i) validate the ability of LLMs to mimic human thinking processes for extracting latent features and (ii) verify that the extracted latent features are accurate, rather than directly comparing LLMs with traditional ML models. Specifically, we simulated a scenario as if the ground truth label is missing and aimed to evaluate LLMs' performance in following instructions and accurately extracting latent features based on human expertise and established risk scoring rules.

To prevent similar confusion for other readers, we will move the plot and chart of the Experiment 1 results to the appendix, removing the comparison with the ML models in the plot and charts. In the main paper, we will shift the primary focus to the remaining experiments, where LLMs are not directly used as predictive models. This restructuring should provide better clarity and emphasis on the core contributions of our work.

Moreover, we want to mention that advancements in black-box pre-trained LLMs have made many models (such as GPT4) accessible through simple API calls. This significantly reduces the barrier to adoption and makes FLAME more practical for a wider range of users and applications.

5. Can we use LLMs as a predictive model to generate the prediction of outcome directly?

Recent works suggest that LLMs still cannot surpass typical traditional ML models such as XGBoost, SVM, Transformer and RNN [1]. The downstream task (e.g., outcome prediction) usually relies on multiple different variables, the training data, and domain-specific relationship, which makes it harder for general LLMs to do the prediction. FLAME is leveraging the domain expertise to infer the latent features and then enhance the downstream tasks. In fact, we have conducted an ablation experiment to test LLMs’ ability in zero-shot, which is equivalent to using LLMs to directly generate predictions of the outcomes as you suggested. As result shown in Appendix D Figure 10 (a), zero-shot demonstrates poor performance in inference. This suggests that direct outcome prediction is challenging for LLMs .

6. What are the number of trainable parameters used in the MLPs trained here?

The MLP trained has the input size of 30, three hidden layers (50, 60, and 20 neurons), and an output layer of 10 neurons has a total of 6,040 trainable parameters. These are calculated as follows: the connection from the input to the first hidden layer contributes (30 × 50) + 50 = 1,550 parameters, from the first to the second hidden layer contributes (50 × 60) + 60 = 3,060, from the second to the third hidden layer contributes (60 × 20) + 20 = 1,220, and from the third hidden layer to the output layer contributes (20 × 10) + 10 = 210. Summing these gives a total parameter count of 6,040.

---

We sincerely thank you for your feedback and comments, and hope this response addresses your question. We are looking forward to further discussion. If our responses have addressed your concerns, we kindly request a reconsideration of the rating score. Thank you again for your valuable input!

---

[1] Chen, C., Yu, J., Chen, S., Liu, C., Wan, Z., Bitterman, D., ... & Shu, K. (2024). ClinicalBench: Can LLMs Beat Traditional ML Models in Clinical Prediction?. arXiv preprint arXiv:2411.06469.

We sincerely thank the reviewer for their time and thoughtful feedback. We are particularly grateful to reviewer cb5e for acknowledging that "the quality of the work is solidly supported by experiments" and recognizing that our model, FLAME, represents a creative and significant step towards enhancing model interpretability and performance in constrained data settings. Below, we provide detailed responses to the reviewer’s questions and concerns:

---

1. How to ensure that the method learns latent information that is causally predictive of the outcome ?

Our approach leverages expert knowledge to mine latent features. In particular, the human crafted rationales have a justified causal link to the outcomes based on the human expert knowledge, as well as known relationships in the literature. For example, in the criminal justice domain, social-economic status is known to have a causal effect on probation program outcomes[1,2], and this is verified by our community partner (human experts) when developing the basic rationales. The design of our approach places significant emphasis on the integration of human expertise and domain knowledge: We decompose the complex rationales to a series of simple rationales. In each rationale, we have clear instructions to guide the reasoning, and finetune on human crafted rationales to make LLM’s reasoning aligned with human instruction. In other words, we are not letting LLMs randomly mine features that may have just correlations with the outcome – we are using a rigorous approach to allow LLMs mimic the human reasoning process that follows a causal relationship. Since the human crafted rationales have the casual relationship, the fine-tuned LLMs that can accurately mimic the reasoning process should also be able to identify latent features that are causally predictive of the outcome.

To further validate this, we conduct two additional human evaluations on the LLMs generated rationales.

In the first evaluation, we randomly sampled LLMs rationales used in experiment 1 and experiment 2 of the criminal justice case study (20 rationales in total). We invited three experts in criminal justice ( denoted as e1, e2, e3) , and two students who don't have any prior knowledge on this work (denoted as u1, u2) to annotate each rationale. We asked following two questions: (1) "The correctness of the reasoning process: Is the LLM's reasoning consistent with how humans causal reasoning?" (2) "The correctness of final conclusion: Would you agree with the reasoning step as a valid human response to the task?"

The results are shown below:

These results showcase the strong ability of LLMs to replicate human crafted rationales, and make accurate decisions.

The second evaluation is designed to test whether humans can distinguish the reasoning process generated from human or LLMs. We mix the sampled rationales with 20 more human crafted rationales, and we asked the five human annotators with this question: "Do you think these reasoning steps were generated by a Human or an LLM?"

The result shows that, on average, 75% of human crafted rationales are wrongly annotated as LLMs generated, and 60% of LLMs generated rationales are wrongly annotated as human crafted. We believe these results further validate the quality of generated rationales by LLMs.

In conclusion, the two additional human evaluations demonstrate that LLMs is able to accurately mimic the human crafted rationales (that are justified to capture the underlying causal relationship), so the latent features mined by LLMs should also be causally predictive of the outcome.

2. What is the advantage of FLAME over structured methods like topic models or ideal-point models ?

Our approach is fundamentally different from topic models or ideal-point models. These traditional methods often rely on predefined parametric assumptions, such as specific distributions (e.g., Dirichlet for LDA or normality for ideal-point models) and primarily work with structured or semi-structured datasets, such as text corpora for topic models or specific preference/ranking data for ideal-point models. Also their output is not directly interpretable – Latent features in topic models are typically abstract clusters or distributions over words that lack direct interpretability unless post-processed or annotated by humans.

Ideal-point models produce latent traits tied to a specific metric (e.g., ideology or preference score) but do not capture richer, contextualized relationships beyond the given data. In contrast, FLAME is designed to process both structured and unstructured data, making them highly versatile. Our approach can incorporate contextual, domain-specific information expressed in natural language, which allows for reasoning beyond rigid data structures.This makes it more flexible and better suited for handling complex, diverse datasets. Moreover, the latent features inferred by LLMs are generated through reasoning and can capture nuanced relationships (including causal relationship as we addressed in your first question), domain-specific knowledge and other contextual information. The mined latent features are directly interpretable because they can be aligned with natural language rationales, allowing for greater transparency in how they are derived. As a result, FLAME can adapt more effectively to real-world tasks and uncover insights that traditional models might miss, especially for domains where contextual, domain-specific information is critical, such as healthcare or criminal justice. The interpretability is particularly useful for high-stakes decisions.

3. How to validate the inferred latent features when there is a lack of ground truth?

We have human validation before the finetune as described in the paper, and experiment 1 in the criminal justice case study shows that the inferred latent feature is trustworthy. Moreover, we have conducted additional human evaluation for the extracted latent features (see our response to your first question).

4. Can you clarify the extent of domain-specific adaptation required?

We need input from domain expertise to help craft rationales in natural language (Step 1). The rest of the steps can be easily adapted to new domains. From our experience in the criminal justice and healthcare domain, this step definitely requires human experts but the actual workload is not heavy for experts since they don’t need to do the programming-required tasks. As discussed in the last section of our paper, we believe this effort is worth it as it allows incorporation of useful contextual information and expert knowledge.

5. Are there any guidelines or principles required for adapting the FLAME to a new domain?

The general guidelines to adapt FLAME to new areas is to decompose the complex rationales to a group of simple rationales, which could be best understanded and learned by LLMs.

6. Are there safeguards to prevent biases from the LLM from impacting inferred latent features?

We have human validation before the finetune. In addition, we further conducted bias analysis (as shown in the Appendix D) to ensure there’s no bias. We are also conducting human evaluation.

---

We sincerely thank you for your feedback and comments, and hope this response addresses your question. We are looking forward to further discussion. If our responses have addressed your concerns, we kindly request a reconsideration of the rating score. Thank you again for your valuable input!

---

[1] Steinmetz, K. F., & Henderson, H. (2015). Inequality on probation: An examination of differential probation outcomes. In Journal of Ethnicity in Criminal Justice (Vol. 14, Issue 1, pp. 1–20). Informa UK Limited. https://doi.org/10.1080/15377938.2015.1030527

[2] Administrator. (2024, February 7). Socioeconomic status and criminal justice outcomes - iresearchnet. Criminal Justice. https://criminal-justice.iresearchnet.com/criminal-justice-process/racial-and-socioeconomic-disparities/socioeconomic-status-and-criminal-justice-outcomes/