Latent Feature Mining with Large Language Models

Thank you for your comments and feedback. Here are our responses to your questions:

• Question 1: Suggestion on adding experiments about social bias and fairness of using LLMs for inference.
• Response: We appreciate your concern about the bias and fairness of using LLMs for inference. Following your suggestion, we conduct additional experiments to validate the LLMs’ inherent bias is not carried into the inference process.
  • Experiment setting: Determine if the reasoning process within generated texts exhibits biases related to social-economic status or risk assessment, specifically racial biases.
  • Experiment results:
    • For social-economic status: We implemented a pretrained keyword extraction model YAKE [1] to search for racial terms in the reasoning steps of the text, with results indicating that such keywords were not found, suggesting no explicit racial bias in this context.
    • For risk level assessment: we closely examined the race distribution in the ground-truth data versus the distribution in the predictions made by the model. The analysis revealed that the race distributions between the ground-truth and the predicted outcomes are similar. This similarity suggests that the model does not introduce additional racial biases in its predictions and reflects the distributions present in the input data accurately.

• Question 2: How much human labor effort and expert labor effort will be needed to have the latent features?

• Response: To obtain latent features, there are two stages that need human expert input.

  • The first stage involves developing foundational guidelines (i.e., the standard solutions to guide LLMs). We propose a crafting rationales strategy to enhance the efficiency of this step, which simplifies the creation of baseline rationales, ensuring that the framework for latent feature extraction is both robust and effectively grounded (please see page 6, under "In the second sub-step...). This stage only requires expert knowledge, minimum labor.
    Moreover, the expert input required is not labor-intensive but rather experience-based. For instance, we successfully applied the same framework to a different domain (healthcare) in a short timeframe, indicating that the process of adapting the trained LLMs to new domains is efficient and not overly demanding in terms of labor.

  • The second stage requiring human involvement is the validation phase, which occurs as step 2 in our process. Although this stage has been designed to be as streamlined as possible, it still requires human oversight to ensure the accuracy and relevance of the features being extracted. However, this process has been optimized to minimize labor, focusing on quality control without demanding extensive time from our experts.

  We note that our framework requires significantly less than what is traditionally required in human annotation approaches for latent feature mining, yet this efficiency does not compromise the quality of the outcomes.

• Question 3: How to infer the first intermediate predicate ?
• Response: From our interpretation of your question, we assume that you are asking how the first intermediate predicate (P_1) be inferred. By leveraging domain expertise, we formulate initial hypotheses or predicates about potential relationships between features. This relates to our above response to your question 2, where the first stage of our framework requires human expert opinion to develop foundational guidelines (i.e., the standard solutions to guide LLMs).

We appreciate your comments and questions, and hope this response addresses your questions. We look forward to any further feedback you may have.

[1] Campos, R., Mangaravite, V., Pasquali, A., Jatowt, A., Jorge, A., Nunes, C. and Jatowt, A. (2020). YAKE! Keyword Extraction from Single Documents using Multiple Local Features. In Information Sciences Journal. Elsevier, Vol 509, pp 257-289.

Thank you for your comments and valuable suggestions, here are our response to your suggestions:

• Suggestion1: Add ablations exploring generalizability with x% features removed.

• Response: Thank you for your valuable suggestion regarding the inclusion of ablation studies. We recognize this as an important aspect of framework evaluation and have thus conducted additional experiments to gauge the robustness and dependence of our model on specific features. We systematically removed different proportions of features and reran experiments on the risk level prediction task.

  • Experiment 1: Removing Features Mentioned in the Provided Guidelines As discussed in the paper, we use domain knowledge and expert input to provide guiding principles for the LLMs' inference process. For this experiment, we removed two features explicitly mentioned in the guiding principles—age and marriage status—and observed the following impacts on model performance:

  Model	ROC_AUC score
  MLP	50%
  GBT	43%
  GPT3.5 Baseline	56%
  GPT3.5 Full-feature	75%

  The results demonstrate that excluding features specified in the guiding principles (e.g., GPT3.5 Baseline) reduces the final accuracy but still outperforms traditional machine learning models such as MLP and Gradient Boosting Trees (GBT).

  • Experiment 2: Removing Features Not Included in the Provided Guidelines In this second experiment, we removed the feature “referral source”, which was not explicitly recommended to be used in the guiding principles:

  Model	ROC_AUC score
  MLP	54%
  GBT	58%
  GPT3.5 Baseline	71%
  GPT3.5 Full-feature	75%

These results indicate that the exclusion of features not included in the guiding principles (e.g., GPT3.5 Baseline) only slightly reduces final accuracy, yet continues to outperform the machine learning baselines (MLP and GBT).

• Suggestion 2: Release code for more comprehensive evaluation
• Response: We planed to release the code after the review. Following your suggestion, we are providing access to the code for the MIMIC experiments as a preview. Following the rebuttal instruction, we have submitted an anonymous link to the code to the AC. This link will become available for preview upon approval by the AC.

Thank you again for your valuable input! We are looking forward to further discussion.

Thank you for your valuable feedbacks and comments, here are our response to your questions:

• Question1: Concern about potential biases from LLMs in latent variable finding.

• Response: Thank you for highlighting this concern. Our framework aims to minimize biases by leveraging domain-specific data and expert input during the fine-tuning processes. We carefully design rules based on domain knowledge and expert input. For example, the client shown in Figure 1 primarily uses marijuana but has a property offense as the admitting offense. Our community partner noted that marijuana usage often correlates with other substance use (though not recorded in the admitting offense) and that “the prevention of heavy marijuana use could potentially reduce property crime in the future” [1]. Therefore, LLMs, following expert guidance, add substance abuse treatment to the potential requirement list.

  As addressed in another comment by Reviewer U9i4, we conducted a thorough sanity check, showing that LLMs trained in our framework do not amplify bias but adhere to domain expert principles (the “standard solutions” provided for training).

  While bias may be inherent in input data, our framework ensures transparency in inference and prediction, aiding result interpretation and bias correction through human-in-the-loop processes, which allows domain experts to adjust and refine the model’s reasoning pathways. Consequently, inferred latent features align with nuanced real-world outcomes rather than broad statistical correlations.

• Question 2: Can you add experiments on other domains/dataset to prove generalizability.

• Response: We have conducted additional experiments in the healthcare domain with the MIMIC-IV dataset to prove the generalizability of our framework. Here are the experiment details and results:

  • Dataset: The MIMIC dataset is a comprehensive dataset containing detailed de-identified patient clinical data and is widely used for various prediction tasks in the machine learning literature.
  • Task Description: The discharge location prediction task uses patient data to forecast the most likely discharge destination, aiding hospital management in preparing for discharges. We leverage our framework to introduce a "social support" feature, which captures the healthcare and community support available to the patient. This involves repeating the four-step process of our framework:
    • Step 1. Create rationale: we leverage domain expertise in hospital inpatient management and patient flow to help us craft rationales to infer social support.
    • Step 2. Enlarge synthetic data for LLM training: similar to our approach for the outcome prediction task using the criminal justice data (task 2 mentioned in the main paper), we use GPT-4 to generate 4500 data points to fine-tune GPT-4o-mini.
    • Step 3 & 4. The remaining two steps are the same as those used for the outcome prediction task (task 2 mentioned in the main paper).
  • Experiment Setting: Due tack the ground-truth label for the latent variable "social support," making our experiments akin to Task 2 in the main paper. After generating the latent features, we train four machine learning classifiers: Logistic Regression (LR), Multilayer Perceptron (MLP), Gradient Boosting Trees (GBT), and Random Forest (RF). Training is conducted with and without latent features for comparison. Each experiment is run five times with different random seeds, and the results are averaged to ensure reliability.
  • Experiment Results: The table below demonstrates the experiment results, showing an average improvement of approximately 8.64% in accuracy and 8.64% in F1 score when latent features are added to the models. This is similar to the percentage increase reported in Table 4A in the main paper.

  Model	Accuracy (std.)	F1_score (std.)
  LR	65.22% (0.01)	65.46% (0.01)
  MLP	63.19% (0.02)	63.19% (0.02)
  GBT	64.84% (0.01)	65.09% (0.01)
  RF	65.11% (0.01)	65.44% (0.01)
  LR w/ LF	71.22% (0.01)	71.26% (0.01)
  MLP w/ LF	74.40% (0.01)	74.50% (0.01)
  GBT w/ LF	75.56% (0.02)	75.38% (0.02)
  RF w/ LF	75.31% (0.01)	75.22% (0.01)

  The results demonstrate another strong evidence of using our framework to improve downstream prediction power with the addition of latent features. Connecting to Lemma 1, our experiment results show that the added features are informative – likely because the human experts (case managers or physicians) are making decisions based on more than the explicitly recorded features (X) in the dataset.

• Question 3: How does this compare with other forms of data augmentation?

• Response: Our framework leverages LLMs to augment observed features in datasets with latent features. This approach differs from traditional data augmentation methods like VAE, as we augment the dimensions of X rather than the sample sizes. A detailed comparison: VAEs aim to represent the original feature space in a lower-dimensional space, whereas our goal is to add mined features to enhance prediction power for downstream tasks. VAEs use probabilistic approaches to describe data distribution with latent variables, but their mappings can be difficult to interpret, while our framework offers better interpretability. Section 2 contains more detail comparison with other methods.

We sincerely thank you for your feedback and comments, and hope this response addresses your question. We are looking forward to further discussion. If our responses have addressed your concerns, we kindly request a reconsideration of the rating score. Thank you again for your valuable input!

[1] Green KM, etc al. Does heavy adolescent marijuana use lead to criminal involvement in adulthood? Drug Alcohol Depend. doi:10.1016/j.drugalcdep.2010.05.018

Thank you for your valuable feedbacks!

• Question 1: How to measure the impact of errors? How to ensure the approach doesn't amplify potential errors?

• Response: As discussed in Section 4, we incorporate human-in-the-loop interventions to identify and remove erroneous synthetic reasoning steps from the training data. Additionally, we utilize automatic evaluation metrics to detect errors within the reasoning process and excluded from the training dataset to maintain model integrity. Furthermore, we conducted an additional error analysis for the Risk Level Prediction (Task 1 in the main paper) to evaluate the impact of errors. We recruit human volunteers examine all 1168 instances used in Step 2 (generating synthetic rationales) of the framework. Our analysis found that logical reasoning leads to correct predictions, and incorrect reasoning does not produce correct results, thereby minimizing error amplification. This analysis supports the effectiveness of our framework in avoiding error amplification.

• Question 2: Clarification on the difference with dimension reduction.

• Response: Our method extrapolates information not explicitly present in the original data, unlike dimension reduction techniques like VAE or PCA, which represent the original feature space in lower dimensions while retaining as much information as possible. Instead, our framework enriches the dataset by adding mined features to improve prediction power for downstream tasks, mimicking human experts' reasoning by considering features holistically and inferring additional socio-economic information not recorded in the data. Figure 1 in our paper illustrates this step-by-step extrapolation process using LLMs.

• Question 3: Suggestion on adding experiments on other domains/dataset to prove generalizability.

• Response: We have conducted additional experiments in the healthcare domain with the MIMIC-IV dataset to prove the generalizability of our framework. Here are the experiment details and results:

  • Dataset: The MIMIC dataset is a comprehensive dataset containing detailed de-identified patient clinical data and is widely used for various prediction tasks in the machine learning literature.
  • Task Description: The discharge location prediction task uses patient data to forecast the most likely discharge destination, aiding hospital management in preparing for discharges. Our latent feature framework creates features that enhance machine learning models for this task. Notably, we introduce a "social support" feature, which captures the healthcare, familial, and community support available to the patient. This involves repeating the four-step process of our framework:
    • Step 1. Create rationale: we leverage domain expertise in hospital inpatient management and patient flow to help us craft rationales to infer social support.
    • Step 2. Enlarge synthetic data for LLM training: similar to our approach for the outcome prediction task using the criminal justice data (task 2 mentioned in the main paper), we prompt GPT-4 to generate 4500 data points to fine-tune GPT-4o-mini.
    • Step 3 & 4. The remaining two steps are the same as those used for the outcome prediction task.
  • Experiment Setting: Due to the lack the ground-truth label for the latent variable "social support," making our experiments akin to Task 2 in the main paper. After generating the latent features, we train four machine learning classifiers: Logistic Regression (LR), Multilayer Perceptron (MLP), Gradient Boosting Trees (GBT), and Random Forest (RF). Training is conducted with and without latent features for comparison. Each experiment is run five times with different random seeds, and the results are averaged to ensure reliability..
  • Experiment Results: The table below demonstrates the experiment results, showing an average improvement of approximately 8.64% in accuracy and 8.64% in F1 score when latent features are added to the models. This is similar to the percentage increase reported in Table 4A in the main paper. | Model | Accuracy (std.) | F1_score (std.) | |:----------:|:-----------------:|:-----------------:| | LR | 65.22% (0.01) | 65.46% (0.01) | | MLP | 63.19% (0.02) | 63.19% (0.02) | | GBT | 64.84% (0.01) | 65.09% (0.01) | | RF | 65.11% (0.01) | 65.44% (0.01) | | LR w/ LF | 71.22% (0.01) | 71.26% (0.01) | | MLP w/ LF | 74.40% (0.01) | 74.50% (0.01) | | GBT w/ LF | 75.56% (0.02) | 75.38% (0.02) | | RF w/ LF | 75.31% (0.01) | 75.22% (0.01) |

  This experiment on a different dataset from a different domain shows the effectiveness and generalizability of our framework. Connecting to Lemma 1, our experiment results show that the added features are informative – likely because the human experts (case managers or physicians) are making decisions based on more than the explicitly recorded features (X) in the dataset.

• Question 4: How to ensure that the method learns latent information that is causally predictive of the outcome ?

• Response: We acknowledge that our current framework does not explicitly perform causal inference. Instead, relationships between features and outcomes are identified through domain knowledge and expert input, leveraging expert understanding to guide the identification of meaningful latent features. We agree on the importance of causally relevant latent features and recognize this as a crucial direction for future research. However, incorporating causal inference into our framework is beyond the scope of this paper and would require a dedicated study to thoroughly examine and address this issue.

We sincerely thank you for your feedback and comments, and hope this response addresses your question. We look forward to further feedbacks. If our responses have addressed your concerns, we kindly request a reconsideration of the rating score. Thank you again for your valuable input!

Thank you for thoughtful comment! Following your suggestion, we plan to consider adding following discussions into the main paper.

• Discussion 1: What are limitations of the framework when high-risk predictions in CJS are very high-stakes?

• Response: - Thank you for this insightful question. Below we list more detailed discussion on the limitations of the framework for high-risk predictions in the criminal justice system (CJS). We will incorporate this discussion into the main paper during our revision process.

  • Data Quality and Dependency: The performance and reliability of the framework depend on the quality of both the training data and the input data used during operation. Inconsistencies, errors, or gaps in data can lead to inaccurate predictions, which is critical when these predictions aim to influence decisions about individuals’ freedoms in the CJS. We are cautious of such errors and this is why we closely involve human experts during the whole research process. While acknowledging that bias sometimes is inherent in the input data, we emphasize that our framework provides the transparency of how the inference/prediction is made, which can significantly help interpret results and correct for such bias by introducing human-in-the-loop. In other words, our framework allows an iterative fine-tuning process, where we can incorporate feedback loops with domain experts to adjust and correct the model’s reasoning pathways. This iterative method ensures that the latent features inferred, such as the need for substance abuse treatment, are aligned with nuanced real-world outcomes rather than broad statistical correlations.

  • Impact on Public Trust: The use of AI in areas impacting fundamental rights can affect public trust in the justice system. If the public perceives these tools as opaque or biased, it could undermine confidence in judicial processes, which is critical for the effective functioning of any democratic legal system. LLMs provide the interpretability that is useful for gaining public trust and allows an iterative process to refine the prediction results.

• Discussion 2: What kind of latent features are your framework particularly good at constructing ?

• Response: Thank you for such an insightful question. Our framework excels at constructing latent features from observable and structured data with clear relationships, such as socio-economic status indicators and risk levels. These features benefit from well-defined guidelines and domain knowledge, allowing our framework to infer them with higher accuracy and reliability by mimicking the human reasoning process.

  The framework can be less effective with subjective latent features or those with subtle relationships, like personal behavioral tendencies. These complexities can introduce bias and lead to overfitting and systematic errors. To mitigate these issues, we use domain expertise and a human-in-the-loop approach to refine latent feature construction. Despite these measures, some biases may persist, and addressing them remains a focus for future research. We will continue to explore improvements.

Thank you again for your valuable input! We are looking forward to further discussion.

Thank you for your comments and feedbacks. Here are our responses to your questions:

Question 1: Suggestion on adding experiments on other domains/dataset to prove generalizability.

Response: We have conducted additional experiments in the healthcare domain with the MIMIC-IV dataset to prove the generalizability of our framework. Here are the experiment details and results:

• Dataset: The MIMIC (Medical Information Mart for Intensive Care) dataset [2] is a comprehensive dataset containing detailed de-identified patient clinical data and is widely used for various prediction tasks in the machine learning literature.

• Task Description: The discharge location prediction task uses patient data to predict the most likely discharge destination, aiding hospital management in preparing for discharges. We apply our latent feature framework to create features that enhance machine learning models for this task. Specifically, we introduce a "social support" feature, capturing healthcare, familial, and community support available to the patient. We repeat the four-step process of our framework:

  • Step 1. Create rationale: we leverage domain expertise in hospital inpatient management and patient flow to help us craft rationales to infer social support.

  • Step 2. Enlarge synthetic data for LLM training: similar to our approach for the outcome prediction task using the criminal justice data (task 2 mentioned in the main paper), we use a self-instructing approach to prompt GPT-4 to generate 4500 data points to fine-tune GPT-4o-mini.

  • Step 3 & 4. The remaining two steps are the same as those used for the outcome prediction task (task 2 mentioned in the main paper).

• Experiment Setting: Note that we do not have the ground-truth label for the latent variable "social support," making our experiments similar to Task 2 in the main paper. After generating the latent features, we train four machine learning classifiers: Logistic Regression (LR), Multilayer Perceptron (MLP), Gradient Boosting Trees (GBT), and Random Forest (RF). We conduct training with and without latent features for comparison. The dataset is split 70/30 for training and testing. Each experiment is run five times using five different random seeds, averaging the results to ensure reliability.

• Experiment Results: The table below demonstrates the experiment results, showing an average improvement of approximately 8.64% in accuracy and 8.64% in F1 score when latent features are added to the models. This is similar to the percentage increase reported in Table 4A in the main paper.

  Model	Accuracy (std.)	F1_score (std.)
  LR	65.22% (0.01)	65.46% (0.01)
  MLP	63.19% (0.02)	63.19% (0.02)
  GBT	64.84% (0.01)	65.09% (0.01)
  RF	65.11% (0.01)	65.44% (0.01)
  LR w/ LF	71.22% (0.01)	71.26% (0.01)
  MLP w/ LF	74.40% (0.01)	74.50% (0.01)
  GBT w/ LF	75.56% (0.02)	75.38% (0.02)
  RF w/ LF	75.31% (0.01)	75.22% (0.01)

  The results demonstrate another strong evidence of using our framework to improve downstream prediction power with the addition of latent features. This experiment on a different dataset from a different domain shows the effectiveness and generalizability of our framework. Connecting to Lemma 1, our experiment results show that the added features are informative – likely because the human experts (case managers or physicians) are making decisions based on more than the explicitly recorded features (X) in the dataset.

Question 2: Suggestion on using text embedding to show the effectiveness of learning

Response: - Thank you for this interesting suggestion. From our interpretation of your comment, we assume that you are asking if we have compared our results with an alternative approach where text embeddings (numerical representations of text) of the descriptions are used as input features directly. We make two clarifications. First, the given input features X from our datasets are either continuous (numeric) or categorical (discrete) numbers. There is no natural text embedding directly available from the data. Thus, we use the Chain of Thoughts (CoT) prompting as a critical component of our framework, to provide the interpretable text input after processing the numeric features X. Hence, there is no direct alternative to compare if you were thinking of replace the input features with text embeddings.

Second, integrating text embeddings after profile writing is an idea worth exploring. We acknowledge that it could be an interesting alternative and warrants further exploration. Meanwhile, we note that the replacement of natural language to text embeddings as input could obscure the step-by-step logical flow that is essential for the transparency and interpretability of the CoT process. Nevertheless, though we have not specifically compared this approach in our current study, we recognize that it could provide valuable insights. Future work could explore integrating text embeddings alongside our fine-tuning strategy to potentially combine the strengths of both methods. Our conjecture is that fine-tuning strategy is necessary for achieving good performance. This aligns with your suggestion that the process of learning an intermediate classification rule during fine-tuning is crucial for the model’s success.

Thank you for your insightful comments and questions. We hope our response has been helpful in addressing your concerns. We are looking forward to further discussion.

[1] Johnson, A. E. W., Pollard, T. J., Shen, L., Li-Wei, H. L., Feng, M., Ghassemi, M., ... & Mark, R. G. (2016). MIMIC-III, a freely accessible critical care database. Scientific Data, 3, 160035.