LLMSelect: Knowledge-based Feature Selection with Large Language Models

We show that the prior knowledge of pretrained LLMs can be leveraged to select high-signal features for downstream supervised learning tasks.