CliBench: A Multifaceted and Multigranular Evaluation of Large Language Models for Clinical Decision Making

The use of publicly available MIMIC-IV data raises concerns about potential data leakage during LLM pre-training. While the authors discuss this, they should consider strengthening their argument by conducting further analysis or experiments to quantify the impact of potential leakage. In addition, solely on MIMIC-IV from a single medical center limits the generalizability of findings.

The absence of few-shot/ICL experiments restricts the understanding of LLM adaptability and learning potential in these tasks. Zero-shot setting on either clinical reasoning or outputting desired ontologies/structured outputs is not a realistic task.

While the authors justify using billing codes as a proxy for "collective knowledge," including a physician performance baseline would provide a more direct comparison and better contextualize LLM performance.

Ground truth relies solely on EHR records (line 198). The authors do not mention obtaining physician agreements or validation of the ground truth labels used in CLIBENCH.

• The main issue with this work is that the task formulation does not closely align with real-world clinical decision-making. Although the paper emphasizes that its tasks are grounded in clinical diagnosis (e.g., L61, Section 3.1) due to the use of the realistic MIMIC-IV dataset, there are significant aspects to consider, such as (1) the clinical decision-making process and (2) the benchmarking process itself.
• Regarding the connection of this benchmark to clinical decision-making, predicting ICD-10-CM diagnosis and ICD-10-PCS procedure codes is primarily a medical billing task, not a direct diagnosis or treatment task, which the authors have already recognized (see Section F. Limitations).
• Even if the benchmark assumes the ability to perform a billing task or a reasonable clinical workflow, its effectiveness as a benchmark is quite limited. Specifically, it is unclear whether sufficient input data has been properly formatted or extracted from MIMIC-IV for each task, raising doubts about whether a perfect score is achievable. Additionally, there is limited insight into how well LLMs are performing in this context. At a minimum, the paper should provide a baseline score, such as an expert clinician score or a majority-class prediction, to contextualize the LLMs' performance.

My main concern lies in the evaluation approach of this work. Unlike other mainstream medical benchmarks, the authors use clinical code prediction as the downstream task and employ precision, recall, and F1-score as performance metrics for the four tasks. While this approach is indeed closer to the real deployment environment of medical AI, it also introduces the following issues:

1. Test Prompt: I noticed that the authors prompt the language model with “provide as many diagnoses as you can until you are not confident about your diagnosis decision.” Has the impact of different prompt styles on performance been tested? The current prompt format seems to encourage outputting as many predicted codes as possible, which may be a key reason why, in Figure 3c, the F1-score increases as the number of ground-truth diagnoses grows. I believe it is necessary to test prompts with different phrasing (e.g., “please provide an appropriately sized set of diagnoses”) to further improve the stability of the evaluation results.

2. Answer Extraction: I noticed that the authors allow the language model to output either clinical codes or predictions in text form. For text-based predictions, they use a BERT model pretrained on 1B sentence pairs to calculate sentence similarity and select the closest code as the predicted result. I have the following questions:

   • When the model provides both code and text-based results, is priority given to extracting the code or to parsing the text? In such cases, is the accuracy higher when parsing the code directly or when interpreting the text result?
   • What is the accuracy of this text parsing method based on sentence embeddings, and has any related analysis been conducted? Are there alternative methods that could further improve matching accuracy?

3. Human Physician Performance: Although the authors provide some reasons in the appendix for not including human physician performance, I still believe it is essential to add human performance data (even if on a small scale) for this dataset. First, the tasks in this dataset are inherently challenging; for example, ICD-10-CM contains over 70,000 codes, and ICD-10-PCS has over 87,000 codes. Even for medical experts, accurately completing the coding without consulting the specific ICD-10 code set is very difficult. Including evaluation results from human experts under close-book conditions would help us better understand the benchmark’s upper bound, which is highly valuable for this evaluation set.

• Reliance on Zero-Shot Evaluations: Although the benchmark provides valuable insights, the study primarily relies on zero-shot evaluations, despite the sensitivity of LLM performance to prompt configurations. Exploring few-shot settings or incorporating multi-prompt variations could enhance robustness and offer a more comprehensive assessment of the benchmark's applicability.

• "Domain-specialized models do not work": This statement could benefit from a more nuanced analysis, with additional experiments to substantiate the findings. A deeper exploration into why domain-specific adaptations underperform in this context would provide valuable insights and help clarify the challenges involved.

• Lower Performance on Specific Tasks: Performance on several tasks, particularly procedures and lab test orders, is noticeably lower. Further investigation would clarify whether this discrepancy reflects inherent model limitations or potential areas to strengthen the benchmark's design.

• Ambiguity in Prompt Construction: While prompt construction is outlined, the study lacks an in-depth evaluation of prompt effectiveness across different clinical tasks. Analyzing how specific prompt structures influence model performance could yield valuable insights and improve task-specific optimization.

1. For procedures, lab tests, and prescriptions, the inputs are patient profiles and medical records. Is this information sufficient for human clinical experts to make real-world decisions? If clinicians need to rely on additional data, such as ICU monitoring, it might be unrealistic to expect LLMs to make accurate decisions solely based on such information. Section 6.3 shows that missing important patient information can significantly impact LLM performance. If this subset of information aligns with what clinicians use, it would be helpful to justify the design and the validity of the benchmark in the paper.

2. Section 3.3 mentions that admissions without records for the target tasks are filtered out. For AI systems deployed for decision support, I think it is also important for them to recognize cases where no procedure, lab test, or prescription is required (i.e., true negatives). Is there a plan to include such "negative" admissions in the evaluation set, or a justification that they should not be included?

3. In section 3.6, micro-level precision, recall, and F1 are used as metrics. Figure 1(b) shows significant variance in the number of cases across disease types. Including macro-level metrics or disease type-specific metrics (similar to the patient breakdown analysis in Section 6.2) would provide more insights into the strengths and weaknesses of LLMs across different disease types, e.g. some LLMs might perform well on certain disease types but poorly on others, while other LLMs might demonstrate different behavior.

Note: I am not a clinician and may not fully understand certain aspects of clinical decision-making. I am open to changing my score based on clarifications regarding these concerns.