AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments

We appreciate the reviewer’s recognition of our work’s clinical relevance and for the helpful feedback. We provide a point-by-point response below:

1a. Many crucial details are in the appendix rather than presented in the main text. For instance, it is unclear what specific biases are explored in this work and how they are defined.

Thank you for this observation. We recognize the importance of including critical details in the main text for clarity and accessibility. We had many results in this work and had limited space in the manuscript for extensive details. However, according to your advice, we have incorporated a succinct summary of the key biases explored in this work into the main text. This includes an overview of the two categories of biases (cognitive and implicit), their definitions, and their specific clinical relevance as is now presented in Section 2 Language agent biases, which incorporates relevant details outlined in Appendix A.2, E, E.1-E.2, and L.5. These updates aim to provide a more self-contained narrative in the main body.

1b. Additionally, how agent tools are integrated within the pipeline, what specific data are extracted from the MIMIC-IV dataset, the nature of the NEJM case challenge dataset, and whether it includes QA pairs all need clarification.

We thank you for the suggestion. To clarify, agent tools are integrated into the pipeline as modular components that the doctor agent invokes for specific tasks, such as retrieving test results, searching references, or summarizing notes. For MIMIC-IV cases, the extracted data includes de-identified patient histories, lab results, and diagnostic findings, focusing on structured data relevant to the presented symptoms. The NEJM case challenge dataset contains diagnostic questions derived from real clinical cases and includes QA pairs for validation. To further clarify this, we provide detailed set of replication scripts for cases in our work are provided at the following de-identified link: https://github.com/AgentClinic/AgentClinicAnonymized/tree/main/generate_cases.

1c. Furthermore, the difference between specialist case use reports and general QA tasks should be explained. I recommend providing brief explanations for any non-standard terms in the main text for improved clarity.

To better clarify the distinction, we have included the following explanation in the main text: "Specialist cases use reports that are derived from datasets focusing on specific medical specialties (e.g., internal medicine, psychiatry) and are designed to simulate complex diagnostic scenarios requiring in-depth expertise. In contrast, general QA tasks involve static, single-turn multiple-choice questions such as those found in medical licensing exams." This has been added to Section 3.4: Specialist and Multilingual Cases in the Results section. We believe that this better clarifies the difference between specialist case use reports and general QA tasks and better contextualizes the specialist cases.

2a. Although the paper claims to simulate a real-world clinical environment, several settings and use cases seem impractical. For example, what is the intended purpose of the measurement agent?

The inclusion of the measurement agent is intended to simulate a realistic clinical workflow where a doctor requests specific imaging or laboratory tests rather than having unrestricted access to all retrospectively collected data. This design choice aims to better reflect real-world constraints in clinical environments. The distinction between an agent and a tool lies in their interaction style: tools require highly specific and structured instructions, while the measurement agent offers a flexible interface capable of interpreting diverse linguistic styles used by different LLMs. This flexibility enables smoother integration across models that may vary in their use of acronyms, word order, or phrasing. While the decision to use a measurement agent versus a database agent or a simple tool is ultimately somewhat arbitrary, both approaches can serve the purpose of enabling the doctor agent to collect necessary data. Discussions with our collaborating clinicians further validated the relevance of the measurement agent, as it aligns with emerging systems (e.g., conversational copilots) designed to retrieve specific patient information from electronic health records (EHRs) through interactive queries.

Thanks to the authors for their response. That's a fair point. Though in my opinion the settings in this work may still have gaps from the real practice, I'll not object to accepting this work.

We thank the reviewer for their insightful feedback and for suggesting analyses that have improved our work. Below, we provide a detailed, point-by-point response to each comment:

1 & 2. Can the author provide an intermediate metric, such as the amount of useful information obtained by the doctor agents in both AgentClinic-MedQA and MedQA, to further determine the causes of this phenomenon? The article does not conduct an in-depth analysis of the reasons for the model's performance differences on AgentClinic-MedQA and MedQA.

We thank you for the suggestion and believe this analysis has improved the quality of our work. At your suggestion, we ran a sample of MedQA and AgentClinic-MedQA cases side-by-side to find out the average information coverage by the doctor agent. To do this, we collect all of the relevant information from the MedQA case (e.g. the patient has difficulty walking, the patient has back pain, etc). We then ran the AgentClinic-MedQA simulation with GPT-4 on these cases and manually reviewed the dialogue in order to mark each information item as covered if the doctor agent was able to extract this information from the patient agent from dialogue or through the measurement agent. We find that the average amount of information in the original MedQA question that is revealed (denoted coverage) is 67% We find that in cases where the doctor agent misdiagnoses the patient, the average coverage is 63% and in cases where the doctor agent provides a correct diagnosis, the coverage is 72%. We believe that it is for this reason that there is a discrepancy between MedQA and AgentClinic-MedQA performance–in MedQA all of the information is provided up front, whereas in AgentClinic-MedQA the LLM must extract the information by interacting with the environment under uncertainty, ultimately making a decision with less information. The details of this experiment are in Appendix F.3.

3. The article's simulation of interactions between multiple agents is relatively simple, with the main interactions in the experiment limited to between the doctor agent and the patient agent.

We thank you for this observation. We would like to clarify that the primary goal of AgentClinic is not to create simulated environments identical to real-world settings. Instead, our objective is to demonstrate the impact of transitioning from static medical QA benchmarks to a dynamic, interactive framework. By "bringing the agents to life," where the medical LLM must actively collect information, order lab tests, and perform sequential decision-making, we highlight the significant discrepancies in performance compared to static evaluations. This shift provides a more nuanced assessment of an LLM's capabilities, demonstrating the complexities of sequential clinical reasoning over static question-answering scenarios.

4. Why is a measurement agent needed, and how does its role differ from that of a tool? I hope the author can provide a deeper explanation.

The reason why there is a measurement agent is that the doctor agent is not allowed to have access to all retrospectively collected data to properly simulate that the doctor requests a given imaging or lab test. The difference between an agent and a tool would be that the tool would need a highly specific instruction, whereas by creating a measurement agent, we can have a flexible interface where different LLMs which may have different styles of requesting measurements (different acronyms and word ordering) and it will be interpreted with less difficulty. However, whether it is a measurement or database (DB) agent or simply a tool to be called, is in the end arbitrary - both can be used to allow the doctor agent to collect measurement results. It is possible that we can add the ablation, seeing what happens if all the data handling is operated as a tool - although we do not believe it will be of high relevance to the main takeaways of the paper. We discussed with clinicians who found the measurement / DB agent relevant in that there are agent systems in development (say copilots) that can be chatted to to retrieve specific patient information in the EHR.

The authors' response effectively addressed my concerns, and I have accordingly raised the score.

We thank the reviewer for insightful feedback and for noting the paper is well-written, the experiments are solid, and that the discussion is impressive and comprehensive. The reviewer asks us great questions which we provide detailed answers below.

1. “Uncertainty issues”

We appreciate your concern regarding uncertainty, particularly the possibility that models might be overfitted to datasets like MedQA, which could potentially diminish the significance of high AgentClinic-MedQA accuracy. This is a valid and thoughtful observation. However, we argue that what might be perceived as a potential limitation is actually one of our most significant findings. While a model's training on MedQA could raise questions about the significance of its high accuracy, our results show that the performance gap between AgentClinic and MedQA is independent of any potential overfitting. To substantiate this, we present results from multiple models—such as Mixtral-8x7B, Llama 2 70B, and Llama 3 70B—which are widely recognized as not being overfit on MedQA.

These findings suggest that regardless of a model's prior knowledge of MedQA answers, our main conclusions remain robust and unaffected by overfitting concerns. In other words, even if a model "knows" the answer to a MedQA question, the dynamic nature of AgentClinic—requiring sequential decision-making, such as interacting with the patient to gather information and engaging with measurement agents to obtain medical results—makes mere knowledge of the answer far less valuable than the model's ability to reason and act effectively within the environment.

2. “Include a statistics table for utilized/built datasets, covering sample size, dataset modalities included, task types/descriptions”

Thank you for this valuable suggestion. We have incorporated the requested statistics and added detailed tables to the manuscript. Table 1 provides a breakdown of dataset modalities along with accuracy metrics by modality, while Table 2 offers comprehensive dataset descriptions. To further improve transparency, we have included the exact generation files we used at the following de-identified link: https://github.com/AgentClinic/AgentClinicAnonymized/.

Table #1

Table #2

We thank the reviewer for acknowledging the novelty of our work and for noting its relevance to medical AI and its real-world applicability. We have made improvements based on your insightful feedback which is addressed point-by-point below.

1. What is the rationale behind an N of 20 for the interaction time when assessing the diagnostic accuracy of AgentClinic-MedQA? I’m surprised at the meaningful drop in diagnostic accuracy from 52% to 25%.

One of the variables that can be changed during AgentClinic evaluations is the amount of interaction steps (N) that the doctor is allotted. In initial experiments, we discovered that the doctor agent’s diagnostic performance is surprisingly sensitive to the number of interaction steps the agent has access to. When allowed, the doctor agent will continue asking questions without any foreseeable stopping point. We thus determined it was necessary to find a reasonable value for N, which is the way we arrived at our rationale for choosing N=20 for the interaction time in our experiments. In order to determine a value to select for N, we studied varying the number of interaction steps through our experiments shown in Appendix F.1 “How does limited time affect diagnostic accuracy.” We note from Section L.1 “Doctor Agent Instructions” that the doctor is reminded each step how many interactions they have left. When varied (e.g. N=10, N=20, or N=30) the doctor agent adjusts to the new value and is not optimized for a particular N in any way, but rather can flexibly adapt to any value N that is given.

N < 20: We find that on AgentClinic-MedQA accuracy with GPT-4 decreases from 52% when N=20 to 25% when N=10 and 38% when N=15, showing a near-linear relationship between interactions allotted and diagnostic accuracy (~13 accuracy points per 5 interactions).

N > 20: When N is set to a larger value, N=25 and N=30, the accuracy actually decreases slightly from 52% when N=20 to 48% when N=25 and 43% when N=30. The reason that increasing N to N=30 decreases accuracy is likely due to LLMs' tendency to focus on the first and last parts of large contexts, neglecting the middle [1]. Future work on AgentClinic could incorporate a short-term memory module, as seen in previous agent work, to better manage context and potentially improve accuracy as N increases.

[1] Liu, Nelson F., et al. "Lost in the middle: How language models use long contexts." Transactions of the Association for Computational Linguistics (2024).

2. Although Claude 3.5 is reported to have the highest accuracy on both AgentClinic-MIMIC-IV and AgentClinic-MedQA, its performance varies by as much as 13 percentage points.

The performance of Claude-3.5-Sonnet on AgentClinic-MedQA is 62.1% and on AgentClinic-MIMIC-IV is 42.9% showing a 19.2% discrepancy. We also note that the average score across all LLMs on AgentClinic-MedQA is 35.1% and AgentClinic-MIMIC-IV is 25.7% (9.4 points lower). We also note the average relative reduction in performance [i.e. (acc_1 - acc_2)/acc_1] from MedQA to MIMIC-IV is 32.6%, from which Claude-3.5-Sonnet is 30.9% different, which means that it is actually showing a relative performance decrease aligned with the other models.

The reason that there is an accuracy difference between MedQA and MIMIC-IV cases in general is due to the inherent difficulty of MIMIC-IV cases versus the MedQA cases. The factors contributing to this is that in MedQA, cases are based on the USMLE, which are synthetically designed by medical board examiners for medical students to be simpler and more common cases, whereas MIMIC-IV is from real medical cases using real data which is a lot more challenging. This is also reflected in existing MIMIC-IV-based QA benchmarks, where USMLE-based/MedQA questions are shown to be easier for LLMs to solve than MIMIC-IV questions [1, 2].

[1] Bae, Seongsu, et al. "Ehrxqa: A multi-modal question answering dataset for electronic health records with chest x-ray images." Advances in Neural Information Processing Systems 36 (2024).

[2] Nori, Harsha, et al. "Can generalist foundation models outcompete special-purpose tuning? case study in medicine." arXiv preprint arXiv:2311.16452 (2023).

3. The paper’s "Bias and Patient Agent Perception" section could be expanded to explore patient trust in the healthcare system

We appreciate the suggestion to expand the "Bias and Patient Agent Perception" section by exploring patient trust in the healthcare system. While this is an intriguing area of study, the primary focus of our work is on advancing the doctor agent. This is because the ultimate goal is to deploy the doctor agent in hospital settings, where it can directly contribute to advancing medical practice and improving patient outcomes.That said, if preferred, we could include follow-up experiments with the patient agent to examine their trust in the healthcare system, in line with similar experiments presented in this section.

We thank the reviewer for acknowledging the paper is well-written, our findings on biases are of great importance, and that the key findings are well illustrated. We note that this review was submitted after the review period (by 26 days) and after the revision period (3 days), but will do our best to answer the points raised over the holiday weekend.

1. The study does not consider the impact of longitudinal data on the performance of LLMs, which is an important aspect of clinical decision-making.

We appreciate the suggestion to include longitudinal data. However, the primary focus of this study is on transitioning from static, complete information case scenarios to dynamic, incomplete information scenarios that better simulate real-world clinical interactions. This focus aims to address a different gap in the literature.

2 and 3. While the paper introduces several novel evaluation metrics, such as patient compliance and consultation ratings, these metrics may be subjective and difficult to measure accurately. The evaluation metrics used in the paper, such as patient compliance and consultation ratings, are subjective. How do you plan to validate and improve the reliability of these metrics?

We appreciate the reviewer’s concern regarding the subjectivity of metrics like patient compliance and consultation ratings. While compliance is self-assessed by the patient agent, it has the potential to be objectified in future work by analyzing correlations between self-reported compliance and actual adherence to diagnoses and treatment plans in real-world settings. Subjective metrics, such as consultation ratings, are equally important because they capture patient experience, a critical yet underrepresented aspect of clinical care. Historically, clinical evaluations have prioritized quantifiable outcomes like survival rates, often overlooking the patient’s perspective. By integrating these metrics, we aim to promote empathetic, patient-centered AI systems.

4 and 5. One of my main concerns is that the core framework of this article seems to have a large overlap with AI Hospital, but this article does not mention AI Hospital at all. From the perspective of AI Hospital, some of the core contributions claimed by this paper will be greatly weakened, and I wonder if the authors can explain why AI Hospital is not cited. Alternatively, can you explain the similarities and differences between this article's framework and the AI ​​Hospital framework?

We thank the reviewer for their questions regarding AI Hospital and appreciate the opportunity to clarify the distinctions between the two studies. While both works explore multi-agent systems in medical contexts, they address different research questions and have distinct contributions:

1. AgentClinic is specifically designed to investigate how LLMs transition from static QA benchmarks like MedQA to dynamic, sequential decision-making scenarios under incomplete information. By requiring models to engage with patients, gather relevant details, and request appropriate tests, AgentClinic reveals novel insights, such as the significant drop in performance when static cases are tackled interactively. AI Hospital does not focus on this problem or enable such comparisons.

2. AgentClinic introduces multimodal capabilities, integrating text, images, and lab results, to better reflect real-world clinical workflows. In contrast, AI Hospital primarily focuses on textual data from Chinese medical records, offering a narrower scope of evaluation.

3. Our framework systematically incorporates and studies implicit and explicit biases in individual agents, such as patient biases or diagnostic tendencies, to evaluate their effects on outcomes. This aspect is not explored in AI Hospital.

4. AgentClinic equips doctor agents with advanced tools, including adaptive retrieval systems, persistent note-taking, and reasoning chains, to emulate realistic clinical reasoning processes. AI Hospital does not feature comparable tool support or techniques for enhancing agent performance.

5. AgentClinic evaluates multilingual capabilities across seven languages, including English, Chinese, Spanish, and Persian, enabling global applicability. AI Hospital is focused exclusively on Chinese medical records and does not assess multilingual performance.

6. AgentClinic includes detailed human evaluations by clinical experts, encompassing both offline assessments of agent interactions and live role-playing scenarios where experts act as the doctor. In contrast, AI Hospital does not report quantifiable results from human evaluations. The supplementary materials of AI Hospital include a review form used by medical students (likely Supplementary Figure 8), but no actual results or performance metrics from these reviews are provided. This lack of detailed human validation limits the insights that can be drawn about the real-world applicability of AI Hospital’s framework compared to AgentClinic.

1