Enhancing Small Medical Learners with Privacy-preserving Contextual Prompting

Thank you for your thorough review and valuable feedback. Please find our responses to your concerns below.

1. Concerns about the novelty.

We appreciate the opportunity to further elucidate the innovative aspects of our work. Our paper introduces a novel approach that simplifies the integration of medical knowledge into SLMs by leveraging LLMs as a knowledge base. Unlike prior research that often relies on complex training algorithms and the intricate process of constructing medical knowledge graphs[1-3], our method streamlines this integration, which is substantiated by the performance improvements showcased in Table 1 in the paper.

Moreover, our paper begins a crucial discussion on privacy issues when using LLMs in healthcare. This is a significant issue that hasn't been given enough focus in past studies [4-7]. By bringing this up, we aim to encourage more careful and ethical use of LLMs where patient privacy is a concern.

2. Concerns on using NER.

In our paper, we utilize NER to extract keywords automatically, and we mention in the introduction that keywords can also be extracted by other methods, e.g., a manually created dictionary based on domain expertise. In practice, we can also use rules to post-process the extracted keywords or keep clinicians in the loop to add or delete medical terms, thereby improving the quality of the medical keywords.

We adopt NER here because it is the most convenient way to extract relatively high-quality medical information from raw data. This supports our main idea that by utilizing LLMs as a database, we can generate medical knowledge for auxiliary SLMs in medical decision-making while reducing privacy concerns. We will further clarify the reason to utilize NER in our revision to improve clarity.

3. Suggestion about missed footnote.

We are grateful for your attention to detail. We will add this footnote in the revision.

[1] Yasunaga et al. (2022a) Qa-gnn: Reasoning with language models and knowledge graphs for question answering
[2] Zhang et al. (2022) Greaselm: Graph reasoning enhanced language models for question answering
[3] Yasunaga et al. (2022b) Deep bidirectional language-knowledge graph pretraining
[4] Ho et al., (2022) Large language models are reasoning teachers
[5] Shridhar et al., (2022) Distilling multi-step reasoning capabilities of large language models into smaller models via semantic decompositions
[6] Wang et al., (2022) Pinto: Faithful language reasoning using prompt-generated rationales
[7] Li et al., (2022) Explanations from large language models make small reasoners better

We appreciate the insightful feedback and comments from the reviewer. Their positive observations about the novelty and thoroughness of our experiments are very encouraging. We have addressed your concerns in our response.

1. Concerns on privacy preserving in practical usage. The data we utilized in experiments have already undergone post-processing; however, even well-processed data cannot be directly shared with third parties in a real-hospital setting. Here, we adopt NER methods directly, solely for automation, to show that LLM can be utilized as a medical database to query knowledge under privacy-restricted scenarios. Practically, we can leverage de-identification models and rules to remove personal information and then extract medical keywords to query third-party LLMs for auxiliary knowledge generation. In this paper, we take an initial step to discuss the significant privacy-preserving situations in the medical domain and demonstrate the promising results of utilizing LLM to improve SLM while mitigating privacy concerns.

2.Question about what SLM learns for decision making.

We feed preliminary decisions (PD) as context into SLM with backbone BioLinkBert-Base on three datasets. Three separate runs for each setting are conducted and the average results along with the standard deviation are reported. The results are shown in the Table below.

FTC, which integrates extensive medical knowledge into the decision-making, shows a consistent and significant improvement over the SLM that only uses PD for context. These findings underscore the valuable contribution of leveraging comprehensive medical knowledge, provided by LLM, in enhancing the medical decision-making capabilities.

3. Suggestion about related work in LLM application in the clinical domain.

Thanks for the suggestion in the related work. We will add the suggested work into the related work section in the revision.

4. Question abut address long medical context generated by LLM.

We utilize the Fusion-in-Decoder [1] approach in our general domain experiments. This strategy is also effective for encoding long contexts. It works by dividing the input into smaller passages, encoding each one separately, and then combining the encoded representations for decision-making.

[1] Izacard et al. ( 2020) Leveraging passage retrieval with generative models for open domain question answering

Thanks for replying. Based on the response, I would like to keep my original score.

We would like to thank the reviewer for their comments and feedback. We reply their concerns in the below.

1. Concern on privacy definition. Privacy is a multifaceted concept with varying definitions that depend on the context and use cases. Our definition of privacy differs from that of differential privacy, which offers a theoretical guarantee but may not always be practically applicable. In our setting, privacy issues do not arise during the training process but instead when calling third-party APIs with tokens to query medical knowledge. Specifically, we define the privacy budget as the ratio of the number of words from the original question that are provided to the third party for prompting the language model, as detailed in Section 5. Intuitively, the less information provided to the third party, the better the privacy condition we achieve; if no words are sent, there is no privacy risk.

Furthermore, Figure 1 is for demonstration purposes only and is not used to define privacy in terms of personally identifiable information (PII). In practice, we can employ additional de-identification methods to remove PII from medical terms in the preprocessing or postprocessing stages. In this paper, our primary contribution is to demonstrate the effectiveness of using LLM as a database for generating medical knowledge to enhance SLMs' performance. We also initiate the discussion of LLM utilization under privacy-restricted conditions, which previous works [1-4] have overlooked. We will revise our paper to include a clear definition of privacy in our context, thereby improving its clarity.

2. Concerns about motivation to utilize ChatGPT instead of open-source LLMs.

We evaluate medical domain specific LLMs based on LLaMA [5] : AlpaCare [6], PMC_LLAMA [7], ChatDoctor [8], Medalpaca [9] and Baize-healthcare [10] on the three QA datasets following [11]. The results are shown in the table below.

ChatGPT significantly outperforms domain-specific LLMs across three QA datasets under privacy-restricted settings, demonstrating that the medical knowledge within open-source LLMs still lags behind the GPT-series models. Additionally, our FTC, which further leverages the knowledge provided by ChatGPT, achieves better performance, indicating the effectiveness of our methods.

[1] Ho et al., (2022) Large language models are reasoning teachers
[2] Shridhar et al., (2022) Distilling multi-step reasoning capabilities of large language models into smaller models via semantic decompositions
[3] Wang et al., (2022) Pinto: Faithful language reasoning using prompt-generated rationales
[4] Li et al., (2022) Explanations from large language models make small reasoners better
[5] Touvron et al. (2023) LLaMA: Open and Efficient Foundation Language Models
[6] Zhang et al.(2023) AlpaCare:Instruction-tuned Large Language Models for Medical Application
[7] Wu et al. (2023) PMC-LLaMA: Towards Building Open-source Language Models for Medicine
[8] Li et al (2023) ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge
[9] Han et al (2023) MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data
[10] Xu et al (2023) Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
[11] Gao et Al. (2021) A framework for few-shot language model evaluation

We would like to thank the reviewer for their insightful comments and feedback. Their positive remarks regarding motivation, comprehensive experiments, and our efforts in addressing privacy concerns are truly encouraging. Please see our response to your concerns.

1. Concerns about contribution in the clinical domain.

The reviewer asserts that there seems to be room for improvement in our algorithm, but it is not clear to us from which perspective we should improve. In addition, we wish to highlight that the contributions of a paper can be evaluated from various perspectives, not limited to algorithms alone.

The primary contribution of our paper is the proposal of a simple yet effective method to utilize LLM as a medical knowledge database, which enhances the decision-making abilities of downstream SLMs. Unlike previous works in medical QA that utilize more complex algorithms and extensive engineering to integrate medical knowledge graphs or databases [1-3], our approach simplifies this by foregoing the complex engineering involved in building knowledge graphs and training algorithms such as GNNs. It has been empirically validated to outperform these methods, as demonstrated in Table 1 in the paper.

Furthermore, our work pioneers the discussion on privacy concerns within the utilization of LLMs in the medical field, a critical issue that has been overlooked in previous research. This innovation opens up new pathways for the deployment of LLMs in sensitive domains.

We also note that the reviewer has acknowledged our contributions to the medical domain with comprehensive experiments and our efforts to mitigate privacy concerns in the review of the strengths section. In this case, we kindly request the reviewer reevaluates our paper based on the clear contribution.

2. Concerns about experiments in medical examination QA datasets.

In this paper, our primary focus is on multiple-choice medical QA instead of question generation to evaluate the effectiveness of our approach for several reasons. Firstly, these datasets encompass a wide range of medical topics, including anesthesia, radiology, and pharmacology [4], and different medical applications such as diagnoses and examinations [5]. We can comprehensively evaluate the performance of our methods across various crucial medical domains. Secondly, evaluating question generation tasks presents significant challenges due to limited datasets and the high cost of performing real-world evaluations with clinicians. Hence, we chose medical QA datasets to strike a balance between evaluating medical ability and feasibility, following previous works [1-3,6-9]. We will add the reason for choosing medical examinations in our revision for further clarification.

3. Concerns about medical knowledge generation methods and denoise capacity of SML.

Although LLM cannot always generate perfect medical knowledge and may produce noise, alternative external knowledge auxiliary methods are also imperfect. For example, retrieving information from knowledge graphs and bases not only requires a separate retriever, but also yields noisy and incomplete results since these sources are human-constructed. Consequently, retrieval processes might acquire irrelevant knowledge, potentially hurting model performance.

Additionally, We show that SLM possesses robust denoising capabilities in Section 5. Even when received with partially correct noisy medical knowledge, our SLM can make a correct decision by extracting useful information. Furthermore, other potential methods, such as integrating LLM with the internet, could improve the generation quality of LLM for more reliable medical knowledge.

[1] Yasunaga et al. (2022a) Qa-gnn: Reasoning with language models and knowledge graphs for question answering
[2] Zhang et al. (2022) Greaselm: Graph reasoning enhanced language models for question answering
[3] Yasunaga et al. (2022b) Deep bidirectional language-knowledge graph pretraining
[4] Pal et al. (2022) MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering
[5] Jin et al. (2020) What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams
[6] Mao et al. (2022) Hierarchical representation-based dynamic reasoning network for biomedical question answering
[7] Liu et al. (2020) Interpretable multi-step reasoning with knowledge extraction on complex healthcare question answering
[8] Dai et al., (2022) Mixture of experts for biomedical question answering
[9] Liévin et al., (2022) Variational open-domain question answering