Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models

We deeply thank the reviewer for these detailed suggestions on improving the presentation of our work.

---

W1: The paper mentioned privacy as an issue but I think that the method will not address any privacy issues that are accruing in the original LLM.

A: Thanks for the insightful feedback. We have answered this question in the general response for all reviewers. Please refer to the Patient privacy concerns section for details.

---

W2: Hard to understand what is the base model used for MLM training, is it PubMedBERT? If so, PubMedBERT outperforms all of the other attempts.

A: Thanks for your question. We would like to kindly point out that for the base model, “MLM training” is usually conducted during the pretraining stage, while in our work, we focus on the fine-tuning stage where we adopt the cross-entropy loss towards the target task with the generated synthetic data (see Section 4.3 for details). This is a common setting that is widely used for evaluating the quality of synthetic text data [1,2,3], where better performance indicates better synthetic data utility.

We note that for the base model, we use the same PubMedBERT for ClinGen and baselines to ensure fair comparison. For the PubMedBERT model, we fine-tune it for different target tasks using the synthetic text data without additional MLM training. As evidenced in Table 1, our method, leveraging the same base model (PubMedBERT-base and large), consistently outperforms the baselines. This better performance is attributed primarily to the high quality of the generated data.

[1] Ye et al. "Zerogen: Efficient zero-shot learning via dataset generation." EMNLP 2022.

[2] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023.

[3] Ye et al. "ProGen: Progressive zero-shot dataset generation via in-context feedback." EMNLP 2022

---

W3: Baselines descriptions could be more elaborated

A: Thank you for the suggestion. We provide the baseline details in Appendix E in our original manuscript, and we have expanded these descriptions to provide more comprehensive information in the revision.

---

Q1: The most important question is what is the additional pre trained classifier It's unclear from the paper

A: As mentioned in W2, we utilize the same pre-trained PubMedBERT as a classifier for our method and baselines on downstream tasks to prevent unfair comparison. Please see “Sec 5.1 Experiment Setup” for reference. The word “additional” is to differentiate the classifier with the previous LLM generator. We do not conduct any additional pre-training. We have deleted the word to avoid confusions.

---

Thanks again for your review! Please let us know if you have any further questions.

W1: The proposed prompting strategy is intuitive but lacking in novelty. Leveraging both KGs and LLMs is expected to help.

A: Thanks for the insightful feedback. We have answered this question in the general response for all reviewers. Please refer to the Lack of novelty section for details.

---

W1 (continued): To best of my knowledge, the generated data is still in-domain data, generating the style the same as the training data. So, it is not surprised for me that using more data for training can improve model performance.

A: We would like to clarify that we do not use “more data” compared to the baselines. In our experiments, all the baseline models are evaluated based on their respective generated synthetic datasets, and we maintain the same amount of synthetic training data across all methods, as mentioned in Appendix C. This ensures a fair comparison among all the methods evaluated in our study. In the revised version, we explicitly mention this in Section 5.1 to avoid confusion.

Moreover, it's essential to highlight that merely increasing the volume of training data does not guarantee better results. As shown in Table 1, some standard data augmentation methods cannot even outperform the supervised setting with few-shot examples only. Besides, for LLM-based data generation approaches, Figure 11(a) shows that the best baseline (DemoGen) using 100% of the training data actually shows lower performance compared to using only 50% of the training data. These observations illustrate the importance of the quality of training data over its quantity. Our focus, therefore, has been on developing a methodology to create high-quality synthetic training data. The results from our experiments in Table 1, Figure 5 and Figure 11 confirm that we consistently outperform baselines with the same amount of generated data under different budgets.

---

W1 (continued): It is more make sense to improve small model's over medical capacity, instead of overfit on a specific task with more training data.

A: Thanks for the suggestion. We would like to emphasize that our primary goal is indeed to improve a model’s medical capacity during fine-tuning. Our approach focuses on enriching the training dataset with diverse medical knowledge, which can empower models of varying sizes with improved medical capacity. The generated synthetic data can also be easily and directly applied to improve various existing models without additional design.

Besides, we believe that our approach (e.g., improving the data quality for fine-tuning task-specific models) is orthogonal to those efforts that improve the small models’ medical capacity in the pretraining stage. As shown in Table 1, the average performance of ClinGen with PubMedBERT-base (110M Parameters) as the backbone is better than the strongest baseline using PubMedBERT-large (340M Parameters) on the majority of tasks (e.g. Text Class, Relation Extraction, NLI, Fact Verification, and Medication Attribute Extraction), indicating that better data quality for fine-tuning is also crucial for adapting small model on specific tasks.

Moreover, it is important to note that our strategy of incorporating more diverse training data alleviates rather than exacerbates the overfitting issue. By incorporating diverse training data enriched with medical knowledge, we enhance the ability of models to generalize across various clinical scenarios.

---

W2: No rigorous comparison with other prompting optimization methods in the literature.

A: Thanks for the constructive feedback. We would like to highlight that the majority of existing research on synthetic data generation relies on manually crafted, straightforward prompts without dedicated designs [4,5,6]. That being said, we have tried our best to identify two closely related prompt optimization techniques that are potentially applicable to our case. Please see the Comparison with different prompt designs section of the general response to all reviewers for the experimental results and further discussions.

[4] Ye et al. "Zerogen: Efficient zero-shot learning via dataset generation." EMNLP 2022.

[5] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023.

[6] Ye et al. "ProGen: Progressive zero-shot dataset generation via in-context feedback." EMNLP 2022.

---

Thanks again for your constructive suggestions! We hope our responses address your concerns.

We appreciate the time and detailed comments of the reviewer. We answer your questions as follows.

---

Q1: I am not sure why the two stages training is conducted. Was the results yo mix train your training data with generated data together for training?

A: Thank you for the question. In Stage I, we perform fine-tuning of the classifier using few-shot examples in the original training set, specifically K samples per label (in our experiments, K=5). Subsequently, in Stage II, we conduct further fine-tuning of the classifier using the synthetic training data that we generate. We employ this two-stage training because it has been empirically demonstrated to be stable and is commonly employed in various works, including synthetic data generation tasks [1] and other semi-supervised learning methods [2,3]. Note that we mix the few-shot training data with the generated synthetic data and apply this two-stage training consistently for all the baselines to ensure a fair comparison.

[1] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023

[2] Laine and Aila. "Temporal ensembling for semi-supervised learning." ICLR 2017

[3] Chen et al. "Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification." ACL 2020

Q5: The current SOTA results (regardless of fully supervised or few-shot) should be added to Table 1 as reference points.

A: Thanks for the valuable suggestion. In our study, we present the supervised model performance on few-shot demonstrations and the complete original training set, denoted as "Supervised - Few" and "Supervised - Full," respectively. We use PubMedBERT, the same classifier employed for ClinGen and all baseline models, as a reference point.

Following the reviewer's request, we have tried our best to gather relevant data on SOTA models and their performances. However, this task is nontrivial due to several reasons: (1) Many studies only test their models on a subset of the datasets we used; (2) Original papers for some datasets might report results only for base models (like BERT) and not include more recent models (such as PubMedBERT or LinkBERT); (3) In some cases, datasets report only F1 scores without precision and recall metrics.

Despite our best efforts to compile this information, there are still missing values in the available SOTA performance data. We have documented these findings in Tables 6, 7, and 8 in Appendix G, with references from [5-10]. Note that many SOTA performances are based on different classifiers, so these results should only serve as reference points and cannot be directly comparable with the performance of ClinGen and other baselines.

[6] Peng et al. "Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets." BioNLP 2019.

[7] Tinn et al. "Fine-tuning large neural language models for biomedical natural language processing." Patterns 4.4 (2023).

[8] Gu et al. "Domain-specific language model pretraining for biomedical natural language processing." Transactions on Computing for Healthcare 2021.

[9] Yasunaga et al. "LinkBERT: Pretraining Language Models with Document Links." ACL 2022.

[10] BLURB Leaderboard (https://microsoft.github.io/BLURB/leaderboard.html)

---

Q6: D_train in Equation 1 has only K (K<=5) samples per label?

A: $D_{train}$ has only K samples per label, where we set K=5 in our experiments. We intentionally set K to a very small value to simulate a few-shot learning scenario. Recent work on language model fine-tuning suggests that it can help model to learn a good initialization before fine-tuning on synthetic samples [2].

[2] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023.

---

Thanks again for your feedback! Please let us know if you have any further questions.

Q4: The clinical topics/concepts in section 4.1.1 are picked from KG and LLMs. How many topics are from KGs and how many are from LLMs? How is this ratio determined? Would a different ratio impact results?

A: Thank you for bringing up this question. In our experiments, we keep the topics from KGs and LLMs separate. Specifically, we create a candidate set of approximately 6000 topics from KGs and another candidate set of around 2000 topics from LLMs. We then independently sample topics from these two candidate sets to generate two distinct sets of synthetic training samples. Please refer to Table 1 for the experimental results for both sets of synthetic data.

The motivation behind this is that, in our paper, we present KGs and LLMs as two alternative and complementary sources for obtaining topics for readers to understand the influence of topic injection from KG and LLMs distinctly. However, we agree with the reviewer that combining topics from KGs and LLMs has the potential to enhance performance. Thus, we have conducted additional experiments to demonstrate the impact of combining topics from KGs and LLMs at various ratios. Note that we still keep a total of 5000 generated synthetic samples to maintain a fair comparison.

The experimental results indicate that combining knowledge from KGs and LLMs can yield a performance improvement, though not a substantial one. However, note that in practice, it is challenging to tune the ratio in the few-shot setting due to the limited volume of validation labels [5], and thus we only include the 1:1 results in Table 6,7,8 in Appendix G for all 16 datasets.

[5] Perez et al. "True few-shot learning with language models." NeurIPS 2021.

---

Q4 (continued): Why not ask a clinician to hand-pick clinical topics?

A: Thank you for the suggestion. While we admit clinicians' ability to select clinical topics, it is important to point out that such a manual approach involves significant time and money, thus is less scalable. Furthermore, it can be challenging for clinicians to manually create thousands of clinical topics for synthetic training samples. To this end, we utilize KGs and LLMs as more efficient methods to automatically extract clinical topics.

We thank the reviewer for the constructive feedback. We answer your questions as follows.

---

W1: The prompt design is quite simple, i.e. the method itself lacks novelty.

A: Thanks for the insightful feedback. We have answered in the general response for all reviewers. Please refer to the Lack of novelty section for details.

---

W2: The works claim to "infuse" clinical knowledge into the prompt but the prompt simply incorporates clinical topics/concepts (picked from a knowledge graph and LLMs) like "Stoke", rather than actual clinical knowledge (for example, diabetes, stroke, and CHF are risk factors for CKD). Using the word "infuse clinical knowledge" over-claims the novelty of the method.

A: Thank you for the valuable suggestion. While we would like to kindly point out that we do include certain relational information for relation extraction tasks, we acknowledge that we did not define the scope of the "clinical knowledge" integrated into our model. Accordingly, we have made revisions in the introduction to explicitly specify that we employ clinical concepts and basic relationships among them as the form of clinical knowledge in our study. We have also recognized the potential for incorporating more fine-grained clinical knowledge as part of our future work, which is detailed in Appendix A.

---

W3: It is unclear how the prompts in Appendix F are determined. Also, the work does not experiment with different prompt designs.

A: Thanks for the feedback. We notice that the majority of existing research on synthetic data generation relies on manually crafted, straightforward prompts without intricate designs [1,2,3]. Thus, we design our prompt based on the structures of those existing works and task-specific information mentioned in [4], with additional instructions to further incorporate the selected clinical topics and writing styles. For other existing prompt optimization methods in the literature, we identify two related techniques that may be potentially applicable to our case. Please refer to the Comparison with different prompt designs section of the general response to all reviewers for the experimental results and further discussions.

[1] Ye et al. "Zerogen: Efficient zero-shot learning via dataset generation." EMNLP 2022.

[2] Meng et al. "Tuning language models as training data generators for augmentation-enhanced few-shot learning." ICML 2023.

[3] Ye et al. "ProGen: Progressive zero-shot dataset generation via in-context feedback." EMNLP 2022.

[4] Fries et al. "Bigbio: a framework for data-centric biomedical natural language processing." NeurIPS 2022.

---

Q1: One useful and missing reference: Ive J, Viani N, Kam J, Yin L, Verma S, Puntis S, Cardinal RN, Roberts A, Stewart R, Velupillai S. Generation and evaluation of artificial mental health records for natural language processing. NPJ digital medicine. 2020 May 14;3(1):69.

A: Thanks for mentioning this relevant paper. We have modified our manuscript and added the reference in Section 1 and Appendix A.

---

Q2: The paper did not discuss patient privacy in clinical NLP. Synthetic data should be anonymous and can help AI researchers build models without touching real private patient data.

A: Thanks for the insightful feedback. We have answered this question in the general response for all reviewers. Please refer to the Patient privacy concerns section for details.

---

Q3: Textual data is an important component in clinical data but structured data (vital signs, lab tests) is also critical. This work focuses on clinical text synthetic data generation only and more types of data can be considered as future works. However, the capability of LLMs to generate synthetic structured clinical data is questionable.

A: Thanks for the thoughtful feedback. We have answered this question in the general response for all reviewers. Please refer to the Application to QA and EHR tasks section for details.

Q3: Some of the tasks still have some issues in regards to entity coverage and it looks like generally the performance is low on these tasks. Are these common entities that are missed and not identified by LLM or KB? How will this looks like if a medical LLM was used for generation?

A: We appreciate your observation. We acknowledge that there remains a disparity in entity coverage between our generated training data and the ground truth data specifically on the LitCovid and CHEMDNER datasets. This disparity is primarily due to the extensive and diverse sources of entities present in these datasets. Specifically, the LitCovid corpus and the CHEMDNER corpus include 24,960 and 10,000 PubMed articles, respectively, while the BC5CDR dataset only covers 500 PubMed articles. As a result, we observe that there exist more diverse entities for the original LitCovid and CHEMDNER training data. The wide range of entities present in LitCovid and CHEMDNER can be challenging to be fully covered using KGs or querying LLMs.

Nevertheless, it's worth noting that despite this disparity with the ground truth, our approach consistently outperforms or matches baselines in both entity coverage and downstream model performance. We also attempted to utilize a medical LLM (MedAlpaca) for topic generation, but the results were not promising. We provide its entity coverage and downstream model performance results below:

The experimental results show that the topics generated by a medical LLM do not cover a greater number of clinical entities compared to our approach, and they also exhibit lower downstream performance. This could be attributed to the medical LLMs having fewer parameters than ChatGPT, which results in limited instruction-following capabilities. Upon inspecting the topics generated by MedAlpaca, we observe a lack of diversity among the entities, which may also contribute to their inferior performance. We provide 20 examples of these generated entities below:

1. Acebutolol 2. Acetaminophen 3. Acetylcysteine 4. Acetylsalicylic acid 5. Aciclovir 6. Acid 7. Acid chloride 8. Acid anhydrous 9. Acid citrate 10. Acid nitrate 11. Acid phosphate 12. Acid sulfate 13. Acidic 14. Acidity 15. Acidity regulator 16. Acidity regulator citric acid 17. Acidity regulator phosphoric acid 18. Acidity regulator sulfuric acid 19. Acidity regulator trichloroacetic acid 20. Acidity regulator trifluoroacetic acid

One potential strategy to address this issue could involve collecting more task-specific labeled/unlabeled data so that we have more information about target distribution. However, this approach could deviate from the true few-shot scenario that we are investigating in this study [4]. For this reason, we have not included this method in the current explorations. Investigating effective ways to collect and utilize such data represents a valuable direction for future research.

[4] Perez et al. "True few-shot learning with language models." NeurIPS 2021.

---

Thank you again for your review! We hope our responses can address your concerns. Please let us know if you have any further questions.

We thank the reviewer for the time, valuable feedback, and suggestions for improvements. For your several questions, we clarify as follows:

---

Q1: The reviewer likes that the paper did a direct comparison with the prompts from GPT-3.5. However, I'm interested to know if the current medical LLMs are proficient in the tasks described. Should they already excel in these areas, it might somewhat diminish the necessity for the proposed approach.

A: Thank you for the suggestion. Per your request, we have included additional experimental results that compare the direct inference performance of two recent and popular medical Large Language Models (LLMs): PMC-LLaMa [1] and MedAlpaca [2]. The table below illustrates that these medical LLMs perform less effectively than both ClinGen and GPT-3.5. This result can likely be attributed to two reasons:

• The smaller size of the medical LLMs (7B) compared to GPT-3.5, which results in limited language modeling and reasoning capabilities. While we are aware that larger medical LLMs like Med-PaLM exist, they are not publicly accessible and thus cannot be used in our experiments.
• Furthermore, these medical LLMs are typically trained on general medical knowledge sources, such as Wikipedia and Stack Exchange, as indicated by https://github.com/kbressem/medAlpaca#data-overview. This general knowledge may not translate effectively to specific tasks like chemical-protein interaction prediction or health fact verification, leading to suboptimal performance in few-shot learning scenarios.

This observation aligns with findings from other research [3], which suggest that for certain clinical Natural Language Processing tasks, the fine-tuning of more compact models may outperform the in-context learning capabilities of larger LLMs. The results and discussions have been updated to Table 2 in Section 5.3.

[1] Wu et al. "Pmc-llama: Further finetuning llama on medical papers." arXiv preprint arXiv:2304.14454 (2023).

[2] Han et al. "MedAlpaca--An Open-Source Collection of Medical Conversational AI Models and Training Data." arXiv preprint arXiv:2304.08247 (2023).

[3] Lehman et al. "Do We Still Need Clinical Language Models?." CHIL 2023.

---

Q2: What are the limitations of your approach - what tasks is it limited to? How will it perform on more complex tasks such as descriptive QA (since most of the evaluation is on classification tasks)? Will it work on medical records?

A: Thanks for the thoughtful feedback. We have answered this question in the general response for all reviewers. Please refer to the Application to QA and EHR tasks section of the general response for details.

Comparison with different prompt designs (Reviewer D87k, CY6f):

We appreciate the helpful suggestions on adding additional prompt optimization comparison. For this purpose, we have carried out an additional analysis with two recent and representative prompt optimization techniques, namely Reframe [8] and APE [9]. In our setting, Reframe incorporates several principles (e.g., using low-level patterns, itemizing instructions, etc.) to produce high-quality prompts to enhance text generation, whereas APE leverages the LLM itself to automatically optimize the prompts based on the target task information. We demonstrate their performance on various clinical tasks in the following table.

The results indicate that our proposed ClinGen consistently outperforms both baselines. This performance gain may be attributed to the fact that the prompts generated by Reframe and APE mainly focus on incorporating and decomposing task-specific information but do not adequately address the unique challenges for the clinical data generation task, i.e., distribution shift and lack of diversity.

We have included the detailed experimental results and the designs of the prompts they proposed in Appendix L for further reference.

[8] Mishra, Swaroop, et al. "Reframing Instructional Prompts to GPTk's Language." ACL 2022

[9] Zhou, Yongchao, et al. "Large language models are human-level prompt engineers." ICLR 2023.

---

Lack of novelty (Reviewer D87k, CY6f):

We thank the reviewers for the valuable feedback. ClinGen is rationally designed to address our identified issues of limited domain knowledge coverage and distribution shifts using KGs/LLMs, which have not been fully solved by existing works. We consider simplicity and effectiveness as one strength of our approach. Simplicity often leads to better generalizability, and our method has indeed led to consistent performance gain across 16 datasets covering 7 clinical NLP tasks. We also hope that our intuitive methodology will inspire further research into clinical text data generation using LLM-prompting techniques.

While we have experimented with more complex prompt designs as detailed in papers [10,11], these designs do not lead to better performance when compared to ClinGen. Please see the Comparison with different prompt designs section of the general response for all reviewers for details.

Furthermore, it is worth noting that we view our contribution as identifying the intrinsic challenges for LLM-based data generation on clinical NLP tasks (Section 3.2), and leveraging proper techniques to tackle them (Section 4). Although the basic idea of combining KGs and LLMs to benefit data generation is conceptually simple, the actual development of a concrete and effective approach to utilize these resources for performance enhancement is nontrivial. ClinGen is not a simple combination of KGs and LLMs, but an elaborate design to mitigate the identified drawbacks. As demonstrated in experiments, our design effectively overcomes these challenges (Section 6), resulting in notable performance gains (Table 1).

[10] Mishra, Swaroop, et al. "Reframing Instructional Prompts to GPTk's Language." ACL 2022

[11] Zhou, Yongchao, et al. "Large language models are human-level prompt engineers." ICLR 2023.

---

Thanks to all the reviewers again! We hope our responses have addressed your concerns. Please let us know if you have any further questions, and we are happy to discuss further.