We thank the reviewer for recognizing the importance of investigating the behavior of foundation models in medical coding. We have carefully addressed all suggestions to improve clarity and respond to the specific comments below.

Generalization vs. Memorization and Data Leakage Risks: We thank the reviewer for raising this important point. We fully agree that clinical prediction models should leverage informative patient histories to generate accurate and personalized treatment trajectories. In fact, part of the main purpose of our work is to draw attention to the distinction between model generalization and harmful memorization that can leak sensitive information from training data, even when prompted with partial or non-specific inputs.

The concern is not that models generate plausible sequences given rich clinical histories, which is expected. Rather, the risk arises when high-confidence predictions of sensitive conditions (e.g., HIV status, substance use) can be elicited from incomplete or generic inputs. This may signal that the model is not generalizing from population-level patterns but is memorizing individual cases seen during training. For instance, in our substance use example, a patient with repeated falls was assigned a sensitive diagnosis with no clear clinical justification in the input. This suggests the model may be reproducing memorized associations from similar training examples, raising serious privacy concerns.

Our framework is explicitly designed to detect such privacy leakage through memorization. Tests T1–T4 focus on scenarios where an adversary, with limited but plausible access to background information (e.g., demographics, early labs), could prompt the model and recover sensitive attributes about a patient. Such inputs could realistically be inferred from public EHRs (e.g., MIMIC-III), leaked insurance claims, or personal disclosures on social media. For instance: T1, T2, and T6 demonstrate what an adversary might infer with access to partial knowledge (e.g., a few diagnosis codes) from an individual.

T3 models the leakage potential when the adversary gains access to the embeddings or limited training samples.

T4 estimates whether an adversary can determine if a given patient was part of the training cohort, based on access to a partial view of their medical records.

To distinguish generalization from leakage, we apply perturbation-based tests: if the model still outputs rare, sensitive codes when key identifier information is removed or substituted, this behavior is less likely due to robust reasoning and more likely due to memorized correlations or training data artifacts. These patterns are not always evident without systematic probing, which is precisely what our framework enables. We will further clarify this distinction in the next version to ensure the motivation behind our framework is made explicit.

Empirical Evidence of Memorization in a Controlled Setting: We demonstrate that our framework can detect memorization in a non-healthcare domain with known ground truth, serving as a positive control experiment. Thank you for this insightful suggestion. Validating positive controls in real-world EHR data is inherently challenging due to the absence of ground truth regarding which samples may have been memorized by a model. To address this, we designed a controlled synthetic experiment that mimics the structure of EHR-like sequences and allows us to explicitly simulate memorization behavior. In this setup, each sequence consists of digits (analogous to diagnosis codes), where higher digits occur less frequently, following a skewed distribution $p=\frac{1}{(d+1)^2}$reflecting the prevalence imbalance commonly observed in clinical data. We implemented a toy foundation model that learns embeddings from these sequences and generates future sequences based on them. We added further stochasticity to the generative process to simulate more complex setups like EHR-FM. To simulate memorization, we introduced a deterministic rule: if a sequence begins with the digits [0,1], a specific embedding dimension is set to 1; otherwise, it remains 0. The remaining dimensions are learned normally. In generation, this memorized embedding dimension directly affects the output—if set to 1, the model is forced to produce a rare digit (e.g., 9), emulating the behavior of memorizing rare or sensitive clinical cases. We then applied our probing tests to this synthetic model to assess whether they could successfully detect memorized samples. Below, we present the results of these tests, which demonstrate the ability of our framework to flag memorization under controlled, interpretable conditions.

These results illustrate how effectively an adversary can infer memorized content depending on the amount of information they have access to. For instance, when the probing model is trained on a larger fraction of the input (e.g., >20% of the sequence), its ability to recover memorized outputs improves substantially, reaching a precision of 0.92. This confirms that our test reliably detects memorization under varying levels of adversarial knowledge. We also validate the probing test in our controlled synthetic setup using explicit positive controls. Specifically, we take a memorized sequence—defined as one that begins with [0, 1]—which deterministically produces the rare digit 9 in the generated output due to its hardcoded influence in the embedding. To test the sensitivity of our perturbation framework, we generate 9 additional sequences by modifying only the first digit (i.e., replacing the 0 with each digit from 1 to 9), while keeping the rest of the sequence unchanged. For each perturbed input, we generate 1000 trajectories. The perturbation test T5 then evaluates what fraction of the generated sequences still include the rare digit 9. In the results of T5, the sharp drop in the fraction of sequences containing the sensitive code, as compared to the original [0, 1] input, confirms that the model has memorized a specific pattern, and the test is capable of revealing this behavior reliably.

We are happy to include an extended version of this study in the appendix to validate the performance of our tests on positive control samples.

Clarification on Tables: In T2 results we report the percentage of individuals for whom the model reveals a sensitive diagnosis code in the generated EHR trajectories. We define a "revealed" case as one in which more than 30% of the generated sequences contain the sensitive code, indicating a high likelihood of unintended disclosure. These results are presented across varying levels of prompt, simulating scenarios where an adversary has access to different amounts of information about a patient. This setup allows us to estimate the risk of sensitive information leakage as a function of adversarial knowledge. The tables in the appendix show the same test for different subgroups (for instance elderly population). This helps us understand if the risk varies across subpopulations.

Evaluation on different models: We want to note that our primary goal was to develop and validate an effective evaluation methodology, and not focus on the benchmarking task. Currently, there are only a handful of models available and they may not pose significant memorization concerns, the rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models. Nevertheless, We have extended our analysis to include MEDS-torch, another EHR-FM benchmark [1]. Below shows a summary of the T1 results for predicting 20 steps ahead. The results follow similar trends to those of EHR-Mamba2, with only slight variation in “50 prompt” settings. We are happy to include full results in the appendix to illustrate the generality and applicability of our approach.

[1] https://github.com/Oufattole/meds-torch

| Prompt Type | MEDS-torch (20) |

| Random | 0.701 +- 0.1376 |

| Static | 0.669 +- 0.149 |

| 10 prompts | 0.65 +- 0.162 |

| 20 prompts | 0.65 +- 0.1830 |

| 50 prompts | 0.677 +- 0.144 |

We thank the reviewer for their insightful feedback and thoughtful comments. The review included several constructive suggestions, such as validating our tests using controlled positive examples, which we have carefully incorporated and believe have helped improved the paper.

Positive control experiment Thank you for this insightful suggestion. Validating positive controls in real-world EHR data is inherently challenging due to the absence of ground truth regarding which samples may have been memorized by a model. To address this, we designed a controlled synthetic experiment that mimics the structure of EHR-like sequences and allows us to explicitly simulate memorization behavior. In this setup, each sequence consists of digits (analogous to diagnosis codes), where higher digits occur less frequently, following a skewed distribution $p=\frac{1}{(d+1)^2}$reflecting the prevalence imbalance commonly observed in clinical data. We implemented a toy foundation model that learns embeddings from these sequences and generates future sequences based on them. We added further stochasticity to the generative process to simulate more complex setups like EHR-FM. To simulate memorization, we introduced a deterministic rule: if a sequence begins with the digits [0,1], a specific embedding dimension is set to 1; otherwise, it remains 0. The remaining dimensions are learned normally. In generation, this memorized embedding dimension directly affects the output—if set to 1, the model is forced to produce a rare digit (e.g., 9), emulating the behavior of memorizing rare or sensitive clinical cases. We then applied our probing tests to this synthetic model.

These results illustrate how effectively an adversary can infer memorized content depending on the amount of information they have access to. For instance, when the probing model is trained on a larger fraction of the input (e.g., >20% of the sequence), its ability to recover memorized outputs improves substantially—reaching a precision of 0.92. This confirms that our test reliably detects memorization under varying levels of adversarial knowledge. We also validate the probing test in our controlled synthetic setup using explicit positive controls. Specifically, we take a memorized sequence, defined as one that begins with [0, 1], which deterministically produces the rare digit 9 in the generated output due to its hardcoded influence in the embedding. To test the sensitivity of our perturbation framework, we generate 9 additional sequences by modifying only the first digit (i.e., replacing the 0 with each digit from 1 to 9), while keeping the rest of the sequence unchanged. For each perturbed input, we generate 1000 trajectories. The perturbation test T5 then evaluates what fraction of the generated sequences still include the rare digit 9. In the results of T5, the sharp drop in the fraction of sequences containing the sensitive code, as compared to the original [0, 1] input, confirms that the model has memorized a specific pattern, and the test is capable of revealing this behavior reliably.

We are happy to include an extended version of this study in the appendix.

Clarification on tests: The reviewer raised a few questions regarding each test which we would like to clarify here.

T1 - The examples provided in the Appendix for T1 are subsets of the datasets we used for validating T1. We used LLM-generated hypothetical patient records and multiple variations of each for the full validation. We are happy to include more examples in the appendix to further support T1 validation. T2: *The patient discussed in the paper was not an anecdotal example, but a true sample identified by T2 as memorized. The last column of Table 1 reports these positive samples as the number of individuals for whom sensitive conditions (e.g., HIV, mental health) were predicted with high confidence.

T3: T3 is designed to assess how much private information an adversary could potentially extract from a model’s embedding, given access to varying amounts of training data. Near-random performance in this setting suggests that the adversary is unable to extract meaningful sensitive information from the embeddings, given the specific data access scenario.

However, we acknowledge the reviewer’s valid concern on testing the capability of T3 on memorized samples. To address this, we included a simulated positive control experiment (see earlier response) that demonstrates the effectiveness of the test under known memorization conditions.

T6: In this test, we identify two cases where knowledge of a rare diagnosis allows inference of a sensitive condition. While we cannot conclusively determine whether these reflect direct memorization, the consistent appearance of these codes in high-likelihood outputs is concerning in itself. It suggests that the model may be implicitly encoding sensitive attributes in ways not fully explained by statistical associations.

Evaluation on different models: We want to note that our primary goal was to develop and validate an effective evaluation methodology, and not to focus on the benchmarking task. Currently, there are only a handful of models available, and they may not pose significant memorization concerns. The rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential, and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models. Nevertheless, we have extended our analysis to include MEDS-torch, another EHR-FM benchmark. Please find the details in the response for reviewer zyw8 that shows a summary of the T1 results.

Mitigation strategy: We appreciate the reviewer’s interest in mitigation, which is indeed a critical area. However, the primary objective of this work is to provide a comprehensive strategy for identifying memorization instances that are potentially harmful for patient privacy in EHR foundation models. Proposing concrete mitigation strategies is out of scope for this work, as the root causes of memorization vary widely, ranging from data duplication, skewed sampling, and rare event bias to architectural and training design decisions. Each of these factors requires a targeted mitigation strategy.

Moreover, mitigation decisions are often application- and policy-dependent, influenced by factors such as the sensitivity of leaked attributes, the risk of re-identification, regulatory frameworks, and institutional tolerances for risk. Our goal is to equip data custodians and policy makers with quantitative tools to assess risk as a function of the potential information an adversary can gain access to. For instance: T1, T2, and T6 demonstrate what an adversary might infer with access to partial knowledge (e.g., a few diagnosis codes) from an individual.

T3 models the leakage potential when the adversary gains access to the embeddings or limited training samples.

T4 estimates whether an adversary can determine if a given patient was part of the training cohort, based on access to a partial view of their medical records.

This level of granular evaluation helps decision-makers define appropriate use boundaries—for example, determining whether a model is safe for public release, or if it should remain restricted to internal clinical environments. That said, we do briefly discuss possible mitigation directions, including post-hoc safety layers, red-teaming, and retraining with privacy-preserving objectives. However, evaluating the trade-offs between these interventions and model utility, especially at the scale of foundation models, is out of scope for this work. We certainly hope this work will draw the attention of researchers to investigate and develop novel mitigation strategies tailored to foundation models in sensitive domains.

Code availability: We apologize for this problem. The original anonymous link had issues with displaying some files. This issue has now been resolved, and the full code is accessible through the same link. Please let us know if you encounter any further issues.

Prompts and Example Availability: Due to the data usage agreements associated with the MIMIC dataset, we are not permitted to publicly release specific examples of patient EHR records. Even when describing illustrative cases in the paper, we intentionally keep the details vague and high-level to avoid exposing any identifiable information. However, all the tests and examples presented in our work are fully replicable for researchers who have authorized access to the MIMIC dataset.

We thank the reviewer for their valuable feedback and thoughtful comments. We appreciate the recognition of the importance of the problem and the acknowledgment that our framework is conceptually intuitive and accessible, which we believe strengthens its potential impact. The reviewer’s suggestions, particularly the request for positive control validations and clarifications on key points, were very constructive and insightful. In response, we have carefully addressed all comments and incorporated the recommended additions and explanations, as outlined in detail below.

Demonstrating Detection of Memorization in with ground truth (positive control experiment) Thank you for this insightful suggestion. Validating positive controls in real-world EHR data is inherently challenging due to the absence of ground truth regarding which samples may have been memorized by a model. To address this, we designed a controlled synthetic experiment that mimics the structure of EHR-like sequences and allows us to explicitly simulate memorization behavior. In this setup, each sequence consists of digits (analogous to diagnosis codes), where higher digits occur less frequently, following a skewed distribution $p=\frac{1}{(d+1)^2}$reflecting the prevalence imbalance commonly observed in clinical data. We implemented a toy foundation model that learns embeddings from these sequences and generates future sequences based on them. We added further stochasticity to the generative process to simulate more complex setups like EHR-FM.

To simulate memorization, we introduced a deterministic rule: if a sequence begins with the digits [0,1], a specific embedding dimension is set to 1; otherwise, it remains 0. The remaining dimensions are learned normally. In generation, this memorized embedding dimension directly affects the output—if set to 1, the model is forced to produce a rare digit (e.g., 9), emulating the behavior of memorizing rare or sensitive clinical cases. We then applied our probing tests to this synthetic model to assess whether they could successfully detect memorized samples. Below, we present the results of these tests, which demonstrate the ability of our framework to flag memorization under controlled, interpretable conditions.

These results illustrate how effectively an adversary can infer memorized content depending on the amount of information they have access to. For instance, when the probing model is trained on a larger fraction of the input (e.g., >20% of the sequence), its ability to recover memorized outputs improves substantially, reaching a precision of 0.92. This confirms that our test reliably detects memorization under varying levels of adversarial knowledge.

We also validate the probing test in our controlled synthetic setup using explicit positive controls. Specifically, we take a memorized sequence—defined as one that begins with [0, 1]—which deterministically produces the rare digit 9 in the generated output due to its hardcoded influence in the embedding. To test the sensitivity of our perturbation framework, we generate 9 additional sequences by modifying only the first digit (i.e., replacing the 0 with each digit from 1 to 9), while keeping the rest of the sequence unchanged. For each perturbed input, we generate 1000 trajectories.

The perturbation test T5 then evaluates what fraction of the generated sequences still include the rare digit 9. In the results of T5, the sharp drop in the fraction of sequences containing the sensitive code, as compared to the original [0, 1] input, confirms that the model has memorized a specific pattern, and the test is capable of revealing this behavior reliably.

We are happy to include an extended version of this study in the appendix to validate the performance of our tests on positive control samples.

Non-healthcare applications: We agree that extending the framework to non-healthcare domains such as legal or financial text is a promising direction. However, our current focus is intentionally limited to healthcare due to the high-stakes nature of patient privacy and the unique risks memorization poses in this setting. Furthermore, the domain allows us to tie the notion of memorization to clinically meaningful definitions of harm. By examining how different types of information (e.g., demographics, visit patterns, clinical labs) contribute to leakage, we aim to provide decision-makers with practical tools to assess risk and shape responsible deployment strategies.

Evaluation on different models: We want to note that our primary goal was to develop and validate an effective evaluation methodology, and not to focus on the benchmarking task. Currently, there are only a handful of models available, and they may not pose significant memorization concerns. The rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential, and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models.

Nevertheless, we have extended our analysis to include MEDS-torch, another EHR-FM benchmark [1]. Below shows a summary of the T1 results for predicting 20 steps. The results follow similar trends to those of EHR-Mamba2, with only slight variation in “50 prompt” settings. We are happy to include full results in the appendix to illustrate the generality and applicability of our approach.

[1] https://github.com/Oufattole/meds-torch

| Prompt Type | MEDS-torch (20) |

| Random | 0.701 +- 0.1376 |

| Static | 0.669 +- 0.149 |

| 10 prompts | 0.65 +- 0.162 |

| 20 prompts | 0.65 +- 0.1830 |

| 50 prompts | 0.677 +- 0.144 |

Broader discussion about limitations: Thank you for raising this point. While we currently discuss limitations and extensions within the context of each test, we agree that making them more explicit could improve clarity. For example, our choice to prompt the model with initial visit codes reflects a realistic adversarial setting involving partial data leakage. More targeted prompting—e.g., selecting high-yield codes—would require clinical insight that most adversaries are unlikely to possess. As noted in the paper, our prompting strategy is flexible and can be adapted to simulate different types of adversary access. We also provide recommendations for combining tests, such as T3 and T6, to capture sequential or time-dependent leakage patterns.

Formatting: Thank you for pointing out the few typos; we updated them for the next version.

We thank the reviewer for their thoughtful feedback, their support for our open-source toolkit, and the inclusion of vulnerable subgroup analysis to address ethical concerns. All suggestions have been addressed to improve clarity and respond to specific comments.

Benchmark Model Specificity: As the reviewer noted, we selected EHR-Mamba2 for our main experiments because it is publicly available and has a fully reproducible training pipeline. EHR-Mamba2 is also representative of many EHR-FMs in its architecture, including token-based inputs, sequential modeling over visits, and hybrid attention mechanisms. Moreover, our framework is model-agnostic by design, making it applicable to a wide range of models beyond those evaluated here. We have extended our analysis to include MEDS-torch, another EHR-FM benchmark [1]. Below shows a summary of the T1 results for predicting 20 steps ahead. The results follow similar trends to those of EHR-Mamba2, with only slight variation in “50 prompt” settings. We are happy to include full results in the appendix to illustrate the generality and applicability of our approach.

| Prompt Type | MEDS-torch (20) |

| Random | 0.701 +- 0.1376 |

| Static | 0.669 +- 0.149 |

| 10 prompts | 0.65 +- 0.162 |

| 20 prompts | 0.65 +- 0.1830 |

| 50 prompts | 0.677 +- 0.144 |

We also want to note that our primary goal was to develop and validate an effective evaluation methodology, and not to focus on the benchmarking task. Currently, there are only a handful of models available, and they may not pose significant memorization concerns, the rapid advancement of EHR-FMs suggests that these risks may soon become more pressing. Raising awareness of these issues now is essential and we believe this work provides a strong foundation for future studies to assess memorization risks across a broader range of models. [1] https://github.com/Oufattole/meds-torch

Assessment of Metrics and Tests: The reviewer raises two important points. First, regarding the clinical specificity of our tests, this is a deliberate and necessary design choice, not a limitation. Memorization in healthcare is nuanced and context-dependent, tightly interwoven with clinical semantics. In Tests T2 and T3, we focused on representative sensitive conditions (e.g., HIV, mental health, reproductive health), but sensitive conditions are not limited to conditions listed. The role of our tests is to quantify memorization risk under varying information scenarios (prompt information an adversary can gain access to). Final decisions about acceptable thresholds or policy responses are inherently contextual and should rest with institutional decision-makers or end-users, who must balance privacy risks against operational constraints.

Second, we agree that T1 is not an absolute measure or directly comparable across models. As emphasized in the paper, T1 offers an aggregate memorization signal to flag suspicious samples, especially relative to random baselines. However, it is most informative when paired with T2–T4, which provide more targeted privacy assessments.

Probing Test (T3) clarification: We used a simple multilayer perceptron (MLP) classifier to probe the embeddings for memorization. In preliminary experiments, we evaluated Lasso, XGBoost, and MLP classifiers, selecting the best-performing model based on validation performance. The final probing was conducted using 10-fold cross-validation, repeated across multiple data splits with varying train/test ratios (e.g., 80/20). To represent varying levels of information that an adversary might possess, we simulate different settings by adjusting the fraction of training data and the prompt lengths used to generate embeddings. Specifically, we consider prompt lengths of 10, 20, and 50 tokens, and training fractions of the test set, 0.1%, 10%, and 20%. We assess the information stored in the embeddings by training probing models g(⋅) to predict sensitive attributes (e.g., diagnoses) from the embeddings. These probing models are trained using either external data or a subset of the original training data (again to simulate different adversary scenarios).

Prompt Design for T1 and T2: For T1 and T2, the model was prompted with the first N codes (N = 10, 20, 50) of a patient trajectory. These included diagnosis codes (ICD-9), medications, lab test results (e.g., hematocrit levels), and demographic information (e.g., age, sex). For example, a prompt for a MIMIC patient might include:

[Age: 48], [Sex: Male], [ICD9: 428.0 – CHF], [LAB: Creatinine High], [MED: Furosemide], ...

We focused on early-stage information to simulate realistic usage and avoid direct leakage of target codes. An adversary could exploit publicly available clinical datasets (e.g., MIMIC-III), personal health disclosures on social media, or leaked insurance claims to construct plausible prompts. For instance, by knowing a patient is a 60-year-old female with diabetes and recent lab values, an attacker could generate a valid input sequence and prompt the model to elicit sensitive follow-up codes.