Dear Reviewer C5eR,

We sincerely thank you for the constructive feedback that helps improve our work. We address the concerns you raised below. The full new results are provided in: https://anonymous.4open.science/r/ICML25-7F63/ICML25.pdf

[Q1-Synthetic Priors] We clarify that FLAIR's DDE is meta-trained using tasks from BNNs and SCMs, which model complex dependencies and causal structures fundamental in the data stored in real-world databases. The priors span diverse functional forms reflecting causal relationships in real query-data-output relations and are trained in a fully Bayesian manner over a wide hyperparameter space broader than any single-task point estimate. We also adopt Occam’s razor bias as [1], favoring lower-parameter models, which aligns with Bayesian and cognitive principles.

We validate the generalizability of the synthetic priors empirically: We meta-train FLAIR once and for all and test it across 4 core analytics tasks (i.e., CE, AQP, classification), reflecting distinct data-query patterns and dynamics. The strong and consistent performance across all these tasks, even superior to the strong baseline fine-tuning on real-world data, confirms the generalizability of our synthetic priors.

As suggested, we further validate this with new experiments simulating insert/delete/update-heavy drifts. Results below (details in Table A) show robust and optimal performance, especially in update-heavy (UH) cases. This confirms the alignment between our priors and actual drift patterns.

[Q2-Latency/Partial Feedback] We agree with you that the immediate availability of execution results may not always hold. To address this, as suggested, we extend our evaluation to consider more complex scenarios and design two settings:

Delayed feedback: context memory updates every k steps (5% and 10% of its size) rather than in real time.

Partial feedback: 5% and 10% of context pairs are randomly dropped to mimic missing or partial feedback.

The results below (details in Table G) show that: (1) partial feedback performs slightly better, suggesting recency of execution results matters more than completeness. (2) FLAIR is resilient to both cases, which confirms its applicability in real systems.

[Q3-Larger Datasets] We share your view that scalability is a key consideration beyond effectiveness. As discussed in Appx G, FLAIR is designed with scalability in mind with complexity linear scaling w.r.t. input and context size. Its runtime involves only a feedforward pass over compact context memory for adaptation, without backward gradient updates.

We evaluated FLAIR on real and widely accepted benchmarks STATS and JOB-light, which show low inference latency (Sec 4.3) and stability with context memory scaling (Fig 11, Appx J1). To further evaluate the scalability of our FLAIR, we add new experiments on a larger dataset TPC-H [2](10GB, 86M records, complex many-to-many joins). Results in Table H show that FLAIR outperforms baselines in all cases, with 3.5% and 7.2% gains under mild and severe drift.

Across all benchmarks, FLAIR maintains low latency(~10 ms/query), compact memory footprint(~5 MB), and stable storage overhead, showing its suitability for high-throughput, large-scale systems.

[Remark] We clarify that our work sits at the intersection of machine learning and data systems. The key insights of our work lie in addressing two fundamental ML challenges: how to enable on-the-fly adaptation without retraining and how to achieve context-aware prediction, as stated in lines 036–042. These challenges are central to real-time analytics on dynamic structured data for high stacks applications such as healthcare and stock trading. Thus, we focus on typical structured data analytics tasks, addressing these challenges with contributions grounded in ML, i.e., in-context adaptation and Bayesian meta-training, achieving promising performance. We believe this work contributes to the broader ML community and will resonate with a wide ML audience working on concept drift, structured data analytics, and real-time systems, complementing ongoing efforts in these areas.

We are grateful for the chance to discuss our work's potential and wish to thank you again for your valuable input. We hope to have addressed your concerns and would highly appreciate your consideration for re-evaluating your initial rating.

[1]Tabpfn. ICLR2023

[2]https://www.tpc.org/

Dear Reviewer 5u5N,

We thank you for your recognition of our work and your insightful comments. Below, we provide detailed responses to the specific concerns you raised. Full tables/figures are in: https://anonymous.4open.science/r/ICML25-7F63/ICML25.pdf

expect a comparison with more recent approaches.

As suggested, we have added recent baselines: SOLID [1], a context-aware fine-tuning adaptation method, and DtACI [2], an adaptive conformal inference (ACI)–based fine-tuning method. Results below (details in Table F) show that FLAIR outperforms both under drift. This is because SOLID’s residual-based detection and DtACI’s ACI mechanism miss subtle, continuous query-driven drift in databases, reducing accuracy, especially in tail cases. In contrast, FLAIR’s in-context adaptation ensures timely and better adaptation to continuous drift.

how the authors define the mild drift and severe drift？

Thank you for pointing this out. We clarify the drift setting below:

Mild Drift: randomly select 50% of records from the database and independently permute their column values, altering data distribution and inter-column correlations.

Severe Drift: randomly select 60% of records, independently permuting their columns, and performing random insertions, deletions, and updates, which affects 10% of the total data (keeping the total data size constant).

This setting follows the recent work [3]. We will refine the descriptions in Figure 10 and Appx H6 for clarity.

confused with information conveyed in Figure 7

We appreciate your valuable feedback. We selected classical methods in Figure 7 for their interpretable decision boundaries, which provide an intuitive understanding of model behavior under concept drift.

To enhance comparison, we have added a recent method Type-LDD [4], a drift-aware classifier via knowledge distillation, in Figure C. While Type-LDD surpasses classical methods, FLAIR outperforms it due to the Type-LDD’s delayed adaptation from its detect-then-adapt strategy.

the concept of 'concept drift' appears to be no different from the distribution shift problem

We agree that clearly distinguishing our core focus concept drift in databases from the broader notion of distribution shift is important, and we would like to further clarify this as suggested. Distribution shift typically refers to a mismatch between training and test data distributions, often without inherent temporal evolution. In contrast, concept drift in our work refers to the ongoing, temporal evolution of data and queries in operational database settings. In databases, this form emerges naturally: runtime queries continuously trigger insert/delete/update operations, incurring cumulative changes in both data and query. These drifts are ongoing and unpredictable, requiring models to adapt in real time rather than being trained repeatedly with new data distributions.

To clarify this clearly, we refine our problem definition as follows:

Definition (Concept Drift in Databases) Let $\mathbf{d}_t$ denote the underlying data of a database at time t, and $\mathbf{q}_t$ denote a user query at time t. Given the data-query pair $(\mathbf{d}_t,\mathbf{q}_t)$, let $\mathbf{y}_t$ represent the corresponding prediction output (e.g., row counts in cardinality estimation). Concept drift occurs at time t if the joint distribution of queries, data, and predictions changes, i.e.,

$P_t(\mathbf{q},\mathbf{d},\mathbf{y})\ne P_{t+1}(\mathbf{q},\mathbf{d},\mathbf{y})$.

Here, drift can arise from two distinct but interrelated sources:

(1) Query drift, from evolving user behavior.

(2) Data drift, caused by frequent insert/delete/update operations changing underlying data distributions.

Notably, changing data not only changes the marginal distribution $P(\mathbf{d})$, but also affects the conditional distribution $P(\mathbf{y}|\mathbf{q},\mathbf{d})$, i.e., the same queries may yield different outputs over time. This suggests that concept drift in databases involves shifts in the joint distribution of queries, data, and predictions, and their interaction.

We'll further clarify this distinction and update our problem formulation accordingly.

We hope our responses above have sufficiently addressed your concerns and can improve your evaluation of our work.

[1] Calibration of Time-Series Forecasting- Detecting and Adapting Context-Driven Distribution Shift. KDD 2024

[2] Conformal Inference for Online Prediction with Arbitrary Distribution Shifts. JMLR 2024

[3] Detect, Distill and Update: Learned DB Systems Facing Out of Distribution Data. SIGMOD 2023

[4] Type-LDD: A Type-Driven Lite Concept Drift Detector for Data Streams. TKDE 2024

Dear Reviewer uD1S,

We sincerely thank you for your positive review and insightful comments. We address your concerns below and attach new results here: https://anonymous.4open.science/r/ICML25-7F63/ICML25.pdf

Q1: does the work consider both data and query change?

Yes, we considered both data and query change in Appx H6 (line 1287). We evaluated the query change of SELECT/UPDATE/INSERT/DELETE with varied ratios, and among them, UPDATE/INSERT/DELETE incur data change.

We further evaluate our pretrained model on insert/delete/update-heavy (IH/DH/UD) query changes. Results below (details in Table A) show that FLAIR adapts consistently well to both query and data changes across different settings.

Q2: how to identify the impact of concept drift?

Unlike reactive detect-then-adapt methods, FLAIR uses the dynamic context memory to adapt on the fly via the dynamic decision engine, avoiding the detection overhead.

Incremental drift tests (Figure 5 in Sec 4.4) and our new abrupt drift tests below (details in Table B) confirm that FLAIR achieves the best performance without explicit detection.

Q3: how to ensure efficiency of meta-trained part? have you compared common learning strategies?

DDE is meta-trained only once to approximate $q_{\theta}(y|x,\mathcal{C})$ across diverse tasks. During inference, it remains frozen and adapts efficiently via a forward pass using context $\mathcal{C}$, built from recent query-output pairs, thereby enabling training-free adaptation.

As suggested, we update Figure 3 for clarity (see Figure A) and add training from scratch (RT) besides the fine-tuning (FT) and distillation (KD) baselines (details in Table C). Results show that RT and FT underperform in adaptation to new concepts, as they fit outdated data, while FLAIR archives timely and better adaptation.

Q4: runtime complexity analysis

FLAIR's initialization complexity is $O(\delta \sum n_i)$ with $O(N_v)$ for updating modified records. Query encoding cost $O(N_J+N_F)$ is negligible. TFM incurs $O(d_a\delta^2)$ (self-attention) and $O(d_a\delta)$ (cross-attention). DDE scales linearly as $O(d_a\varrho)$ with context size $\varrho$. Please see Appx G for more details.

theoretical analysis should embed concept drift

We clarify that we model concept drift through insertions and deletions, which are key drivers of drift in databases, and derive worst-case bounds. As suggested, we will make the link to concept drift clearer in Sec 3.4.

[Additional Dataset] We add TPC-H [1] (10GB, 86M records, 8 tables, 61 attributes). Experimental settings follow Appx H6. Results below show that FLAIR outperforms all the baselines, especially in tail and severe cases (details in Table D).

[Parameter Analysis] As suggested, we clarify key parameter settings: the bin number is 40, context memory size is 80, the task encoder 8 attention layers with 8 heads, and DDE 12 layers with 4 heads. Detailed settings for FLAIR and baselines will be added to Appx H4.

We add analysis on bin number $\delta$ (see Figure B), showing trade-off between accuracy and training time. Appx J provides more sensitivity analysis on context memory.

[Related Work] As suggested, we expand related work with more references and discussion (see Table E). We group existing methods into: Lazy: retrain models after drift detection, high cost; Incremental: adapt gradually, respond slowly; Ensemble: maintain model pool to cover varied concepts, resource-heavy. In contrast, FLAIR proposes a new in-context adaptation paradigm, enabling timely adaptation without retraining.

[Setting Clarification] We clarify that our work is in the supervised setting. For CE and AQP, labels come from query execution (Appx H1). For regression/classification, ground truth is available pre- and post-drift, with results in Appx J1/J2, respectively. We'll clarify this in Sec 4.5.

[Definition Refinement] As suggested, we refine Definition 2.1 to distinguish drift in databases from drift in data streams. In databases, predictions depend on both query and data, and the data evolves due to query-driven operations. We thus refine the input into two distinct but interrelated parts: query and data, both can incur drift in databases. Due to space limit, please see our response to Reviewer 5u5N-Q4 for the updated definition.

We hope our clarifications and new results have addressed your concerns and can improve your evaluation of our work.

[1] https://www.tpc.org/