WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales

Thank you very much for your time, interest, & detailed feedback! We refer to the this anonymous link for supplemental figs: https://sites.google.com/view/authorresponse/home

Claims and Evidence:

• SOTA statement: We have removed the quoted statement, & revised it to refer specifically to how our methods compare to the evaluated baselines (regarding adaptation & detection speed).
• See “Suggested baselines/Relation to Lit” section.

Methods And Evaluation Criteria: See “Spillover from 7BRr” in response to oGBa for:

• Specific baseline e-process
• Runtime metric
• Coverage vs alarms

Theoretical Claims:

• Thm 3.2 & 3.3: We are happy to rename “Thm 3.2” to “Proposition 3.2,” and same for 3.3 (we think proposition is more accurate than lemma).

• Intuition for Thm 3.3: The guarantee is controlling the average-run length (ARL) under the null hypothesis, meaning how long the “scheduled” monitoring procedure can be expected to run without a false alarm before it needs to be reset. What you describe about detection delay would be a guarantee under an alternative hypothesis; we do not report such guarantees.

Experimental Designs Or Analyses:

• X-CTM design & ablation of WCTM without X-CTM: The $X$-CTM uses the same design (ie, same betting function) as the WCTM, but with a nearest-neighbor-distance score. Without a XCTM, a WCTM has 2 main options: (1) no adaptation, which reduces to standard CTM; or, (2) assume that the changepoint occurs at deployment time. See S2.1 at the link for an ablation study on (2).

• Eq (17): If every test point is added to the calibration data (as in standard CTMs), then the weights become intractable to compute (Prinster et al. 2024); but, can avoid this by, at some point $t_{ad}$ (given by $X$-CTM), treating the cal data as fixed (no longer adding recent test points to it). (Fixing the cal set results in the WCTM testing a hypothesis that is conditioned on the cal set, vs marginal over it; we will add an appendix section to elaborate.)

• Density-ratio estimator: The density-ratio weights were estimated via probabilistic classification, as described in Tibshirani et al (2019), Eq (9). Tabular experiments used sklearn’s LogisticRegression; image experiments used a prefit MLP with 2 hidden ReLU layers, fit on 5K pre-change and 5K post-change points. An example ablation study for the effect estimator misspecification is provided in S2.2 at the link provided.

• Widths shrunk, betting ablations & benign/severe shifts: See response #4 to b4ce.

• Table 2 questions: We meant to write “Unnecessary Alarm” to refer to alarms raised when there is a mild/benign shift. The rejection threshold for Table 2 was $10^6$ (as in Fig 3). WCTM unnecessary alarms are evidence that density-ratio estimation is harder for image data. We will add clear discussion of empirical vs nominal rates.

• CTM higher ADD: Related to our discussion around Eq (17), once the WCTMs treat the cal data as fixed, they can sometimes reject faster than standard CTMs (the latter adds each test point to the cal set, which confuses the source/target distinction and sacrifices power).

• Benefit of WCTMs for tracking loss metric: A loss metric can take the role of the nonconformity score, and WCTMs then would adapt to distribution shifts to maintain statistically valid estimates of how that loss compares to cal data.

• Suggested baselines/Relation to Lit: (1) weighted CP methods are not comparable to WCTMs; rather, any WCP method offers a different opportunity to construct a WCTM on top of it. Ie, our expts are roughly construct WCTMs on top of WCP methods similar to [1]; in future work it would be valuable to consider WCTMs constructed for continual drift [2] and label shift [3]. We are happy to add comparisons to a subset of (2) and (3) in a camera-ready version, and we will discuss all of these more thoroughly in Related Work. In particular, [4-6] appear to primarily differ from our CTM baseline by betting function and ensembling (which can also be done with WCTMs). [7] is similar to us in the goal of online adaptation, although they focus on adapting the point prediction (while we adapt the weights); we have added more discussion of this. [8] and [9] could be compared to, although the CTM references are likely more closely related.

Other Weaknesses: We have updated sec 3.3 subheading to “Main Practical Testing Objective.”

Other: We are carefully revising to address typos, to improve clarity and readability. See response #1 to reviewer b4ce re contributions outline. $[0,1]^*$ is notation for a vector of any length with entries in [0,1], we will clarify. We meant Eq 7 is the definition for any $t$, we can update if unclear.

Eq 13 Q: Should mean a shift to some $\hat{F}_X^T$ such that the density-ratio is $\hat{f}_X^T/f_X$. So, $f_X$ does not need to be known, only an estimate of the ratio is needed. We will update notation here.

Thank you for your time and feedback! We refer to the following anonymous link for supplemental figures and algorithms: https://sites.google.com/view/authorresponse/home

(1) Clarifying writing, especially novelty of contributions relative to Vovk et al: Regarding your comment on how “writing can be improved,” we are actively revising our paper for clarity. For example, re your comment on clarifying “innovations as compared with the previous literature,” we have revised the last part of our Introduction section to the following (this references a slightly revised Fig 1, available at the provided link):

Recent advances in anytime-valid inference (Ramdas et al., 2023) and especially conformal test martingales (CTMs) (Volkhonskiy et al., 2017; Vovk, 2021) offer promising tools for AI monitoring with sequential, nonparametric guarantees. However, existing CTM monitoring methods (e.g., Vovk et al., (2021)) all rely on some form of exchangeability (e.g., IID) assumption in their null hypotheses---informally, meaning that the data distribution is the same across time or data batches---and as a result, standard CTMs can raise unnecessary alarms even when a shift is mild or benign (e.g., Figure 1a). Meanwhile, existing comparable monitoring methods for directly tracking the risk of a deployed AI (e.g., Podkopaev and Ramdas (2021b)) tend to be less efficient than CTMs regarding their computational complexity, data usage, and/or speed in detecting harmful shifts (Sec. 4.3).

Our paper's contributions can be summarized as follows:

• Our main theoretical contribution is to propose weighted-conformal test martingales (WCTMs), constructed from weighted-conformal $p$-values, which generalize their standard conformal precursors. WCTMs lay a theoretical foundation for online testing of a broad class of null hypotheses beyond exchangeability, including shifts that one aims to model and adapt to.

• For practical applications, we propose WATCH: Weighted Adaptive Testing for Changepoint Hypotheses, a framework for AI monitoring using WCTMs. WATCH continuously adapts to mild or benign covariate shifts (e.g., Figure 1a) to maintain end-user safety and utility (and avoid unnecessary alarms), while quickly detecting harmful shifts (e.g., Figure 1b & c) and enabling root-cause analysis.

(2) Additional Explanatory Ablation Experiments on Covariate Shift Magnitude: To address your comment that our distinction between mild and extreme covariate shifts are somewhat “vague,” we have added additional ablation experiments to illustrate how WATCH performs with different magnitudes of covariate shift--please see Supplement S1.1 at the provided link. Ie, this ablation experiment empirically clarifies that covariate shifts can indeed be viewed as a spectrum from benign to harmful rather than as a binary category. For the monitoring purpose that our paper focuses on, however, we consider “benign” shifts to be those that can be safely ignored without sacrificing system safety (i.e., coverage) or system informativeness/utility (i.e., prediction set sharpness/interval width); our WCTM methods can thus be understood as performing a flexible (nonparametric) statistical test for whether a given shift is benign or harmful to the system, in this sense, with sequential false-discovery guarantees given by our reported theoretical results.

(3) Added Pseudocode, Highlighting How Algorithm Treats Benign vs. Harmful Shifts: We are adding comprehensive pseudocode for all proposed algorithms as a new appendix section (Appendix E). For now, we have provided the annotated pseudocode for the specific sub-module of our methods that responds slightly differently to mild versus extreme shifts in $X$--please see Supplement S.1.2 for this pseudocode at the provided link. Specifically, the pseudocode highlights one way that our methods penalize (and thereby more quickly detect) covariate shifts that are so extreme (i.e., out-of-support or very little support) that the corresponding weighted conformal interval becomes noninformative (i.e., predicting the whole label space, $\hat{C}(X_{n+1})=\mathcal{Y}$ ).

(4) Spillover responses for Reviewer 7BRr (due to char lim) -- Methods And Evaluation Criteria:

• Widths shrunk in Fig 2: Due to the density-ratio estimator appropriately placing more weight on cal points with smaller scores--no further correction.

• Ablations on betting function: In S2.3 at the provided link, we compare the Composite Jumper betting function to the Simple Jumper betting function used in baselines (eg, Vovk et al).

• Classification into benign and severe shifts: See responses #2 & #3 to reviewer b4ce. Regarding a “target risk score,” note that (W)CTMs test for IID uniformity on the $p$-values; IID uniform $p$-values implies valid coverage at any significance level, which is more powerful than for a threshold. So, we argue our def is at least as principled. See sec 4.2 at the link provided for coverage results.

Thank you for your time and feedback! Please find our responses to your questions/comments roughly in order below. We refer to the following anonymous link for supplemental figures and algorithms: https://sites.google.com/view/authorresponse/home

Claims and Evidence:

• Novelty of weighted-conformal $p$-values: The general weighted-conformal $p$-values presented in Section 3 are novel in this paper. References [1] and [2] have a similar setup based on a weighted empirical distribution of nonconformity scores, but they only consider taking quantiles on the weighted distribution to compute prediction sets, they do not define, introduce, or mention weighted-conformal $p$-values nor any equivalent quantity, and they do not discuss how weighted conformal ideas can be used for hypothesis testing or monitoring (the focus of our paper).

• References for concept shift: Concept shift--a shift in $Y|X$, the label distribution conditional on the input--is a common term and topic of study in the ML robustness literature. The earliest refs we could find discussing the idea of concept shift are the following--we have added these to the paper: (a) Webb, G. I., & Ting, K. M. (2005). On the application of ROC analysis to predict classification performance under varying class distributions. Machine learning, 58, 25-32; (b) Fawcett, T., & Flach, P. A. (2005). A response to Webb and Ting’s on the application of ROC analysis to predict classification performance under varying class distributions. Machine Learning, 58, 33-38. While prior CTM methods are also capable of detecting concept shift, our main novel contribution regarding concept shift is to enable valid continual monitoring for concept shift even when covariate shift is also possible, i.e., by enabling diagnosis between the two, and demonstrating fast detection empirically. At the link we have provided above, we have added additional ablation experiments on the simple synthetic data to illustrate how this behavior.

Methods And Evaluation Criteria:

• Eval on larger models and datasets: The theoretical guarantees of our methods make no assumptions on the ML model architecture or on the modality/complexity/size of the dataset, so yes in theory we could demonstrate similar performance with ViT on ImageNet. Our paper focuses on laying a theoretical and algorithmic foundation for WCTM-based monitoring methods, while opening the door to future evaluation on frontier and foundation models.

• Other types of shifts: Yes, there are other types of shifts that can be studied. One example is “label shift,” which is a shift in the marginal $Y$ distribution (but not in $Y|X$); another example is slow distribution drift over time (rather than at a single changepoint). We have added discussion of WCTMs for these settings to our Conclusion.

Other Strengths And Weaknesses:

• Practical use in large-scale ML system: In large-scale ML deployments, the proposed WATCH framework could be used to monitor the performance of the system to detect harmful behavior patterns (e.g., increases in LLM hallucinations, jailbreaking attacks, toxic prompts/generated text), while also adapting online to more benign shifts (e.g., seasonality, user trends, etc). For instance, WCTMs could be run to monitor the distribution of LLM outputs while serving as a “safety filter” to provide probabilistic guarantees on the safety of the output shown to users. The massive computational cost of training large models paired with their wide-reaching effects makes the task of monitoring crucial, to quickly catch unsafe behavior while also minimizing costly retraining.

• Improving model dynamically at deployment time: WATCH implements one approach to dynamically improving the model at deployment time, that is by performing online density-ratio estimation to maintain prediction safety (coverage) and informativeness (widths). Root-cause analysis can also inform other approaches to improving performance at deployment time, eg, via distributionally robust training (to be robust to detected shifts).

Spillover from Reviewer 7BRr (char lim)--Methods & Evaluation Criteria:

• Specific baseline e-process: We compared to the betting-based $e$-process in Podkopaev & Ramdas (2021) (from Waudby-Smith & Ramdas (2024)) since those methods performed best and to standardize. We compared WCTMs to sequential testing variant with standardized anytime-false alarm rate of 0.01; for SR-WCTMs we compared vs changepoint detection variant with common average-run-length (under the null) of 20,000.

• Runtime metric: Runtime can be surprisingly nontrivial for monitoring long time-horizons. Ie, the changepoint baseline in Podkopaev & Ramdas (2021) has $O(t^2)$ complexity (due to starting a new sequential test at each time $t$); our comparable SR-WCTM is only $O(t)$.

• Coverage vs alarms: In our Appendix, we plan to add coverage & width plots for all expts in our paper; some are available already at the link.