Sequential Harmful Shift Detection Without Labels

We thank the reviewer for taking the time to review our paper.

We appreciate the suggestion to include a dedicated section on related work concerning distribution shifts and continuous production environments. Regarding the question of whether "requires no access to ground truth data labels" is a worthwhile background setting to study, we believe it is highly relevant. In many real-world environments, ground truth labels are either unavailable, delayed, or expensive to obtain. Examples include:

• Healthcare: (Raravian et al., 2015, Population-level prediction of type 2 diabetes from claims data and analysis of risk factors)
• Insurance: (Noodhun et al., 2018, Risk prediction in the life insurance industry using supervised learning algorithms)
• Predicting Future Outcomes: (Zhang et al., 2019, Time-aware adversarial networks for adapting disease progression modeling)
• High Collection Costs: (Lieu et al., 2022, Deep unsupervised domain adaptation: A review of recent advances and perspectives)

Regarding the remarks about our method, it is not solely about learning an error estimator. The cornerstone of our method is the calibration step that leverages even an imperfect estimator to achieve performance comparable to methods that have access to ground truth labels. This is particularly useful since it is often unrealistic to expect a highly accurate error estimator in many practical scenarios.

We thank the reviewer for taking the time to review our paper and acknowledging our contributions.

The claims regarding the properties of “traditional” drift detection methods are unfounded and can be seen as misinformation.

Regarding the issue raised about traditional shift methods (Lines 17-19), specifically their inability to distinguish harmful from benign shifts and their inflated false alarm rates, our assertions are based on the findings of Podkopaev and Ramdas (2022). Their analysis in Appendix A ("Issues with existing tests for distribution shifts/drifts") of their paper supports our claims. We will cite this appendix directly in the final version of our paper.

Phrases like “If sufficient”, “Should align”, “would achieve” involve a lot of assumptions.

Yes, we agree. In Line 134, we were describing the naive approach of using a plug-in error estimator. This served as a prelude to introducing our strategy, which aims to achieve good performance using an imperfect proxy for model performance.

If the error estimator is a statistical model, it would open this expression up to further introspection.

We agree. Additionally, as the reviewer highlighted, our approach allows for the use of any proxy, not only error estimation. For instance, in classification problems, we can use calibrated probabilities instead of the prediction of the error estimator as a proxy before our calibration step. We have tested this approach and found that it demonstrated similar performance to the error estimator.

The FDP plot has no blue in it; what does this sentence refer to?

The variance of the distribution was so small that it appeared as a line, making the blue colour not visible as in the production data case. We will clarify this in the final version.

Why not go from Equations 1 and 2 directly to Assumption 4.1?

We will simplify the presentation by going directly from Equations 1 and 2 to Assumption 4.1.

Is "learnt" the old variation of "learned" or is there a specialized meaning to it?

Yes, it is the British variant, and they both have the same meaning.

Does the predictor for the model error serve as an approximation of the conditional probability to produce a higher error in areas of the feature space with sparse or contradictory behavior? Would the representation of this not be best captured by a Gaussian Process model?

Yes, exactly. It is an implicit way of defining the regions where the model performs poorly. Particularly in the case of tabular data, we have considered tree-based models as they are well-known to work best. In addition, it opens the possibility of examining the leaves of the tree to define hyper-rectangular region. These regions can be represented as rules, providing valuable insights to users about which covariate spaces cause the model to perform poorly.

You do not mention the model you used to produce your results. (Or do you always use the same model for the main prediction and the error prediction, if yes, why?)

We use the same class of model and hyper-parameters by default to avoid biasing the results by explicitly fine-tuning the error estimator. However, the objective is to find the best possible error estimator, and users can choose any model that suits their problems.

In which domain do X and Y reside?

In our experiments, the domains of X and Y have been $\mathbb{R}^p$ and $\mathbb{R}$, with $p$ being the number of variables. The continuity or discreteness of the data is not important. It can be anything, as long as a bounded score value is attributed to each combination of prediction and true labels.

Which type of R² score is used here? What is the reference model? How is the R² score used in the production scenario, as it is batched based the application on sequential data points is not natively defined. Against your assumption in section 2, the R² score is not bounded! but be always smaller than 1. R² is only bounded if it is viewed as a data descriptor that assumes that a linear model is used to describe the data, then it is a measure for the explainable variance. This specification confuses more than it help. The R² formulation you use should be clearly specified in mathematical notation.

We use the classical R² score from the scikit-learn library, defined as $R²(y, \hat{y}) = 1 - \frac{\sum (y - \hat{y})^2}{\sum (y - \bar{y})^2}$. However, this is not what we monitor in our experiments. Throughout the paper, we only use the R² score to describe the general performance of the error estimator. In production, the loss we monitor and update sequentially is the absolute residual $\ell(X, Y) = |y - \hat{y}|$. We apologize for the confusion and will make it clearer that the monitored loss function is the absolute distance $\ell(X, Y)$, computed with an output Y normalized between (0, 1).

If the Hoeffding interval is based on the Hoeffding Inequality, does your application fulfill all stated requirements?

For both confidence sequences and intervals, the only assumption we need is the boundedness of the loss. In all our experiments, we ensure the score is bounded by using the absolute distance with bounded output.

What error metric is used for the stated experiment? Accuracy?

We use the absolute distance between the predicted probability and the label.

General Question on DDM and PHT

The main differences between these methods and ours are that they require access to ground truth labels to monitor performance, whereas our method does not.

Are you referring to joint probabilities or conditional probabilities? So, S(X)=1 and E>q or S(X)=1 given E>q?

We were referring to the joint probability.

Domain knowledge regarding performance-based drift detection methods is not mentioned.

We will include it along with the suggested references.

computed with an output Y normalized between (0, 1).

The claims regarding the properties of “traditional” drift detection methods

In addition, the use of a reference window is presented as a weakness, but the proposed method also uses a calibration set, which can be seen as a reference window.

You do not mention the model you used to produce your results.

General Question on DDM and PHT

Thank you for your response. Please find below our answers to the remaining points you raised:

Please verify that this does not violate the independence assumptions of Hoeffding's Inequality. Based on my experience, having attempted to use Hoeffding's Inequality in regression for non-stationary environments, I can confirm that Hoeffding's Inequality requires both bounded and INDEPENDENT random variables. By normalizing your data targets, you may introduce interdependencies among them, which could be problematic.

You’re correct that Hoeffding’s Inequality, as well as the inequalities used for confidence sequences, require both bounded and independent random variables. Regarding the normalization, we are only rescaling the output before constructing or splitting the data into production, calibration, and training sets using min-max scaling with the same minimum and maximum values. To the best of our knowledge, this rescaling process should not violate the assumption of independence.

I agree with your point, but your reference specifically mentions tests based on distributions. There is an entire class of performance-based drift detectors that do not have these issues but do require target labels. Please clarify this distinction in your discussion.

In the second paragraph of the Introduction (L 25-32), where we describe methods that might detect harmful shifts without labels, we will also include techniques that do require labels in the final version of the paper, including the ones you suggested.

In addition, the use of a reference window is presented as a weakness, but the proposed method also uses a calibration set, which can be seen as a reference window.

We agree that the calibration set can be seen as a window, but our comment was about inference time. Our method, being completely sequential, does not require defining a window for the production sample size, as it processes the data as it arrives, one by one. In Lines 17-20, we will specify that the prespecified sample set refers to the production set.

You do not mention the model you used to produce your results. While you mentioned that the model can be changed, what specific model did you use for your evaluations? This is important for reproducibility.

For the image dataset, as stated in Lines 268-272, we used a ResNet-50 as the base model, modifying only the head for error estimation. For the tabular data, we consistently used a Random Forest regressor/classifier with the default parameters from the scikit-learn library (num_trees=100, max_depth=None, boostrap=True). We will clarify this further in the paper.

General Question on DDM and PHT. I understand that DDM and PHT require labels, while your approach does not. However, my question was about what happens afterward. After using your proxy model to predict the error, you still need to determine whether there is a drift. This decision-making process seemed quite similar to DDM to me. Could you elaborate on the differences, if any, or clarify if your method was inspired by DDM?

Our paper is not inspired by DDM but by the work of Ramdas et al. (2022). While there may be some similarities in the decision-making process, as DDM computes an approximate confidence interval in the reference set (called the concept set) and compares it with a threshold value, there is a substantial difference. DDM is designed for contexts where the model also changes dynamically. Unlike DDM, the method proposed by Ramdas et al. (and by extension, our methods) uses a calibration step and defines the warning threshold based on statistical tools that provide provable false alarm guarantees in finite samples with minimal assumptions, such as independence and bounded loss.

We thank the reviewer for taking the time to review our paper and for providing valuable references that we were not aware of. We appreciate the opportunity to clarify our contributions in light of these references.

First, we note that paper [2] focuses on detecting distribution shifts in general and not specifically harmful shifts, unlike our method and the first suggested paper [1]. Consequently, using [2] as a benchmark in cases where there are harmful/benign shifts would be unfair since it does not differentiate between harmful and benign shifts and would likely result in a much higher rate of false alarms. Therefore, the first suggested paper is more relevant to our work as it directly targets harmful shifts.

However, we must highlight that comparing our method directly with those in [1] is challenging due to fundamental differences in setup. The methods in [1] are designed for an offline setup, requiring a batch of production data to train their statistics and raise alarms for a batch of production data. In contrast, our methodology was designed for an online setup where shifts occur gradually and continuously, necessitating real-time decisions without observing an unlabelled batch of examples first. Our methodology aims to detect harmful performance shifts on the fly as observations are received one by one and do not require access to the full production data to fit the statistic.

Additionally, applying offline methods in an online setting would be impractical and unfair, as these methods require learning a model or statistics from a batch of data. It would also be computationally expensive, as [1] requires learning a new model for each batch of production data. In this case, we would have to learn a number of models proportional to the size of the production data.

However, we evaluated our method against the best method from [1] (Detectron) in a batch setting, increasing the size of production data or out-of-distribution (OOD) data. We generated the shifts similar to the setting described in Section 5.2 and ensured no shift in the first 1300 samples of the production data. We used NHANESI classification data as Detectron is only available for classification tasks. We replicated the experiments 50 times, resulting in a total of 10200 different shifts.

Our results, summarized in the table below, show that for smaller sample sizes (100 and 1000), our method did not detect any shifts, which is expected since no shift occurred in the initial 1300 samples, while Detectron raised a significant number of alarms (1126 out of 1700 shifts for size 100 and 1493 for size 1000), all of which were false alarms. For larger sample sizes, while the method of [1] shows high power in detecting shifts, it also exhibits high false alarms. In contrast, our method demonstrates lower power but significantly better FDP control, aligning with our goal of minimizing false alarms.

These results validate that our method performs robustly in a batch setting, as well as working effectively in the online setting considered in the paper.

We hope this clarifies the distinctions and the rationale behind our methodological choices. We will also clarify these differences and include additional experiments demonstrating them in the final version of the paper. Thank you again for your insightful feedback.

Thank you for your response.

As far as I understand, in a practical sense, the main difference is that one method is designed for online settings while the other is designed for offline settings. Is that true? I did not understand if your method is designed to detect harmful shifts of X. Is that the case?

Yes, the major difference is that our method is designed for online use, while the other is for offline. Regarding whether our method is designed to detect harmful "shifts of X" (i.e. covariate shifts), our central assumption is Assumption 4.1, i.e. that the selector function generalizes in terms of FDP from calibration to production. This is more likely to hold in the case of pure covariate shift, where the relationship Y | X is invariant. However, it may still be effective if this relationship changes, provided the model tends to have high error in the same regions of the input space before and after the shift. Note that in our empirical experiments, we study shifts in natural datasets, where the Y | X relationship is very likely to be at least somewhat non-invariant. Our method is shown be effective in these cases.

Do you know why the Detectron has such a bad FDP in your experiment?

Detectron’s main idea is to learn a Disagreement Classifier that performs as well as the original model on the training distribution while disagreeing with the original/base model's predictions on the production data. This method is highly sensitive to the performance of the base classifier, the function class used, and the size and nature of the production data. While it effectively detects harmful shifts (as shown by its high power in our experiments), it may fail when the shift is benign.

Below is a simple code example that generates a plot illustrating a failure case. In this example, the training data points is represented in red and green, with the ground truth shaded in corresponding colors. The base model is shown as a solid black line. For simplicity, we assume the model is a perfect classifier. The data has shifted to the right, creating unlabeled production data, all correctly classified by the base classifier.

We’ve also depicted the potential learnable classifier as a dashed line, representing the boundary of all possible functions, which depends on the model type, complexity used for the disagreement classifier, and the nature and size of the data. We have shown a potential disagreement classifier in orange that performs similarly to the original model on training data but disagrees on the predictions of the base classifier in the production data. As shown, even with a benign shift, we can still find a disagreement classifier that performs well on training data but disagrees significantly in production, raising a false alarm.

import matplotlib.pyplot as plt
import numpy as np

# Generating sample data
x = np.linspace(-3, 3, 500)
base_classifier = 2 * np.sin(x) 
shaded_area = 2 * np.sin(x)
boundary_upper = 5 * np.sin(x) 
boundary_lower = 0.5 * np.sin(x)
disagreement_classifier = 3.5 * np.sin(x) * (x <= 0) + 4.5 * np.sin(x) * (x > 0)

np.random.seed(0)
x_class1 = np.random.uniform(-2, -1, 10)  
y_class1 = np.random.uniform(-2, 1, 10) + 3

x_class2 = np.random.uniform(-2, -1, 10)
y_class2 = np.random.uniform(-5, -2, 10) - 3

x_class1_shifted = np.random.uniform(-2, -1, 20) + 3
y_class1_shifted = np.random.uniform(-2, 2, 20) + 4.5
x_class2_shifted = np.random.uniform(-2, -1, 10) + 3
y_class2_shifted = np.random.uniform(-5, -2, 10) + 2

x_unlabelled = np.concatenate([x_class1_shifted, x_class2_shifted])
y_unlabelled = np.concatenate([y_class1_shifted, y_class2_shifted])

# Plotting
plt.figure(dpi=200, figsize=(8, 5))

plt.fill_between(x, shaded_area, 12, color='green', alpha=0.1, edgecolor='none', label='Ground Truth (Class 1)')
plt.fill_between(x, -12, shaded_area, color='red', alpha=0.1, edgecolor='none', label='Ground Truth (Class 2)')

plt.scatter(x_class1, y_class1, c='green', marker='+', label='Training Data (Class 1)')
plt.scatter(x_class2, y_class2, c='red', marker='+', label='Training Data (Class 2)')

plt.scatter(x_unlabelled, y_unlabelled, facecolors='none', edgecolors='black', label='Unlabelled Production Data')

plt.plot(x, boundary_upper, 'gray', linestyle='--', label='Set of Potential Learnable Classifiers')
plt.plot(x, boundary_lower, 'gray', linestyle='--')
plt.plot(x, base_classifier, 'k-', label='Base Classifier')
plt.plot(x, disagreement_classifier, 'tab:orange', linestyle='-', label='Disagreement Classifier')

plt.legend(loc='upper left', bbox_to_anchor=(1, 1), fontsize='small', frameon=False)
plt.grid(False)
plt.tight_layout()
plt.axis('off')
plt.show()

We thank the reviewer for taking the time to review and acknowledge our work.

In the experiments section, the authors only compare with a baseline scheme and ignore other existing distribution shift detection algorithms (e.g., [1-2]), although they may not claim to only detect harmful distribution shifts.

Regarding the suggested paper, neither would be a suitable baseline for our method as they either attempt to detect shifts in general or require access to labels to assess model performance. Comparing them with our method would be unfair since they are likely to have high false alarm rates in a context where there are both harmful and benign shifts. Moreover, the primary purpose of our methodology is to detect harmful performance shifts without using labels.

However, the closest baseline to ours is the one suggested by Reviewer KyAy, although it is designed for batch or offline contexts rather than the sequential or online context of our work. We provide comparisons (please refer to our response to Reviewer KyAy) showing that our method also works in the batch context.

However, there is no free lunch. Despite these excellent properties, does the method will face any other challenges?

Regarding the limitations, we acknowledge that our method may perform less effectively when there is a significant concept shift or when we do not have enough calibration data to train and calibrate our proxy. We will include this consideration in the final version of the paper.

In lines 117-118, could the authors explain how they connect the upper bound of the training dataset to the lower bound of the test dataset to derive this probability inequality?

Below, you will find the detailed derivation of Equation 7 (Lines 117-118):

P_{H_0}\left $ \exists t \geq 1: \Phi_m(E_1, \dots, E_t) = 1 \right $ = P_{H_0}\left $ \exists t \geq 1: \hat{L}(E_1, \ldots, E_t) > \hat{U}(E_1^{0}, ..., E_1^{0})+ \epsilon_{\text{tol}}\right $ = P_{H_0}\left $ \exists t \geq 1: \left(\hat{L}(E_1, \ldots, E_t) -\frac{1}{t} \sum_{k=1}^t \theta(P_{E}^{k})\right) - \left(\hat{U}(E_1^{0}, \ldots, E_n^{0})-\theta(P_{E}^{0})\right) > \epsilon_{\text{tol}} - \left( \frac{1}{t} \sum_{k=1}^t \theta(P_{E}^{k}) - \theta(P_{E}^{0}) \right) \right $ \leq P_{H_0}\left $ \exists t \geq 1: \left(\hat{L}(E_1, \ldots, E_t) -\frac{1}{t} \sum_{k=1}^t \theta(P_{E}^{k})\right) - \left(\hat{U}(E_1^{0}, \ldots, E_n^{0})-\theta(P_{E}^{0})\right) > 0 \right $ \leq P_{H_0}\left $ \exists t \geq 1: \left(\hat{L}(E_1, \ldots, E_t) -\frac{1}{t} \sum_{k=1}^t \theta(P_{E}^{k})\right) > 0 \right $ + P_{H_0}\left $ \left(\hat{U}(E_1^{0}, \ldots, E_n^{0})-\theta(P_{E}^{0})\right) < 0 \right $ $$\leq \alpha_{\text{source}} + \alpha_{\text{prod}}$$

The third last line is because we are under $H_0$ and the second last line is due to if $\exists t \geq 1: \left(\hat{L}(E_1, \ldots, E_t) -\frac{1}{t} \sum_{k=1}^t \theta(P_{E}^{k})\right) - \left(\hat{U}(E_1^{0}, \ldots, E_n^{0})-\theta(P_{E}^{0})\right) > 0$, then either $\exists t \geq 1: \hat{L}(E_1, \ldots, E_t) -\frac{1}{t} \sum_{k=1}^t \theta(P_{E}^{k}) > 0$ or $\hat{U}(E_1^{0}, \ldots, E_n^{0})-\theta(P_{E}^{0}) < 0$