What should an AI assessor optimise for?

We appreciate the comments from the reviewer. We now address the key points and questions:

W1. Only regression and only tabular data

We address the focus on regression and tabular data in Q1.

Q1. Given the generality of the question, why are we focusing on tabular data?

We chose tabular data for its simplicity in illustrating our framework, which can indeed be applied to any type of data, and also because it allows for a higher diversity of methods to treat the data and also for methods to build the assessors. For images, audio or text, for instance, almost every minimally effective technique is based on deep learning or is converted into a tabular representation through embeddings.

We chose regression because it can suffer from the problem of outliers in the base prediction, a phenomenon that does not occur in problems that are evaluated with a binary or discrete metric (correct/incorrect, safe/unsafe). In many other areas, such as machine translation, summarisation, readability, etc., metrics are quantitative, and regression problems were then more appropriate to explore these quantitative scenarios.

Q2. I don't understand what Figure 1 is intended to illustrate. On the picture there are "target assessor" and "proxy assessor" but the caption text doesn't clearly define these. It would be helpful to see an example that illustrates the notion of better that the authors appear to be aiming for? Also, when might this be true?

Fig. 1 shows a possible use case for this work. Assume we have an energy consumption prediction model $M_{1}$ that we want to assess (i.e., predict in advanced the squared error of the model predictions, so we can act in accordance ㅡfor example, by ignoring $M_{1}$ and using a more powerful, albeit more expensive, model $M_{2}$ or to alert the user that the following prediction has a very large error and should be taken with caution). The usual way of doing this is by building an external predictor, called assessor, that tries to predict the squared error (in the figure, that would be the “target assessor”). We propose using another assessor (that would be the “proxy assessor” in the figure) that, instead of predicting the squared error directly, predicts another metric (let’s say, the logistic error) and then applying a transformation.

Results show that, for instance, using a proxy assessor where the proxy metric is the unsigned logistic loss $L_{L}^{+}$ is better to predict the squared error $L_{2}^{+}$ base models make when predicting the Unified Parkinson's Disease Rating Scale (UPDRS), among others, than using a target assessor that predicts the squared error directly.

Q3. Why are the six losses chosen the right ones?

We chose these three loss functions (and their unsigned counterparts) because there is a monotonic transformation between all of them, and because they are frequently used in regression tasks. Other common losses, such as some logarithmic error in the predicted/actual quotient, were excluded because they are not based on the difference between the predicted and actual values. Still, other potential losses that deviate significantly from these monotonic relationships may be candidates for future research.

Q4. If we are interested in AI broadly, why are we using decision trees, Random forests, etc for parameter optimization?

We are interested in AI broadly, and ML more broadly than just deep learning. We used tabular data for simplicity. Then, the choice of tree-based models stems from their proven effectiveness in dealing effectively with tabular data. While we did explore neural networks, we found that XGBoost and similar methods often outperformed them with tabular input (this is consistent with https://doi.org/10.1016/j.inffus.2021.11.011). We wanted to show that our results generalise beyond simple scaling variables such as number of parameters or FLOPS. In fact, one of the main contributions of our work is that we extract scaling variables such as computing time that work across very different families of machine learning methods, and are predictive about performance. This brings generalisability to situations where other factors, such as deployment compute, may be relevant, and an independent (and non-linear) assessor model can replace a simple scaling law (https://aclanthology.org/2024.scalellm-1.1/).

(Given the character limit, we address the last question in the following comment)

We appreciate the review. We now address some of the points mentioned:

W1. Only tree-base models, not neural networks

The choice of tree-based models stems from their proven effectiveness in dealing effectively with tabular data. While we did explore neural networks, we found that XGBoost and similar methods often outperformed them with tabular input (this is consistent with https://doi.org/10.1016/j.inffus.2021.11.011). We wanted to show that our results generalise beyond simple scaling variables such as number of parameters or FLOPS, which are well-known in deep learning but not in other machine learning methods. Actually, one of the main contributions of our work is that we extract scaling variables such as computing time that work across very different families of machine learning methods, and are predictive about performance. This brings generalisability to situations where other factors, such as deployment compute, may be relevant, and an independent (and non-linear) assessor model can replace a simple scaling law (https://aclanthology.org/2024.scalellm-1.1/)

W2. Given limited scope of tree models, results are not surprsising

The results are surprising in that optimising for the target metric is not optimal for the assessor, and further justifies decoupling the uncertainty or error estimation from the base model using assessors. We don’t think this finding is associated with tree models.

W3. Lack of theoretical explanation

We acknowledge the need for more theoretical explanation and aim to include this in future extensions of our paper, considering the distribution of the errors.

Q1. There are no neural network models which the assessors are applied to but neural networks are used to do assessing. Is this correct?

Yes, this is correct, one of the assessor models was a feed-forward neural network. Assessor models are machine learning models, and they do not need to resemble the base models.

Q2. There are no proofs or more formal justifications for the relationships observed. Is this correct?

We acknowledge the need for more theoretical groundwork and aim to include this in future extensions of our paper, considering the error distribution.

Q3. Could the authors replace the figures with PDFs or SVGs intead of what is currently being used (easier to zoom in on)?

We will ensure that in future versions our figures are not bitmaps.

Q4. Do the authors think that there might be different relationships in the case of assessors applied to predicting things about over-parameterised models?

Assessors are inherently designed to learn from test results, regardless of the underlying characteristics of the base models, including whether they are over-parameterised. For instance, if a model is poorly generalised and performs poorly on test data due to issues such as memorisation of the training set (because of over-parametrisation), the assessor can learn from these test results and adjust its predictions accordingly, even determining the areas where it will not generalise well. Consequently, the role of the assessor remains consistent, focusing on the observed performance without being directly influenced by the parameterisation of the base model. Of course, it may be affected by limited test data and over-parametrisation of the assessor itself.

We appreciate the reviewer’s comments. We first address the points raised and then answer the questions:

W1. Limited Metric Scope

We focused on regression metrics because they are particularly sensitive to outliers in the base predictions, a challenge that is less pronounced in problems evaluated using binary or discrete metrics (e.g., correct/incorrect, safe/unsafe). Additionally, many domains, such as machine translation, summarisation, readability, etc., rely on quantitative metrics to evaluate performance. This makes regression a more appropriate framework for investigating scenarios where outcomes are inherently numerical.

W2. Theoretical Foundation

We acknowledge the need for more theoretical groundwork, such as particular error distributions for which we can show the effect of double penalisation. We left this for future extensions of our paper.

W3. Clarification Needed on Assessor Application

We address the practical applications of assessors in Q3.

Q1. “Can the authors elaborate on the specific characteristics of the logistic error that made it the most effective metric across the experiments?”

The main characteristic of the logistic error is that, contrary to the other error functions, it is concave. Since the assessor models are optimised to minimise the mean squared error of the assessor train dataset, we believe that there is some kind of “compensation”: when using convex metrics, such as the squared error, for the base regression model. High errors are “doubly penalised”, i.e., once from the base regression error metric and another for the optimisation metric of the assessor, which is always the squared error. This makes assessors less effective at learning the error distribution, especially when there are outliers, since the focus is put on those outliers. Concave functions like the logistic error mitigate this effect (hence the “compensation”), helping assessors to achieve better results overall.

Q2. “How might the findings change if applied to more complex models or different domains? Are there limitations in the current experimental setup that could affect generalizability?”

Our findings suggest that the current setup is adaptable to different complex models and different domains, as long as the base models provide some form of prediction (be it a real value, a vector of probabilities, an action…) and a corresponding outcome metric (difference error, correct/incorrect label, Brier score, safety measure…) The distribution of the outcomes for this metric is what needs to be modelled by the assessor and appropriate transformations could be introduced in each of these cases. For instance, if what matters is a safety metric that heavily penalises some types of outcomes, we may get better results by doing a logarithmic transformation of that metric before training the assessor. A case-by-case analysis may be required but the lessons learned from this paper should be generally applicable to these other scenarios.

Q3. “It would be beneficial for the authors to provide concrete examples of how their findings could inform the design of assessor models in practice”

Our findings can inform the design of practical assessor models by using logistic error or other concave functions to improve training. These metrics help models perform well with different error distributions. For example, in healthcare, an assessor could predict recovery time estimation success by considering patient factors without overfitting for rare cases. In autonomous systems, models could predict safety outcomes by scoring errors without heavily penalising rare but important deviations.

We thank the reviewer for their comments. We respond to some of the comments first and then answer the questions:

W1. Narrow scope for only working with regression

We focused on regression metrics because they are particularly sensitive to outliers in the base predictions, a challenge that is less pronounced in problems evaluated using binary or discrete metrics (e.g., correct/incorrect, safe/unsafe). Additionally, many domains, such as machine translation, rely on quantitative metrics to evaluate performance. This makes regression a more appropriate framework for investigating scenarios where outcomes are inherently numerical.

W2. Lack of theoretical explanation

We acknowledge the need for more theoretical groundwork and aim to include this in future extensions of our paper, as well as experimenting with different optimisation algorithms.

Q1. Are the artifacts of this work going to be made available to the public (e.g. code)?

Yes, code, data employed and instance level data produced (as in compliance with https://doi.org/10.1126/science.adf6369) will be made public when accepted.

Q2. Are there any findings or insights that are specific to some of the datasets or do all results apply to all datasets individually? It would be useful to have some breakdown per datasets and see if there are any characteristics of the data that strengthen the results (or otherwise make them less applicable)

Our analysis suggests that the results are generally consistent across datasets. We will provide the dataset-level results in the Appendix.

Q3. Which optimization algorithms did you use?

All base models and assessors used default hyperparameters. Preliminary experiments using grid search to find the best hyperparameters for the assessors showed consistent results, so we decided not to optimise the assessors further. This decision was made to keep the experimental setup as simple and computational lightweight as possible (given the dozens of thousands of runs performed).