CALM: Critic Automation with Language Models

W1: My main concern with the current version is the validation through human studies. The human study is pretty small, just 3 PhD students and in addition to that, the outcomes of the study are not discussed in depth. Usually, for studies that are so small, one would want to see actual qualitative user feedback and insights, which are missing. For such qualitative studies, the common practice is also to report the background of the participants, how familiar they are with the data, the tests being used etc. Again, it is also hard to judge how significant the outcomes are here given the very small sample size. I would recommend the authors to work with a user researcher that can set up a study with at least 10-15 users and set up a rigorous qualitative evaluation. Appendix A7 has some notion of this but it is very limited.

W2: Section 3 can benefit from providing a clear architectural diagram that also describes how design decisions in CALM contribute to addressing common issues with Naive solutions. Currently, it is not clear when LLMs are used and how they interact with other parts of the system that are not LLM based.

W3: Some of the results are somewhat obvious. For example, the result on sliced statistics can also be considered as a general criticism that is quite known all around.

• generalizability to other models: the proposed framework is validated using gpt-4-turbo, a costly language model, which may compromise the applicability of the framework at scale. The paper could be further improved by showing how running the experiments using a cheaper model (e.g., gpt-4o) and/or open source models (e.g., Llama 3.1) would affect the obtained results.
• generalizability of the results: the conducted experiments are either too simple (simple synthetic regression setting with 4 variables) or include few data-model pairs (6 in Section 4.2, 36 in Section 4.3, and 10 for the human studies), raising questions about the generalizability of the proposed framework to more complex datasets.
• lack of meaningful baselines: despite mentioning various model criticism techniques in Section 2, the authors limit their comparisons to simple naive baselines. For example, the authors could compare with a chain-of-thought prompting approach.
• few insights about the generated and correctness of summary statistics: while the authors provide one example in Section 4.3, the paper could be further improved by adding additional insights and contrasting the proposed discrepancies with commonly discussed discrepancies in the literature (e.g., do these resemble the ones commonly found by humans?)

This paper is missing a clear baseline. The results may seem impressive but lack context without a baseline. Motivation seems weak for this study. This paper should expand on why automating model criticism is necessary, emphasizing the limitations of traditional approaches or the risk when critiques are inaccurate.

The paper does not fully explain how the LLM selects test statistics and tailors its critiques for different models. This is a critical aspect of the system but the process by which the LLM determines which statistics to use is only hinted at without further explanation. Although the paper does focus on hypothesis testing to avoid hallucinations, there is limited discussion on how CALM handles false positives and false negatives in practice, especially in real-world scenarios. The results from synthetic datasets are promising, but further analysis on how it handles borderline cases in complex datasets. The paper does not discuss in detail what happens when the metadata is incomplete, messy, or unstructured. The paper primarily focuses on certain types of models, such as regression and Bayesian models. These are important but lack of diversity in the models and datasets used for evaluation leaves questions about the generalizability of CALM.

I have three primary critiques of this paper that influenced my recommendation to reject.

• W1: Unclear novelty relative to related work, particularly Li et al. [1]. My primary concern is that the authors do not properly attribute, engage with, and distinguish what is novel in their proposed approach (CALM) from past work. In particular, the authors state that they adopt methods from Li et al. [1] as components of their proposed method (e.g., to generate code). When I read through this paper, I noticed that Li et al. already propose a process to critique proposed scientific models, that seems very similar to the authors' proposed process.

  • From Li et al: "in the criticism step, we ask the critic LM to produce natural language criticism of fitted models, we use this criticism to drive model revision. To enable the critic LLM to do something akin to a posterior predictive check, we obtain samples from the posterior predictive distribution and then compute summary statistics of these posterior predictive samples."
  • My understanding is that your methodology is very similar; but instead you have more flexibility for an LLM to propose exactly which summary statistics are computed?
  • The authors should update the draft to properly attribute Li et al.'s methodology and clearly distinguish how their methodology is different.

• W2: Unclear notation and exposition makes it difficult to understand the author's proposed method. Despite re-reading the main text and looking through the Appendices, I still found it difficult to understand the authors' proposed hypothesis testing method in Sections 3.1 and 3.2. I list points of confusion and make recommendations on how to better clarify your methodology below.

  • What exactly does it mean to "compute the test statistic over the model samples" to get the "null distribution"? For example, say that the test statistic is the average (dataset-level) difference in radon levels between the basement and first floor. From your notation, it looks like you will sample $m$ different datapoints, but it is unclear to me how this can create a distribution. Is the idea instead that you will sample $m$ different datasets, and then construct an empirical distribution of the value of the test statistic for those different datasets?
  • I am confused by the notation in Equation 2. I think what you're trying to say is that the empirical p-value is the proportion of test statistics (from the posterior samples) that are "more extreme" (greater than or equal to) the test statistic for the observed data.
  • I was trying to make the connection between your proposed methodology to calculate an empirical p-value and past literature on hypothesis testing using test statistics. If you're able to more explicitly distinguish what your methodology is inspired by/drawing from, and what exactly is distinct (that you're supporting customized test statistics), I'd find that helpful as a reader.

I've noted other places in the manuscript that are otherwise unclear and should be clarified by the authors:

• I recommend that you make the draft clearer early on about assumptions that are necessary in order to use your proposed method. You frame the paper as broadly being able to aid with "scientific discovery/scientific modeling", but then it turns out that it seems like you're implicitly assuming the users' data columns are tabular and column names are "semantically plausible"; that sufficient "metadata" about the dataset is available (e.g., this approach may not be appropriate for image modeling/modeling bioinformatic data).

• The title of the paper is not descriptive enough. I recommend that the authors revise the title to make clear that their contribution is on critiquing scientific models.

• As early as possible, it may be helpful to give an example to illustrate clearly what you mean when you use the terms "discrepancy" (between what?), "summary statistic", "statistical test", "null distribution" (for what null hypothesis?). I only understood what you meant by these terms when I reached Section 4.1 where you contextualized each of these terms using the radon dataset.

• I am still confused about what exactly is meant when you use the term "dataset metadata", and looking at Figure 9 did not help. Can you define precisely what you mean/what is given as input to the LLM? This is necessary information for a reader to know to use your proposed method.

• You should provide more details in your text about how exactly you used other systems. For example, exactly did you generate the model revisions from the critiques? All you do is cite that you used Li et al.'s "system", but more details should be included about how this system works. If Li's system does in fact support all of these things, why not just use it from the beginning to recommend the revisions itself (or as a more competitive baseline)?

  • For experiment 3, can you report the "win rate" for the human raters, rather than just the alignment between the human raters and LLM judges? Can you make clear if the win rates reported in Tables 1 and 2 are with LLM or human judges?

• W3: The right baseline should be a human, not another LLM. It seems to me like the LLM is relying a lot on "domain knowledge" about what features might be most important, based on their column names, to recommend test statistics. I wonder if a human would already hold this same intuition, and would come up with the same test statistics as CALM, without any LLM support. I would be interested if the LLM would result in a significant speed-up or would propose new test statistics that a human otherwise would not have thought of.