GPQA: A Graduate-Level Google-Proof Q&A Benchmark

Thank you for your thoughtful review and question! We agree that because it’s on the smaller size, resolving small differences in accuracies with statistical significance will often not be possible.

Regarding your concern about generalization: This is a fantastic question that directly engages with the core question scalable oversight poses. While ultimately we can’t be certain scalable oversight methods will generalize, we can cleverly design our experiments to help us build confidence. Specifically, we want to be able to show that non-experts can supervise experts, even when the non-experts do not have all of the knowledge or understanding of the experts. If we can show this, then the question reduces to whether there are fundamental differences between the kinds of knowledge that human experts currently have, and the kinds of knowledge that future AI systems have.

So, if we can show that non-experts are able to supervise a wide range of questions, knowledge, and expertise in different domains, then as long as the kinds of questions, knowledge, and expertise future AI systems have is similar enough to what we’ve evaluated with non-experts, then we can be confident the oversight will generalize. We don’t think this assumption is guaranteed (e.g. we could imagine systems having different internal ontologies from us that are intractable to translate), but we think it should allow us to supervise a very wide class of questions or tasks.

We also don't need to purely rely on empirical evidence—we can also learn/build theoretical justifications for how oversight will scale. For example, in the paper “AI Safety via Debate”, which introduces debate as a method for scalable oversight, a complexity theory analogy is given that demonstrates how polynomial-time verifiers (i.e. judges) can formally verify “debates” (i.e. alternating witnesses) from computationally unbounded debaters on questions in PSPACE (and later work extends this to NEXP). Ideally we can bridge the theory-empirical gap from both ends, by improving the scope of our purely theoretical models, like the complexity theory proofs, and by building empirical models of our observations of scalable oversight methods.

We hope this helps clarify how we think about scalable oversight and supervising frontier AI systems—please feel free to ask follow-up questions during the discussion period!

Thank you for your thoughtful review, and for your kind words about the quality of the questions, our construction process, and our analysis/experiments! We agree that because it’s on the smaller size, resolving small differences in accuracies with statistical significance will often not be possible.

Regarding your comment on our lack of references and comparison to existing evaluation datasets, we agree that we could provide more context and comparison in the main paper. We currently have a detailed related work section in the appendix (section A.2), but we’ll happily add this back to the main paper for the camera-ready version to help contextualize our benchmark and highlight its novelty, especially given the newly increased page limit!

Thank you for your hard work on the paper, and the rebuttal! Comments by the authors addressed my 3rd concern, and I understood that the authors mentioned the small size in Limitation section. So I'll raise my score.

Thank you for your generous review! We appreciate your words regarding the novel nature of the dataset, and the quality and rigor of our validation process. We agree that because it’s on the smaller size, resolving small differences in accuracies with statistical significance will often not be possible.

Thank you for your kind words about the quality of the questions, and for your thoughtful and engaged review! Responding to your comments/questions:

• Multiple choice vs. free-response: To try and make the questions usable in a free-response setting, we instructed all question writers to format their questions such that they can be answered by an expert who doesn’t see the answer choices—while we didn’t enforce this strictly, it was an explicit prompt that expert validators were instructed to review, and anecdotally many of the questions seem to be structured properly regarding this.
• Question writers gaming the rules: I agree this is in principle a possibility—qualitatively it doesn’t appear that the questions are written in this way, but I’d be curious to hear of strategies that could be used here!
• In-domain expert validator accuracy w/ 30mins and internet access: Expert validators were actually allowed to access the internet, and they were allowed to spend as much time as they wanted answering the questions. However, on average they spent less than 30 minutes by choice, likely informed by their judgement of the expected return of spending more time on the questions.
• Ablations for incentives: This is a great question—we’re also very interested in understanding better how incentives affect behavior. Unfortunately, we’d expect these experiments to be quite expensive, and given the costly nature of this project, we ultimately decided to use the payment structure we felt from previous data collection projects would be most likely to produce the best results.
• Answering with help from LLMs: We completely agree—this is one of the directions we’re most excited to see work using GPQA focus on, particularly as it relates to scalable oversight!