Omni-MATH: A Universal Olympiad Level Mathematic Benchmark for Large Language Models

Weakness:

Thank you for your thoughtful feedback. We will address your concerns as follows:

1. Verifier Training Details: The training details of the verifier are already discussed in lines L285-L294 of the paper. Additionally, we provide a more in-depth description of the training process, including comparisons of different base models used as verifiers, in Appendix I. If you feel that any additional specifics are needed, please let us know, and we will be happy to provide further clarification.

2. Scaling Test-Time Inference: We appreciate your suggestion regarding the use of more advanced search algorithms, such as Monte Carlo Tree Search (MCTS) or tree search, for tackling more challenging competition-level problems. While these techniques are indeed widely used in the field, our current evaluation is limited by computational resources, which prevents us from incorporating MCTS at this stage. However, we plan to continue monitoring developments in the open-source community and will update our work to reflect any relevant advancements in search algorithms or inference techniques.

We hope these clarifications address your concerns, and we look forward to your further feedback.

Thanks for your detailed feedback. Below, we address each of your concerns:

1. Evaluation Details: We appreciate your concerns regarding the evaluation process. We will provide additional details in the appendix to clarify the evaluation procedure. The 100 samples mentioned in the text were derived from the inference results of DeepSeek-Coder-V2, GPT-4o, and Qwen2.5-MATH, sampled according to difficulty to ensure that all levels of difficulty were represented. For annotation, we hired two PhD and two Master's students, with each annotator labeling 50 problems. During individual annotation, 3 instances of inconsistent results were observed. However, after discussion, we reached a consensus on the answers. Among 100 cases, there are 2 failed cases of GPT-4o. The failures were mainly due to GPT-4o misinterpreting the “Student Answer” as the “Final Answer” under the current prompt, which we attribute to prompt formatting. This issue was resolved by adjusting the few-shot prompt format.

2. Contradiction between GPT-4o and OmniJudge Results: We understand your concern regarding the discrepancy between the alignment rates of GPT-4o and OmniJudge. We believe there may have been a misunderstanding. The reported 91% consistency was obtained using the test set described in lines L288-L289, which includes a larger number of samples, thus ensuring greater reliability. As shown in Table 6 (Appendix), we used 2,220 samples to automatically evaluate the consistency between GPT-4o and OmniJudge, while human evaluation was performed on a randomly selected set of 100 samples. In this evaluation, GPT-4o aligned with human judgments at 98%, and OmniJudge at 86%. It is important to note that these are not directly comparable through simple multiplication, as they were derived from different test sets. We hope this clears up the misunderstanding.

3. Rule-Based Evaluation: In response to your suggestion regarding the use of rule-based systems for evaluation, we addressed this concern in Section 2.4. We argue that applying a rule-based approach to the Olympic-level data is nearly impossible, as such rules lack generalizability. Even if one were to devise specific rules, they would only apply to a narrow set of cases, excluding more complex answer formats. Removing such cases would contradict the primary goal of our dataset design. Specific examples can be found in Figure 6.

4. Data Leakage: We have already addressed the issue of potential data leakage in Section 4.1. Our tests revealed minor data leakage, but we found that it does not affect model evaluation. We believe that the impact of data leakage has two main aspects: (1) the authenticity of current experimental results and (2) the potential effect on the training of future models. For the first aspect, we have confirmed that the leakage does not meaningfully impact our experimental conclusions. For the second aspect, we believe that addressing contamination in training data should be the responsibility of model developers, not evaluation datasets.

Questions:

1. Cross-validation refers to assessing the consistency of annotations by multiple annotators. The principles for annotation is aligned with Appendix C1, and include checking the completeness of both problems and answers (e.g., verifying whether a question is a proof question, and whether the answer forms a complete chain of reasoning).
2. "Prop." refers to the proportion of leaked questions out of a total of 4,428 items, with Qwen-2.5-MATH having the highest leakage rate at 0.70%. "Avg.D." denotes the average difficulty of the leaked questions. We have provided specific leakage figures in Appendix B, where we also discuss other findings, such as the fact that leaked questions tend to have lower average difficulty when correctly answered.
3. See response to Weakness "Evaluation Details".
4. Regarding few-shot prompts, we followed prior work in this area. The prompt details will be included in the supplementary materials. We hope this provides sufficient clarification and addresses your concerns.

"Regarding few-shot prompts, we followed prior work in this area. The prompt details will be included in the supplementary materials. We hope this provides sufficient clarification and addresses your concerns."

It seems you use few-shot prompts. I believe for chat/instruct models, a common practice is to use zero-shot prompting [4, 5, 6, 7]. Few-shot prompts are used for evaluation base/L0 models.

From the rebuttal, I will decrease your presentation to 2, as I find that the writing is really confusing for me. I will not decrease the overall score for now (although in my point of veiw, this paper deserves a score slightly higher than 3 (much lower than 5)).

By the way, I believe it is OK to extract answers from LLM outputs using GPT4 (like in [8]). It it a much simpler task, which is quite different from using GPT4 to judge the answer correctness.

[1] Panickssery, A., Bowman, S. R., & Feng, S. (2024). Llm evaluators recognize and favor their own generations. arXiv preprint arXiv:2404.13076.

[2] Ye, J., Wang, Y., Huang, Y., Chen, D., Zhang, Q., Moniz, N., ... & Zhang, X. (2024). Justice or prejudice? quantifying biases in llm-as-a-judge. arXiv preprint arXiv:2410.02736.

[3] Wang, P., Li, L., Chen, L., Cai, Z., Zhu, D., Lin, B., ... & Sui, Z. (2023). Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926.

[4] He, C., Luo, R., Bai, Y., Hu, S., Thai, Z. L., Shen, J., ... & Sun, M. (2024). Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008.

[5] Huang, Z., Wang, Z., Xia, S., Li, X., Zou, H., Xu, R., ... & Liu, P. (2024). OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI. arXiv preprint arXiv:2406.12753.

[6] Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., ... & Ganapathy, R. (2024). The llama 3 herd of models. arXiv preprint arXiv:2407.21783.

[7] Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., ... & Fan, Z. (2024). Qwen2 technical report. arXiv preprint arXiv:2407.10671.

[8] Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., ... & Gao, J. (2023). Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255.

"OlympaidBench and OlympicArena are Olympic-level data, they also adopt rule-based methods to judge the answer correctness. OlympicArena use LLM as judge to do process grading (I assume this is a bonus because it already has rule-based codes to judge the answer correctness). However, in your paper, the answer correctness is judged by LLMs, which is quite controversial recently [1, 2, 3]. Specific examples can be found in Figure 6", but how many such examples? What is the percentage of examples that are hard to judge by rule-based methods. I believe the way MATH, OlympaidBench and OlympicArena judge the answer correctness is reliable and widely adopted. At least you should provide evidence to show at what percentage the answers can not be judge by rule-based methods (It seems you have not even tried this). "

We will provide further rebuttal on the rule-based evaluation on the following aspects:

Statistics of answer types

In response to your feedback, we sampled 200 data samples based on difficulty and manually annotated the answer types and finally obtain the following statistics.

Among these, the categories that can be processed using rules and SymPy are "number," "latex," and "tuple." The latter categories are nearly impossible to evaluate using rules:

• "Multi latex": The answer consists of a combination of multiple LaTeX formulas.
• "Func": The answer is a custom-defined function.
• "Multi func": The answer includes multiple custom-defined functions.
• "Open text": The answer requires a written explanation.

The first three categories account for 74% of the total. Even if we assume that all rule-based methods could achieve 100% accuracy in predicting these categories, this would still be significantly lower than the accuracy of Omni-Judge, which stands at 86%. The relevant documents have been added to the supplementary materials, and we will also include this experiment in the appendix of the paper. In summary, we believe this is sufficient to demonstrate that rule-based methods cannot adequately cover our evaluation criteria.

The unreliability of Rule-based evaluation

To further confirm the unreliability of rule-based methods, we utilized the same model inference file on MATH500 [1] and evaluated it using different rule-based evaluation realizations from the open-source community: Qwen-2.5-MATH[2] and MetaMATH[3]. We found significant discrepancies in the results produced by these two evaluation scripts.

The experiments indicate that even well-defined rules struggle to effectively handle the evaluation in LaTeX format, let alone the more diverse evaluations at the Olympic level. In contrast, Section 4.2 of our paper has demonstrated that the evaluation provided by Omni-Judge and GPT-4o offer a more reliable evaluation. Additionally, we will release the OmniJudge weights and VLLM implementation, making our approach efficient and easier to follow.

Related work using rule-based evaluation

The two benchmarks you mentioned primarily focus on answer-level data that consists mostly of numbers or LaTeX, which differs significantly from the complexity of the answer-level data in our dataset. OlympiadBench and OlympicArena artificially limit the range of answers to a few categories, such as tuples, multiple-choice questions, and LaTeX, allowing for rule-based evaluations. In our work, however, we did not impose such restrictions at the answer level when filtering the data, thereby presenting a substantial challenge to existing models.

Additionally, there are also other related works[4] trying to provide solely on model-based evaluation using GPT-4.

Thanks for your valuable response, we will further address each of your concerns below:

Q1: “There are only 100 samples, but from three different models. This human evaluation is not that reliable due to the small number of examples per model.”

A1: We believe there is a misunderstanding here. Although the inference results come from different models, our ultimate evaluation target is the human alignment of judgment models—GPT-4o and Omni-Judge—rather than the solution generators. Furthermore, GPT-4o achieved highly accurate results(98/100) under conditions of high agreement among human annotators. Given that the 100 samples were randomly selected, we assert that this number is statistically significant for measuring the reliability of the judgment model. Similar works, such as MT-Bench[8], annotated only 80 samples, while [7] demonstrated that "for human evaluation, a test size of 50 is sufficient," even for tasks that are inherently more subjective, like summarization.

Q2: "Were the annotators well-trained? What are their background (e.g., are they from the math department?)? It is still unclear how many annotators are assigned to the same example."

A2: All annotators are students in the fields of mathematics and computer science, possessing sufficient domain knowledge to understand the questions and evaluate the LaTeX formatting. Moreover, they were instructed to assess whether the model's answers were correct based mostly on the provided correct answers, rather than relying entirely on their own judgment for the problem. Each annotator was responsible for 50 questions, and with 4 annotators, we ensured that each question was annotated by two different annotators. This setup allowed us to perform cross-validation to assess inter-agreement among the annotators.

Q3: “It seems that you use the same 100 samples to do human evaluation. My concern remains: "If the results reported for OmniJudge are indeed reliable (as they seem to be, given the detailed information provided in the appendix), then GPT-4o aligns with human judgments at most 95%.”

A3: We used the same 100 examples for human evaluation of both Omni-Judge and GPT-4o. Since Omni-Judge and GPT-4o's alignment utilized a larger set of examples for evaluation (2,220), it is unreasonable to directly multiply the accuracy between the different test set. Furthermore, we believe even if it comes to 95% that you claimed, it does not conflict with our assertion that "GPT-4o can essentially achieve human-level judgment at the answer level".

Q4: “It seems you use few-shot prompts. I believe for chat/instruct models, a common practice is to use zero-shot prompting. Few-shot prompts are used for evaluation base/L0 models.”

A4: We evaluated the performance of GPT-4o using the zero-shot prompting method you mentioned, as shown in Table 2 and Appendix F.2 of our paper. Regarding your concern, we would like to clarify the following two points on Section 4.3:

1. The primary motivation for using few-shot prompting (prompt following [5]) in Section 4.3 was not to enhance or measure the capabilities of GPT-4o. Instead, it aimed to explore the interactions between different domains and levels of difficulty through prompt learning, as discussed in our paper at L407-L422. We chose few-shot prompting because it can be recognized as supervised training to some extent[6].
2. Additionally, few-shot prompting can also be utilized to enhance the capabilities of GPT-4 series models. This is supported by our experimental results in Section 4.3 and findings from other works[1, 2, 3, 4]. While we acknowledge the examples you provided, it seems that using few-shot prompting to improve or evaluate GPT-4o's reasoning abilities is not an uncommon setting.

[1] Chen, Wenhu, et al. "Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks." arXiv preprint arXiv:2211.12588 (2022).

[2] Towards Geometry Problems Solving Employing GPT-4 Vision with Few-Shot Prompting: An Empirical Study of What Matters

[3] An, Shengnan, et al. "Skill-based few-shot selection for in-context learning." arXiv preprint arXiv:2305.14210 (2023).

[4] Kim, Jeonghwan, et al. "FinePrompt: Unveiling the Role of Finetuned Inductive Bias on Compositional Reasoning in GPT-4." Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.

[5] Kojima, Takeshi, et al. "Large language models are zero-shot reasoners." Advances in neural information processing systems 35 (2022): 22199-22213.

[6] Dai, Damai, et al. "Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers." arXiv preprint arXiv:2212.10559 (2022).

[7] Shaib, Chantal, et al. "How Much Annotation is Needed to Compare Summarization Models?." arXiv preprint arXiv:2402.18756 (2024).

[8] Zheng, Lianmin, et al. "Judging llm-as-a-judge with mt-bench and chatbot arena." Advances in Neural Information Processing Systems 36 (2023): 46595-46623.

Thank you for your detailed response and clarification! I still have some questions.

"Furthermore, we believe even if it comes to 95% that you claimed, it does not conflict with our assertion that "GPT-4o can essentially achieve human-level judgment at the answer level"".

You are releasing a new benchmark. The quality is very important. Although 95% does not conflict with your claims. However, there are models that have minor differences in accuracy in your benchmark, as shown in Table 2. How could we believe this rankdings of your leaderboard.

"We evaluated the performance of GPT-4o using the zero-shot prompting method you mentioned, as shown in Table 2 and Appendix F.2 of our paper. Regarding your concern, we would like to clarify the following two points on Section 4.3:"

Thank you for your clarification. I read through Appendix F.2 and notice that the maximum output length is set to 2048. Are there any cases that exceed this length and can not reach the answer (as the problem is difficult, maybe the solution is very long?)? A detailed error analysis is beneficial.

"The first three categories account for 74% of the total. Even if we assume that all rule-based methods could achieve 100% accuracy in predicting these categories, this would still be significantly lower than the accuracy of Omni-Judge, which stands at 86%. The relevant documents have been added to the supplementary materials, and we will also include this experiment in the appendix of the paper. In summary, we believe this is sufficient to demonstrate that rule-based methods cannot adequately cover our evaluation criteria."

Could you provide some examples for the last four categories, i.e. Multi latex, func, multi func, open text? Why not divide the problems into two main category and use rule-based method to judge the most of problems (number, latex, tuple)? "Even if we assume that all rule-based methods could achieve 100% accuracy in predicting these categories, this would still be significantly lower than the accuracy of Omni-Judge, which stands at 86%." This is not the shortage of the rule-based method, it is the weakness of your methodology. It is still surprising that 3/4 of problems that can be judged through the common practice and you still resort to gpt-4o as a judge.

"To further confirm the unreliability of rule-based methods, we utilized the same model inference file on MATH500 [1] and evaluated it using different rule-based evaluation realizations from the open-source community: Qwen-2.5-MATH[2] and MetaMATH[3]. We found significant discrepancies in the results produced by these two evaluation scripts."

There are subtle differences of these two evlauation repos. For MetaMATH, they use "The answer is: " to extract the answers from the model prediction. They do so because they organize the format of their training data to have this pattern, which is not suitable for evluation of other chat models. Qwen-2.5-MATH's evaluation codes are adapted from llm-harness (if I remeber it correctly), mainly relying on parsing "\boxed{}" to extract the answer.

In fact, they share almost the same judging criteria.

I believe the discrepancies are from answr extraction, not the rule-based method itself. Using LLMs to extract the answers [1] might be a smart approach to solve this. Simply using the same answer extraction pattern as Qwen-2.5-MATH and referring to MATH answer judgement codes could lead to near perfect answer judgement for the "number, latex, tuple".

[2] has not published yet and I read it before. They did even worse regarding "The answer judgement part".

By the way, for evaluating "Multi latex, func, multi func, open text", I think [3] provides a good way.

I will raise soundness to 3 as the additional experiments you conduct. Please remember to update your paper as you promise.

[1] Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., ... & Gao, J. (2023). Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255.

[2] Fang, Meng, et al. "Mathodyssey: Benchmarking mathematical problem-solving skills in large language models using odyssey math data." arXiv preprint arXiv:2406.18321 (2024)

[3] Liu, H., Zheng, Z., Qiao, Y., Duan, H., Fei, Z., Zhou, F., ... & Chen, K. (2024). MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark. arXiv preprint arXiv:2405.12209.

Thank you for your thorough and insightful feedback. We will incorporate the results and conclusions of our discussions into the paper, and we appreciate your suggestions regarding our work.

For your Questions:

Could you please give the proportion of these three categories "Rule-based: Correct and GPT-4o-based: Correct", "Rule-based: False and GPT-4o-based: False", and "Rule-based and GPT-4o-based judgments inconsistent"?

How many cases within the "Rule-based and GPT-4o-based judgments inconsistent" are due to the complexity of answers (within "Multi latex, func, multi func, open text")? Are there cases that Rule-based correct but GPT-4o-based wrong (at what percentage)?

We found that less than 2% of cases (fewer than 66 out of 4428) were identified by GPT-4o as incorrect answers while still matching the rules correctly. We believe these instances reliably indicate evaluation errors by GPT-4o. Upon careful examination, we discovered that some cases experienced parsing failures, resulting in both the predicted and ground truth outputs being matched as empty strings, which accounted for 30 out of the 66 cases. After identifying this issue, we removed these cases from the testable dataset. Consequently, the actual number of instances categorized as "Rule-based: Correct and GPT-4o-based: False" is reduced to 36. Among these, several remain contentious; for example, GPT-4o indicated that a condition was omitted in the answer.

Regarding the category "Rule-based: False and GPT-4o-based: Correct," we selected 50 cases for verification, all of which were attributed to errors in rule evaluation due to the complexity of the answers.

"It seems you start from the consistency of rule-based method and GPT4o evalution, where you implicitly assume the results of GPT4o evaluation is reliable. I believe that is why the rankings remain the same as GPT4o evaluation."

For this portion of the data, we employed manual annotation to ensure that these cases could reliably undergo rule-based evaluation. This approach allows us to provide an assessment that is independent of the model, effectively creating a set of simpler evaluations. While we recognize the importance of manually checking inconsistent cases, particularly those classified as Rule-based: False and GPT-4o-based: Correct, we currently have limited time. Therefore, we confidently rely on the reliability of our sampling check conducted above.

Could you please give GPT4o results using the two methods? It might be over-confident to use GPT4o to evaluate itself.

While it is true that GPT-4o's evaluation scores are somewhat higher than those of rule-based assessments, this variation is identical to other model variations we previously tested, which does not indicate over-confidence.

My major concern remains, i.e., using GPT4o for evaluation

Based on our discussions and experimental results, we present the following evidence to demonstrate the reliability of GPT-4o's evaluation:

1. In the 100 meta-evaluation test cases manually annotated by us, the agreement rate between GPT-4o and human annotators is 98%. This demonstrates that GPT-4o's evaluation performance aligns closely with human judgment. We also show that even among different rules in rule-based evaluations, there can be substantial differences in consistency (with agreement rates lower than 98%).
2. From the aforementioned experimental results, we observe the following:
   • GPT-4o exhibits stronger generalization in evaluating answers compared to other systems, and it has fewer known misclassification cases (in our experiments, fewer than 36, but needs further checking in other categories).
   • GPT-4o does not suffer from significant overconfidence issues.

Furthermore, as part of our ongoing discussion, we have also developed a subset used for rule-based evaluations. This subset can be reliably and efficiently utilized for rule-based assessments.

Thank you once again for your detailed response and valuable feedback. You have provided us with additional ideas and perspectives. We would appreciate it if our response could address your concerns.

Thank you for your responses and additional experiments!

I expect you should start with the answer type annotation and resort those with type "number, latex, tuple" to rule-based methods. It seems you start from the consistency of rule-based method and GPT4o evalution, where you implicitly assume the results of GPT4o evaluation is reliable. I believe that is why the rankings remain the same as GPT4o evalution. You have filtered out all examples that are not consistent, so there is not much information from this evaluation. You should use rule-based method independently, and set out to see the consistency of these two methods.

Several additional questions:

1. Could you please give the proportion of these three categories "Rule-based: Correct and GPT-4o-based: Correct", "Rule-based: False and GPT-4o-based: False", and "Rule-based and GPT-4o-based judgments inconsistent"?

2. How many cases within the "Rule-based and GPT-4o-based judgments inconsistent" are due to the complexity of answers (within "Multi latex, func, multi func, open text")? Are there cases that Rule-based correct but GPT-4o-based wrong (at what percentage)?

3. Could you please give GPT4o results using the two methods? It might be over-confident to use GPT4o to evaluate itself.

My major concern remains, i.e., using GPT4o for evaluation. Although the new dataset itself is a contribution, I believe this paper is flawed in its evaluation and will significantly benefit from a thorough revision. From this moment, I decide to remain my score (5). But AC is welcome to ignore my score if AC believe evaluation is not much a problem.

Remark: The assigned score (5) only reflects my overall assessment of the paper's level of rigor. Given that this submission falls under the "Dataset and Benchmark" track, I believe it is OK to recommend acceptance, with the expectation that the authors will update the evaluation codes as outlined in their responses. Taking into account the considerable effort the authors have demonstrated in addressing my concerns and improving their work, I will raise the score to 6 to reflect their diligence and additional experiments. But honestly speaking, this paper would still greatly benefit from a thorough revision to address the identified weaknesses and improve its overall quality.

Thanks for your valuable feedback. We would like to address each of your concerns below:

Weakness Thank you for your suggestion. We agree that providing more examples of the dataset would be helpful. We will include additional examples in the appendix for greater clarity.

Questions

1. Verification of GPT-4o Evaluation Reliability: As mentioned in Section 4.2, to validate the reliability of GPT-4o’s evaluations, we manually annotated 100 meta-evaluation test samples. The results showed that GPT-4o achieved an accuracy rate of 98%, which aligns closely with human judgments. These details are further elaborated in Section 4.2.

2. Tool-Integrated Reasoning Evaluation: We agree with your suggestion regarding the importance of assessing Tool-Integrated Reasoning (TIR) in our paper. We plan to update our experiments to include TIR, which will be reflected in the revised version of the paper.

3. Best-of-N Evaluation: In Table 2, we used two settings for "N": N=8 and N=256.

We hope this addresses your concerns and look forward to your further feedback.

We would like to express our sincere gratitude for your thorough and insightful review. Your feedback has been invaluable in guiding the refinement of our work. Here is our response:

Weakness 1: The reason why RM@256 does not perform better than RM@8 or even vanilla decoding could be attributed to two potential factors:

1. The policy model fails to generate high-quality reasoning traces, resulting in the absence of correct solutions among the available candidates.
2. The reward model cannot have sufficient supervision over Olympiad-level reasoning tasks. In other words, it cannot pick up the correct reasoning path among several candidates precisely.

To explore these two points in more detail, we conducted the following experiment:

We measured the performance of the policy model: Qwen2.5-MATH-Instruct across N samples. If at least one reasoning trace is correct, we consider the example to have passed. To assess the interference in RM selection, we also measured the proportion of correct COTs within the passing examples. Considering the inference cost, we sampled a subset of 500 instances according to difficulty to ensure that the distribution of problem difficulty is similar to the original dataset. The experimental result is shown below:

The results show that as the inference sampling increases, the policy model is capable of solving the majority of the problems. However, even after 32 samples, 33.8% of the problems still do not yield the correct solution. The second row of the table illustrates that as the number of samples increases, the proportion of interference items also increases. In the case of 32 samples, the correct answer proportion of the passed cases drops from 100% to 58.5%. This indicates that as the sampling count increases, it becomes increasingly challenging for the reward model to select the correct COT.

Question 1:

All annotators are voluntary students from the fields of mathematics and computer science, possessing sufficient domain knowledge to understand the questions and assess the LaTeX formatting. We have detailed the specific annotation procedures in Appendix C. Additionally, we ensure that each question is annotated by two annotators, allowing cross-validation to evaluate inter-annotator agreement, further guaranteeing the accuracy of our annotations.

Question 2:

The 100 samples are derived from the reasoning results of DeepSeek-Coder-V2, GPT-4o, and Qwen2.5-MATH-Instruct to ensure the evaluation robustness of GPT-4o and OmniJudge. We also ensure that each question is annotated by two annotators, enabling cross-validation to assess inter-annotator agreement and further ensure the correctness of our annotations. For difficulty sampling, we followed the difficulty grading outlined in Section 2.2, ensuring that questions of all difficulty levels are represented. Relevant details will be provided in the appendix.

Dear Reviewer ssE3,

Thank you again for your valuable feedback and comments! We would greatly appreciate it if you could let us know whether you are satisfied with our response and then adjust our overall rating. We will be happy to address any remaining concerns.

Sincerely,

Omni-MATH Team

We deeply value your recognition of our work, as well as your constructive feedback, which has allowed us to significantly improve the manuscript. If you find our revisions have addressed your concerns effectively, we would be grateful if you could confirm this and let us know if our efforts have further strengthened your overall assessment of the work.

Best regards, Omni-MATH Team

Thank you for your feedback. In fact, our experiments not only demonstrate that the reward model fails to select the correct samples but also highlight the flaws in this test time scaling mechanism: as the number of samples increases, the challenges posed by incorrect solutions become more significant. The cost of an incorrect solution is higher than the benefit of a correct solution. Similar conclusions can also be found in [1]. This situation does not change with a different reward model. Below are our results using another reward model. We selected the top-ranking model in the "reasoning" domain on RewardBench, Skywork-Reward-Gemma-2-27B-v0.2, also with the policy model qwen2.5-MATH-72b-Instruct. We found similar results:

Additionally, we want to clarify that our current experiment is aimed at demonstrating that the existing vanilla BoN test-time scaling technique is far less efficient than O1-like strategies. Enhancing performance is not our primary focus, nor is it the main contribution of this paper.

We would appreciate it if you could confirm this and let us know if our efforts have adequately addressed your concerns and further strengthened your overall assessment of the work.

[1] Stroebl, Benedikt, Sayash Kapoor, and Arvind Narayanan. "Inference Scaling $\scriptsize\mathtt {F}$ Laws: The Limits of LLM Resampling with Imperfect Verifiers." arXiv preprint arXiv:2411.17501 (2024).