MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models

We would like to sincerely thank the reviewer for the useful comments on our work. We take every comment seriously and hope our response can address the reviewer’s concerns. If there are any remaining questions, we are more than happy to address them.

Q1. Some baseline approaches to compare are missing, e.g., MAmmoTH, Platypus and Code LLaMA.

A1. As suggested, we added the mentioned baselines in Table 2 of the updated paper and the additional comparisons are shown in the table below. As can be seen, MetaMath outperforms other CoT-based models and Code-LLaMA with the same number of parameters.

Q2. The ablation study is not comprehensive enough. Only the 7B model is tested.

A2. As suggested, we conducted an ablation study on LLaMA-2-13B and the results are shown below. The observation is consistent with our ablation study on the 7B model in Section 4.3: Combing answer augmentation and rephrasing augmentation data for finetuning leads to a slightly higher accuracy. The accuracy can be further improved by merging the FOBAR and SV augmentation data. We added the results in the updated paper (Table 9 in Appendix A.7).

Q3. Table 3 is confusing - should add a line breaker between SFT and MetaMath

A3. Thanks for your suggestion and we have fixed it accordingly in the updated version.

We would like to sincerely thank the reviewer for the useful comments on our work. We take every comment seriously and hope our response can address the reviewer’s concerns. If there are any remaining questions, we are more than happy to address them.

Q1. Potential for Benchmark Hacking & Dependence on High-Quality Initial Questions. To some extent, both the weaknesses can be addressed by doing 0-shot evaluation on some other datasets like DROP (https://allenai.org/data/drop)

A1. As suggested, we perform a zero-shot evaluation on the DROP dataset to compare MetaMath with baseline models. Since we focus on mathematical reasoning, we only consider the DROP questions with numerical answers. The table below shows the testing accuracy. As can be seen, MetaMath-7B and MetaMath-13B still outperform the baseline models by a large margin, which shows MetaMath does not suffer benchmark hacking on GSM8K and MATH datasets. We have added the results in the updated paper (Table 10 in Appendix A.8).

Q2. In Table 3, MetaMath finetuning always begins with the AnsAug split, right? Do the authors have any thoughts on what would happen if we start training from (say) SV or FOBAR?

A2. We apologize for the confusion caused by Table 3. When using data from multiple bootstrapping methods for finetuning, we mix all augmented data instead of using them sequentially (AnsAug -> Rephrasing -> SV -> FOBAR). In other words, at each training step, we randomly take a batch of samples from the mixed augmented data for updating the model parameters.

Thanks for including the results on DROP and for the clarification. The gains are impressive and convincing. Did you get a chance to try the 70B model? I understand if running inference on 70b is not possible due to resource constraints.

We would like to sincerely thank the reviewer for the useful comments on our work. We take every comment seriously and hope our response can address the reviewer’s concerns. If there are any remaining questions, we are more than happy to address them.

Q1. The major weakness I see is the lack of novelty. The paper in essence combines existing methods for bootsrapping.

A1. This paper proposes a novel question bootstrapping idea to augment the training dataset. Compared with previous methods (e.g., SFT, RFT, WizardMath), our MetaMath introduces two novel augmentation methods:

(i) Rephrasing questions. Existing methods (e.g., RFT and WizardMath) focus on enlarging the answer data, which can be achieved by sampling more answers from LLMs. MetaMath proposes to create more questions, which is more challenging. To the best of our knowledge, we are the first to use a rephrasing prompting method for augmenting questions.

(ii) Question bootstrapping by backward reasoning. Backward reasoning ability is crucial to solving many mathematical questions. However, the training set of mathematical tasks (e.g., GSM8K) lacks such data to improve backward reasoning. To deal with this problem, we introduce the templates (i.e., masking a number in the question and asking the LLM to predict the masked number) proposed in Self-Verification and FOBAR to create backward questions. Note that both Self-Verification and FOBAR use backward reasoning for verification rather than data augmentation.

In summary, our paper proposes rephrasing questions and bootstrapping by backward reasoning to augment a diverse dataset --- MetaMathQA. MetaMath, which finetunes from state-of-the-art open-source LLMs on our MetaMathQA dataset, demonstrating excellent elementary mathematical problem-solving capability. In addition, we have released the MetaMathQA dataset for public use to improve the forward and backward reasoning capabilities of the LLM.

We would like to sincerely thank the reviewer for the valuable comments on our work. We take every comment seriously and hope our response can address the reviewer’s concerns. If there are any remaining questions, we are more than happy to address them.

Q1: It is unclear how the proposed bootstrapping approach generalizes to other types of multi-hop reasoning problems.

A1. The core idea of the proposed bootstrapping approach is to diversify the questions in both forward and backward reasoning directions. Our approach can be extended to other reasoning tasks. We conducted an additional experiment to show a successful application of our bootstrapping method to the Game of 24, which involves multi-hop reasoning steps to reach 24 given 4 numbers. Given an original question with 4 numbers (2,3,4,12), its answer (2*3 - 4) * 12 is an expression that can reach 24. We can apply answer augmentation and question bootstrapping to generate more question-answer pairs.

Answer augmentation. The solutions of obtaining 24 given 4 numbers may not be unique, e.g., 2*12*(4-3) = 24 is another solution to the given numbers (2,3,4,12). Hence, for a question with 4 numbers, we enumerate all the correct solutions and obtain the answer augmentation data.

Question bootstrapping. Game of 24 can be extended to Game of $n$, i.e., given 4 numbers (one number is 24), the goal is to obtain $n$ using basic arithmetic operations (+, -, *, /). We use Game of n for question bootstrapping. We replace a number in the original question with 24 and the question is to obtain the number. This idea is similar to creating backward questions in our paper, i.e., masking a number in the question and asking the LLM to predict the number. For a Game of 24 question, we can bootstrap it and obtain 4 Game of $n$ questions, as an example shown in the table below.

Game of 24 Setup. We randomly select 1362 Game of 24 questions from 4num.com, where 681 questions are for training and the remaining 681 questions are held-out for testing. We apply the above augmentation methods to generate more training data from the 681 questions. We apply answer augmentation by enumerating all the correct forward solutions and obtain an AnsAug dataset consisting of 6052 question-answer pairs. We apply question bootstrapping to obtain a bootstrapping dataset (consisting of 2724 Game of n question-answer pairs). To verify the effectiveness of the bootstrapping approach, we randomly sample 4000 question-answer pairs (Game of 24) from the AnsAug datasets, and 2052 backward question-answer pairs (Game of n) from the bootstrapping dataset. We finetune LLaMA-2-7B on AnsAug and the mixed data separately for comparison.

Results on Game of 24. Table below shows the testing accuracy. As can be seen, our proposed augmentation approaches (AnsAug and AnsAug+Bootstrapping) have higher accuracy than SFT, which trains on the original 681 question-answer pairs. Furthermore, using question bootstrapping for augmentation can boost the performance of AnsAug. Hence, the proposed bootstrapping method is useful for Game of 24.

Results on Game of n. For each question-answer pair in the testing set of Game of 24, we create 4 more testing questions of Game of n using the above question bootstrapping method. In total, we obtain 3405 testing questions. Table below shows the testing accuracy. Again, using our augmentation methods performs better than SFT by a large margin. Furthermore, AnsAug + Bootstrapping performs the best, demonstrating our proposed method is also useful for Game of n.

We have included all the above experiments in Appendix A.4 of the updated paper.