Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models

W2 (2): Real-world scenarios for AutoIF program verification may require various dependencies and APIs.

Thanks for the advice! Addressing concerns about APIs, the models reported in Table 1 are deployable open-source models, Llama3-70B and Qwen2-70B, used as supervision models, rather than GPT-4. To further alleviate your concerns, we present the self-alignment using Llama3-7B as the supervision model, as well as the setup where Qwen2-3B-instruct enhances Qwen2-7B. These supervision models can be lightweight deployed using a single GPU.

Additionally, as detailed in our supplementary materials, all compiler execution processes can be completed with a single command on the CPU. We hope these results alleviate your concerns regarding computational resources!

---

W3: IFEval and FollowBench lack practicality, as they only include verifiable instructions like word count. The authors should consider more realistic scenarios, such as those in BigCodeBench [4].

Thank you for your suggestion. First, we think reviewer may be underestimating the difficulty of FollowBench. As shown in appendix J, FollowBench has the following two characteristics:

• Six Dimensions's Tasks of Followbench: All constraints being evaluated for instruction following under the combination of various integrated tasks

1. Content Constraint: Data-to-Text Generation, Document-Level Event Argument Extraction, Document-Level Named Entity Recognition, Text Generation with Language Constraints, Open-ended Question Answering
2. Situation: Suggestion Generation, Role-playing, Complex Situation Reasoning
3. Style: Open-ended Question Answering
4. Format: Text-to-Table Generation, Open-ended Question Answering
5. Example: 40 diverse NLP tasks
6. Mixed: Text Editing, Summarization, Machine Translation, Story Generation

• Examples of Constraints in Six Dimensions: Each instruction’s complexity cannot be resolved solely through surface semantics or 1-to-1 translation. Detailed cases are listed in the Appendix J (Page 23-24).

Additionally, as mentioned in our W(2) response, we also evaluate AUTOIF on more practical scenarios in Table 2, including:

1. InfoBench：OOD instruction following benchmark
2. MT-Bench：More natural and general instruction following capability
3. Arena Hard (Chatbot Arena)：Realistically challenging comprehensive chatbot evaluation

This paper primarily focuses on evaluating AUTOIF's general instruction-following capabilities. Evaluating instructions in specific domains, such as code and mathematics is not the core focus of this work; However, we are will to expanding evaluations in code-related areas like BigCodeBench in the future.

---

W4: AUTOIF should be evaluated in a multilingual context to demonstrate its generalization ability.

Thank you for your insightful suggestion. In fact, based on our simple and efficient workflow, AUTOIF framework is completely language-agnostic. Therefore, we believe that extending it to multiple languages requires only changing the seed instruction to the specific instruction in another language, along with selecting a supervision model with additional capabilities. All other processes can be reused. Due to the tight rebuttal timeline, we are very willing to follow your advice and include more multilingual benchmarks in our future work to enhance its generalization ability.

Thanks for your helpful comments and we are glad that you acknowledged the strengths of our work! Below we will address your separate concerns.

---

W1: Using programs to verify instructions following has already been implemented in IFEval, Moreover, using execution feedback for training is not novel, as it has been widely applied in training code LLMs [1-3].

Thank you for your suggestion. We would like to clarify the distinctions between our work and the three related Code LLM efforts as follows:

• From the perspective of code generation goals: References [1-3] focus on code generation tasks aimed at producing more reliable code, while AutoIF's goal is to enhance the general instruction-following capabilities of LLMs under complex natural language instructions, with code used solely to verify whether the model's responses meet the criteria.

• From a technical standpoint: CodeRL combines pre-trained language models with deep reinforcement learning, focusing on assessing the functional correctness of generated programs through a critic network. Opencodeinterpreter integrates code generation, execution, and iterative optimization. Both of these works are significantly different from AutoIF. While CodeT shares some similarities with AutoIF in generating test cases, it provides tests for generated code, whereas AutoIF uses program execution to validate the model's instruction-following ability, representing a fundamental difference.

Additionally, AutoIF introduces Executor Feedback along with model scoring and recall verification mechanisms. Specifically, instruction-following tasks are more diverse and semantically rich, so AutoIF not only relies on code execution results but also utilizes a scoring mechanism to validate generated responses from multiple angles, enhancing data reliability and quality. This further differentiates it from the IFEval dataset, which focuses solely on rule-based verification instructions. We sincerely hope that reviewers will appreciate the comprehensiveness of our innovations in verification. The ablation study (Table 4) also demonstrates that these mechanisms complement each other.

---

W2 (1): AutoIF struggles to scale to complex instructions and has difficulty handling semantically rich instructions that cannot be evaluated through execution.

Thank you for your comment. The motivation behind AUTOIF is to generalize complex instruction-following abilities using a scalable, verifiable method through simple handwritten instructions. We will respond in two parts:

Generalization of Semantically Rich Instructions: Actually, the datasets evaluated in Tables 1 and 2 contain semantically rich instructions:

1. FollowBench: This dataset evaluates instruction following across over 50 integrated NLP tasks with six difficulty levels. Many instructions are unverifiable and require GPT-4 scoring, as detailed in the case examples in Appendix 6.

2. MT-Bench: A comprehensive natural instruction evaluation set that focuses on challenging questions to differentiate model capabilities, consisting of 80 high-quality multi-turn dialogue questions across eight use cases: writing, role-playing, information extraction, reasoning, mathematics, coding, knowledge I (STEM), and knowledge II (humanities/social sciences).

3. InfoBench: This dataset further validates our OOD instruction-following capabilities and lacks verifiable instructions, with all evaluations requiring external model scoring.

4. Arena Hard: A highly challenging evaluation set aimed at realistically assessing comprehensive chatbot performance.

We believe these integrated tasks effectively validate AUTOIF's generalization in real-world applications.

Potential Optimization Approaches: Furthermore, we are open to discussing optimization strategies for the more high-level semantic instruction constraint. Here are a few ideas:

1. Establish principles for style, such as defining "politeness," which might include honorifics and exclude offensive language. Verification methods could extend beyond executor feedback.
2. Clearly define style types and manually create style-related seed instances, using model pairwise comparisons for quality validation.

These are preliminary thoughts and not the main focus of this paper, but we are willing to explore further style optimization as future work.

Thank you for the response. It addressed some of my concerns and helped me understand the work better.

However, regarding OAIF, I would like to respectfully argue that RAILF+Online DPO still works in the case of instruction following. Considering that LLMs can evaluate whether generated responses are aligned with the instructions, it should be one of the baselines. As discussed, such methods should work well with style- and semantic-related instructions.

Q1: How do the accuracy rate thresholds in the verification steps affect the model performance?

Thank you for your suggestion. In fact, in the left two graphs of Figure 4, we have explored the impact of increasing the accuracy threshold in the verification step from both the instruction and query perspectives, as well as its effect on the final data volume.

Here are our core findings in Figure4: As the pass rate threshold increases, the amount of SFT data decreases at the instruction level, while model performance consistently improves. This indicates that the quality of instructions is a crucial factor influencing instruction-following (IF) performance. At the query level, the amount of SFT data also decreases with higher pass rate thresholds. Notably, performance peaks at a pass rate of 0.8 and declines beyond 1. This observation aligns with our expectations, highlighting a trade-off between data quality and quantity as the threshold increases. We hope these can address your concerns.

---

Q2: How does RAILF+Online DPO (e.g., OAIF) compare to AUTOIF?

Thank you for your insightful suggestion. After carefully reviewing the OAIF work you pointed out, both AutoIF and OAIF employ online DPO training strategies to enhance model performance. The key difference is that AutoIF focuses on using code validation to improve the model's ability to follow complex instructions, while OAIF utilizes LLM annotators to label positive and negative samples, aligning with the model's preferences. The core motivation of this paper is to emphasize the automation and scalability of enhancing instruction adherence with minimal human effort. Therefore, OAIF is not suitable as a baseline for comparison in our study.

---

Finally, we sincerely hope that our response addresses your concerns, and we once again thank you for your thorough review!

Thanks for your helpful comments and we are glad that you acknowledged the strengths of our work! Below we will address your separate concerns.

---

Weakness: Failed cases will help understand the scope of the next steps and areas for improvement. Additionally, the seed instructions are limited to the domain of string manipulation, which could also be diversified.

Thank you for your insightful suggestions. As you mentioned, failed cases can indeed reveal many areas for improvement. From the lower pass rates in cross-validation samples, we found that constructing and verifying high-level semantic instructions (such as those with emotional or creative elements) is a key direction for enhancing the LLM alignment with human instruction following. Specifically, we believe there are several optimization avenues for AUTOIF to better accommodate high-level semantics:

• Handwritten prompts: We can consider fine-grained emotional differences in the prompts by handwriting instructions that allow for nuanced distinctions.
• Instruction rewriting phase: We can establish creative principles (e.g., for an emotional assistant, qualities like humor and empathy) and allow humans to iteratively optimize these principles based on the quality of generated outputs from small batches, potentially using instruction evolution techniques like AutoEval instructions [1].
• Principle of LLM verification: Inspired by CAI [2], we also need to incorporate fine-grained emotional differences in the verification prompts during the verification phase or use creative metrics for scoring, rather than solely focusing on instruction correctness, to overcome the limitations of executor-based verification that only addresses verifiable prompts.
• Online/Offline DPO data construction: For creative tasks, we should avoid using executor-based success rates to construct positive and negative samples. Instead, a combination of LLM verification scores and executor-based scores should be employed to balance correctness with higher-level emotional semantics.

These are just some experiential thoughts and are not the main focus of this paper. We are open to addressing further optimization in future work

---

Q1: Please improve consistency of terms in figure.

Thank you for your correction! We will work on improving the consistency of terminology between the text and figures, as well as proofreading the entire paper.

---

Q2 & Q3: An example case of how a sample develops in the AutoIF pipeline and providing reasoning and examples for cases with lower accuracy.

---

Thank you very much for your comments. In Appendix, Tables 6 and 7 present case examples of AutoIF, including the query, response, and verification function, to help readers understand our process. In the supplementary materials, we provide a series of JSONL files in the sample_data folder that illustrates the development process of our samples within the AutoIF framework, particularly the intermediate data results in "dpo_query_w_funcs.jsonl" for reviewer reference.

In the revised version, we plan to include an image that details the process from a seed instruction to the final constructed sample, highlighting the verification at each stage. Additionally, we will include more detailed error cases to enhance the presentation of our paper.

---

Q4: Most seed instruction showcase restriction over string manipulation domain only.

That is good point! Our motivation is to synthesize high-quality instruction-following alignment datasets through fewer than 100 simple instructions in an automated, verifiable, and scalable process; therefore, our paper did not specifically optimize the seed instruction set. To increase the diversity of instructions, we could manually introduce higher-level semantic instructions or domain-specific instructions during the design phase, followed by targeted optimization using AutoIF.

---

Q5: line 319, what is the value of K, for which the results have been sampled from?

Thank you for your advice. Our value of K is 8. For details, please refer to the dpo_query_eval_score_results.jsonl file in the sample_data folder of our supplementary materials, which displays eight sample entries from the data. These samples were extracted from our AUTOIF SFT training set. We will incorporate this information further into our implementation details.

---

Reference:

[1] Automatic Instruction Evolving for Large Language Models

[2] Constitutional AI: Harmlessness from AI Feedback

Q4：Please clarify the main technical differences with StarCoder2-Instruct.

This is a good point. While StarCoder2-Instruct utilizes a base model for code and test case generation, AutoIF introduces significant innovations and optimizations in four key areas:

1. Task and Motivation:

   • StarCoder2 enhances task completion rates for code generation tasks. AutoIF focuses on cross abilities enhancement, improving general instruction adherence through diverse capabilities like code generation and Instruction rewriting.

2. Verification Mechanism:

   • StarCoder2-Instruct filters responses by executing generated test cases in a sandbox, essentially sampling outputs. In contrast, AutoIF employs Executor Feedback, Back-translation Verification, and LLM Quality Verification., validating generated responses through execution results, NLI Models and the scoring mechanism based on LLM, enhancing both reliability and quality.

3. Scalability and Versatility:

   • While StarCoder2 excels in SFT dataset generation, it primarily targets programming task responses. AUTOIF automates instruction-response pair generation and validation across various training algorithms (SFT, Offline DPO, Online DPO), offering greater scalability.

4. Experimental Aspects:

   • StarCoder emphasizes coding task performance. AUTOIF enhances general instruction-following capabilities and validates its plug-and-play framework across domains without conflicts math, code and general domain. Its varied settings and diverse supervision models (GPT-4, LLaMA, Qwen series) further broaden its applicability.

---

Q5: Can improved IFEval performance be linked to more practical tasks? GSM8K and HumanEval may not show additional gains.

Thank you for your suggestion. In fact, we believe benchmarks evaluated in Tables 1 and 2 are all relevant to real-world tasks:

• FollowBench: This assesses instruction-following across over 50 integrated NLP tasks with six difficulty levels. Most instructions cannot be validated and require GPT-4 for evaluation, as detailed in Appendix 6.

• MT-Bench: A comprehensive evaluation set focusing on challenging questions to differentiate model capabilities, it includes 80 high-quality multi-turn dialogue questions across eight use cases: writing, role-playing, information extraction, reasoning, mathematics, coding, STEM, andhumanities/social sciences.

• InfoBench: This dataset further validates our out-of-distribution instruction-following capabilities, also lacking verifiable instructions, with all evaluations requiring external model scoring.

• Arena Hard (win rate): One of the most challenging evaluation sets, aimed at realistically assessing comprehensive chatbot performance.

Thus, the results in Tables 1 and 2 effectively validate the generalization of AutoIF in real-world tasks.

Regarding the GSM8K and HumanEval results in Table 1, we consider there may be a misunderstanding for reviewer. As we mentioned in our contributions, we include general (MMLU, C-Eval), math (GSM8K), and coding (HumanEval) abilities to show that improvements in instruction-following capabilities do not conflict with other abilities. Therefore, we do not need to enhance these abilities, their stability without significant decline proves that AutoIF avoids performance conflicts, and it even achieves slight improvements.

Moreover, we are willing to exploring further instruction-following validation in specialized code domains (BigCodeBench / LiveCodeBench / LeetCode) as future work.

---

Q6：Reliability Analysis of Generated Code and Tests

Thank you for your insightful suggestion. In Table 5, we discuss the relationship between the code capabilities of the supervision model and instruction-following abilities, presenting the pass rates of test cases. To enhance our experiments' completeness, we incorporated your suggestion and conducted a reliability analysis of the generated code and tests. We sample 10K instruction verification codes generated by GPT-4 and Qwen2-72B and evaluatetheir code execution rates. After excluding external factors (e.g., compiler environment bugs), we present the code execution rate results alongside Table 5 for analysis.

The experiments show that the code execution rate is reasonable and positively correlated with the code capabilities of both models. Notably, MBPP performance, code pass rate, and instruction-following capabilities are consistently positively correlated. This reinforces our conclusion on "Data Efficiency," where the supervision model's coding ability significantly impacts data synthesis quality and final instruction-following capability.

Thanks for your helpful comments and we are glad that you acknowledged the strengths of our work! Below we will address your concerns point by point:

---

W1 & Q3: It is unclear whether the self-alignment setting applies to weaker base models with less than 10B params (e.g., Llama3-8B).

Thank you for your comments. We have evaluated the Llama3-8B-instruct version as a supervision model for self-alignment with the Llama3 8B base model. The experimental results are as follows:

The left four columns represent the results of IFEval, while the right side shows the results of FollowBench SSR AVG.

Results indicate that compared to the SFT version of the Llama-8B (ShareGPT), Llama3 (AutoIF) achieves a significant improvement of over 12% on IFEval(Loose Instruction ACC.), while maintaining a 6% improvement on FollowBench. This validates the effectiveness of AutoIF in the self-alignment setup with the weaker model (only 7B). We will incorporate these results and analyses into the revised version.

---

W2：Lack of weak-to-strong setup.

Thanks for the advice! Honestly, we believe that the weak-to-strong setup is indeed quite demanding and challenging, as the data synthesis process of AutoIF comprehensively tests the supervision model's capabilities in instruction rewriting, code generation, and quality scoring. Further focusing on real-world scenarios, we rarely use weak models to enhance the instruction-following capabilities of strong models, which is somewhat counterintuitive. Given these reasons, we do not conduct experiments in the main text.

To improve the completeness of the AutoIF setup and further address the reviewer's concern, we align the model series and ultimately chose Qwen2 3B-Instruct to perform weak-to-strong alignment with the Qwen2 7B base model. The experimental results are as follows:

Results show that the weak-to-strong setup can also achieve stable improvements. We attribute this gain to the cross-capability benefits derived from code generation and instruction rewriting by the weak model, demonstrating that the AutoIF framework possesses strong generalization. We will also include these experimental results and analyses in the revised version.

---

W3: Missing baseline where responses are validated through the LLM itself without code

Thank you for your suggestion. As shown in Table 4 of the ablation study, the setting for ablation of cross verification（executor-based flitering） refers to the synthetic data of AutoIF, where we retain only the final stage of LLM query quality verification and do not conduct the executor feedback cross-validation process. This is essentially equivalent to the baseline process you mentioned, as in this setting, the data originally filtered by the code validation will ultimately undergo LLM Query Quality Verification.

As noted in the analysis: "The most significant performance drop occurs when the Cross Verification of instructions is removed, underscoring its importance over query quality verification. This verifies that a high-quality instruction set is fundamental to the AutoIF process." We believe this result validates the value of the verification function and test cases in AutoIF.

---

Q2: Typos in line 107.

Thank you for your correction; it should be the "Strong-to-Weak" setting. However, as we verified the "Weak to Strong" setting in W2, we are happy to incorporate this aspect into both the contributions and the experiments.

Thanks for your helpful comments and we are glad that you acknowledged the strengths of our work! Below we will address your concerns point by point:

---

Weakness & Q2: Cross-verification may be unreliable because it essentially assumes the correctness of the test function. This is due to (i) poor test samples potentially undermining a correct function, and (ii) the possibility that both the test and the function may be incorrect simultaneously, leading to low-quality samples.

This is indeed a great question! I would like to address your concern from two aspects:

Effectiveness of Self-Consistency: In fact, our cross-validation incorporates a self-validation process with four criteria （line 214-234）, and their combination makes this process more rigorous. As demonstrated in our ablation study, we show the impact of removing this cross-validation:

It is evident that our self-consistency verification can significantly enhances the instruction following capability of LLMs.

Manual Error Rate Validation: Furthermore, to alleviate your concerns, we manually checked 50 test cases after cross-validation, verifying the function and identifying any errors that occurred simultaneously.

Statistics indicate that simultaneous errors of both test cases and verification funcs are relatively rare. This is because our motivation stems from automated synthesis, utilizing simple and verifiable instructions that ultimately generalize to complex, general instruction-following tasks, ensuring the overall process is reliable. Additionally, AUTOIF incorporates "LLM-based quality verification" and "NLI model-based back-translation verification", which we believe can effectively address any oversights in executor-based validation.

---

Comment1: It is generally good to discuss more of code execution works.

Thank you for your suggestions. We are happy to add a section in the appendix to discuss Code execution, including RLTF, LEVER, Self-OSS-Instruct, ODEX, and other related works mentioned by reviewers. Here is an overview of these works we summarized:

• RLTF generates data in real-time during training to continuously update and optimize the model. It incorporates multi-granularity unit test feedback, using fine-grained signals to pinpoint specific errors in the code, guiding the model toward generating higher-quality code.

• LEVER trains a verifier to assess whether programs generated by large language models (LLMs) are correct. The verifier's judgment is based on natural language input, the generated program, and its execution results. After generating a program, it is reordered based on the verification score combined with the LLM generation probability, marginalizing programs with the same execution result to select the optimal solution.

• ODEX introduces the first open-domain execution-based natural language to Python code generation dataset. This dataset includes 945 pairs of natural language-code pairs, covering 79 different libraries, and provides 1,707 manually written test cases for execution validation.

• Self-OSS-Instruct employs context learning techniques to enable the StarCoder2 model to self-generate diverse programming instructions from seed code snippets. This process includes concept extraction, where the model identifies key concepts in the code, and instruction generation, where it creates specific programming tasks based on these concepts.

We will integrate and expand upon these points in the revised version and conduct proofreading.

---

Comment2&3: More accurate to cite the use of rejection sampling and Typos.

Thank you for your comments. We will revise the citation and correct the typos to improve the professionalism of the paper.

W3: AutoIF is overly focused on executor feedback and does not consider other methods, such as human feedback or model-based evaluation.

Thank you for your suggestion. We have already provided an response in W1 (1). In fact, AutoIF introduces a triple verification mechanism. In addition to incorporating "Executor Feedback" to automate the synthesis of verifiable instructions, it also introduces a dual mechanism of "LLM's Quality verification and Back-Translation Verification" to address more diverse and semantically complex instruction-following tasks. Therefore, AUTOIF relies not only on code execution results but also on the diverse scoring mechanism to validate generated responses from multiple perspectives, thereby improving data reliability and quality from semantic level. This significantly differs from the IFEval dataset, which focuses solely on rule-verifiable instructions. We hope the reviewers can appreciate the comprehensiveness of our innovations in verification. The ablation study (Table 4) also demonstrates that these mechanisms complement each other.

---

W4: Allowing human insights to supplement the framework enables better adaptation.

This is a good point! The motivation of AUTOIF is to generalize more complex instruction-following abilities using an automatic, scalable method through simple handwritten instructions with minimal human effort. Therefore, our intention is for AutoIF to serve as an automated, plug-and-play solution widely applicable in the field of large model instruction adherence, which is its advantage. To address your concerns, we propose two areas where AutoIF can incorporate human intervention:

• Human insights can be involved in the seed instruction construction phase, allowing for intervention at the source to enhance instructions while only requiring fewer than 100 handwritten instructions.
• In the final stage of LLM verification, we can integrate more granular principles established by humans into the verification prompt and iteratively modify the prompts during small batch instruction validation.

These are merely some experiential thoughts and are not the main focus of this paper. We are open to addressing further style optimization in future work.

---

W5: Automatic instruction verification is not novel, with related works emphasizing instruction alignment via code-based feedback. While AUTOIF introduces innovations in scalability and data synthesis, these studies suggest it does not represent a fundamentally new direction.

Thank you for your suggestion. We would like to clarify the distinctions between our work and the Code LLM efforts you mentioned as follows:

• From the perspective of code generation goals: The referenced works focus on code generation tasks aimed at producing more reliable code, while AutoIF's goal is to enhance the general instruction-following capabilities of LLMs under complex natural language instructions, with code used solely to verify whether the model's responses meet the criteria.

• From a technical standpoint: Codellama2’s core innovations focus more on long context and code completion tasks. DeepSeek-Coder-V2 trains a reward model using a Python executor as the basis for subsequent RL optimization. In Llama3, the sections related to code training emphasize filtering overall code and checking for syntax errors in sub-units. These works are significantly different from AutoIF. Although SelfCodeAlign is somewhat similar, as it generates code for test case validation to filter high-quality responses, AutoIF emphasizes using program execution to validate the model's instruction-following ability, which is a cross-capability verification method.

Furthermore, it is important to note that the related works focus on instruction tuning for code generation to enable general code alignment capabilities in LLMs, while AutoIF specifically aims to enhance LLMs' ability to follow complex and natural instructions, which are significantly different objectives.

In addition, AutoIF introduces "executor feedback" along with "LLM-based quality verification scoring" and "back-translation verification mechanisms". Specifically, instruction-following tasks are more diverse and semantically rich, so AutoIF not only relies on code execution results but also utilizes a scoring mechanism to validate generated responses from multiple angles, enhancing data reliability and quality. We sincerely hope that reviewers will appreciate the comprehensiveness of our innovations in verification. The ablation study (Table 4) also demonstrates that these verification mechanisms complement each other.

Thanks for your helpful comments and we are glad that you acknowledged the strengths of our work! Below we will address your concerns point by point:

---

W1(1) & W2 & Q1: AUTOIF will focus on "verifiable instructions," which may lead it to favor easy-to-verify tasks like IFEval, potentially excluding more complex or subjective tasks.

Thank you for your insightful suggestion. First, AUTOIF not only incorporates Executor Feedback to automate the synthesis of verifiable instructions, but also introduces mechanisms of "LLM's Quality verification" and "Back-Translation Verification" to address more diverse and semantically complex instruction-following tasks. Therefore, AUTOIF relies not only on code execution results but also on the diverse scoring mechanism to validate generated responses from multiple perspectives, thereby improving data reliability and quality from semantic level. This significantly differs from the IFEval dataset, which focuses solely on rule-verifiable instructions. We hope the reviewers can appreciate the comprehensiveness of our innovations in verification. The ablation study (Table 4) also demonstrates that these mechanisms complement each other.

Additionally, we would like to clarify that the datasets evaluated in Tables 1 and 2 encompass more complex and subjective tasks that reflect real-world scenarios, with difficulties far exceeding that of IFEval:

• FollowBench: This assesses instruction-following across over 50 integrated NLP tasks with six difficulty levels. Most instructions cannot be validated and require GPT-4 for evaluation, as detailed in Appendix 6.

• MT-Bench: A comprehensive evaluation set focusing on challenging questions to differentiate model capabilities, it includes 80 high-quality multi-turn dialogue questions across eight use cases: writing, role-playing, information extraction, reasoning, mathematics, coding, STEM, andhumanities/social sciences.

• InfoBench: This dataset further validates our out-of-distribution instruction-following capabilities, also lacking verifiable instructions, with all evaluations requiring external model scoring.

• Arena Hard (win rate): One of the most challenging evaluation sets, aimed at realistically assessing comprehensive chatbot performance.

Moreover, we have also included performance metrics on the long-text evaluation dataset FollowRAG [1].

We believe these integrated tasks effectively validate AUTOIF's generalization in real-world applications and hope these results can alleviate your concerns.

---

W1(2): The paper does not delve into the non-code-based verification methods like model-based scoring or human-in-the-loop strategies

Thanks for your comment! Regarding your concerns about the lack of non-code verification methods, in fact, as shown in Table 4 of the ablation study, the setting for ablation of cross verification (executor-based filtering) refers to the synthetic data of AutoIF, where we retain only the final stage of LLM query quality verification and do not conduct the executor feedback cross-validation process. This is essentially equivalent to the baseline process you mentioned, as in this setting, the data originally filtered by code execution will ultimately undergo LLM-based query quality verification. Below are the detailed experimental results:

It is clear that executor feedback plays a vital role in our AutoIF process, while model-based correction methods may be limited by insufficient self-knowledge and exhibit weaker performance. Although more model scoring strategies, such as human-in-the-loop approaches, exist, this paper focuses on automation and scalable enhancement of instruction following with minimal human effort. While this is not the primary focus of our work, we consider it an important area for future optimization and will address it in subsequent research.