Revolve: Optimizing AI Systems by Tracking Response Evolution in Textual Optimization

We thank the reviewer for the detailed evaluation of our work, for acknowledging the soundness of our work, the clarity of the writing, the coverage of related literature, and the comprehensiveness of our experiments. We'll address your concerns in our following response.

Q1. New experiments on the ProTeGi baseline.

We thank the reviewer for suggesting a comparison with ProTeGi. We have now included ProTeGi as a baseline in our prompt optimization experiments. To adapt it to our setting, we used GPT-4o to generate high-quality few-shot exemplars for LLaMA 3.1 8B.

The updated results are shown below:

We observe that while ProTeGi yields slight improvements over standard CoT prompting on Object Counting, our method achieves higher gains. On GSM8K, all methods perform similarly, likely due to task saturation. These results further validate the effectiveness of REVOLVE in improving prompt optimization, particularly in more challenging settings.

Q2. A typo that doesn’t impact clarity too much but ought to be addressed: in page 7, row 353, the results should reference Table 2 instead of Table 5, as Table 5 contains the complete results.

Thank you for pointing this out. We have corrected the reference in the revised manuscript.

We appreciate your valuable time and positive feedback. We'll address your concerns in our following response.

Q1. For the second-order effects, what type of prompt or what type of workflow and LLM did the authors use?

We appreciate the request for clarification on how second-order effects are realized.

In REVOLVE, we approximate second-order effects by combining two elements: (1) first-order feedback from the evaluator, and (2) a similarity signal that tracks how the model's responses evolve across iterations.

This similarity reflects changes in task performance between responses, scaled by how much the prompt has changed (as described in lines 198–200 of the formula). Rather than computing this explicitly, we guide the LLM with instructions like: "Consider how the responses to this variable have changed across previous iterations: <PAST_ITERATIONS>{past_values}</PAST_ITERATIONS>, …, Ensure future responses reflect a meaningful, gradual evolution.".

This setup allows the LLM to reason over both current feedback and the broader trajectory of response updates. We hope this clarifies our workflow.

Q2. Qualitative analysis of response trajectories that shows the second order effects. What type of second-order effects did the authors observe in the distribution of feedback?

We appreciate the request for qualitative analysis. We've added a new section with full response and feedback trajectories to illustrate the differences.

Response Trajectories:

On the Object Counting task, TextGrad repeats the same prompt across the final 8 iterations: "You will answer a reasoning question. … Use standard mathematical notation… Present in bullet points… The last line should be: 'Answer: $VALUE'". This reflects stagnation.

M-TextGrad also plateaus: its last 9 responses repeat a verbose prompt: "Begin by summarizing the problem… Confirm the list is complete… Use consistent notation… Implement a verification step… Consider edge cases… Conclude with: 'Answer: $VALUE'".

REVOLVE repeats a prompt for iterations 5–8: "..., Clearly state the context… List each item… Use a consistent format… Verify the calculation… Final line: 'Answer: $VALUE'" but updates it in iteration 9 with new semantics. This suggests second-order behavior: it detects when progress stalls and makes a more informed shift to move forward.

Feedback Evolution:

In TextGrad, feedback stays repetitive: "Clarify object listing." → "Use bullet points." → "Be more concise." but responses hardly improve.

M-TextGrad shows oscillatory feedback: "Too verbose, streamline." → "Missing verification, add back." → "Too shallow, expand explanation.", feedback changes, but the response bounces back and forth.

REVOLVE aligns feedback with evolving responses. Early iterations focus on structure: "..., Add a verification step.". Mid-phase addresses reasoning: "..., Ensure entity counts match context.". Later rounds provide refinement: "..., Reduce redundancy."

We hope this helps clarify the second-order behavior we observed. Full examples are now included in the revised paper.

Q3. Additional cost of computing the second order effects?

We thank the reviewer for raising this question. To make the cost clearer, we’ve now added a per-iteration breakdown in the main paper:

The extra cost mainly comes from the inclusion of past responses to let the model reason over how things have evolved. We’ve added this clarification to the paper.

Q4. Lack of ablations for the similarity computation.

We thank the reviewer for the suggestion. While similarity is not computed explicitly, we ablate how similarity is conveyed to the LLM. In one version, we replace the task-performance-oriented signal ($L(r(p_t)) − L(r(p_{t-1}))$) with a simpler one based only on raw text differences ($r(p_t) − r(p_{t-1})$).

These results confirm that task-performance-oriented similarity offers more gains, as it offers deeper understanding of response efficacy. We have added these results in the revised manuscript.

Q5. Lack of confidence intervals or significance testing.

Thank you for pointing this out. For all tasks, we run five independent trials using different random seeds. The reported accuracy (e.g., 95.3% on Object Counting) reflects the average across these runs. As noted in Appendix E, we also use consistent seeds [15, 17, 21, 55, 91] for code optimization.

We’ve now added confidence intervals to the main results, e.g., Table 1 would be updated as follows:

For REVOLVE:

• GPT-3.5: 95.5 ± 0.9% (3.9%↑)
• GPT-4: 96.3 ± 0.6% (2.2%↑)
• Gemini 1.5 Pro: 94.0 ± 0.0% (0.0%)
• Llama 3.1: 83.0 ± 1.4% (7.8%↑)

We’ve revised the paper accordingly to reflect this.

We appreciate your valuable time, insights, and highlight our strengths. We'll address your concerns in our following response.

Q1. The authors claim that “REVOVLE can escape local optima.”. However, they do not explain why the similarity function can be interpreted as a gradient from an implementation perspective? In other words, how can REVOLVE be understood as presented in lines 198–200 of the formula?

We thank the reviewer for the helpful question on how REVOLVE can be interpreted as a gradient-like method, and how this relates to lines 198–200 in the formula.

How REVOLVE identifies local optima:

REVOLVE tracks when responses stop improving, even as prompts continue to evolve. This is captured by the numerator of the similarity function: $L(r(p_t))-L(r(p_{t-1}))$, which reflects how much the task performance changes between responses. In implementation, we provide the LLM with both evaluator feedback and the context that generated it, allowing the model to implicitly assess whether performance is improving or stagnating across iterations.

How REVOLVE escapes local optima:

To move beyond stagnation, the model is guided by the denominator: $p_t-p_{t-1}$, which represents the degree of change in the prompt. If the response isn’t improving much despite prompt updates, this signals stagnation. To help the model escape, we encourage it to make stronger updates when past changes haven’t helped, and smaller ones when the trajectory is already improving. This is achieved through instructions like: "Ensure future responses reflect a meaningful, gradual evolution based on past iterations, ..., avoiding abrupt shifts".

Why this resembles a gradient in practice:

Together, the performance difference (numerator) and the prompt change (denominator) form a ratio that functions like a gradient: it reflects whether updates are effective and how future steps should be adjusted. While we don’t compute this numerically, the LLM is given all the elements needed to assess it implicitly, which functions as a curvature-aware, gradient-like signal that informs future updates.

Q2. The authors need to clarify the difference between their similarity function and momentum, as neither involves second-order computation, and they appear similar in terms of implementation.

We thank the reviewer for raising this important point.

In essence, M-TextGrad focuses on repeating feedback, whereas REVOLVE targets repeating model responses.

Implementation-wise, M-TextGrad looks at repeated feedback from the evaluator. If similar feedback appears across steps, it increases the update size, following the intuition that prior changes weren’t enough. This is similar to momentum in traditional optimization. The relevant instruction is: "Similar feedbacks across different steps suggest that the modifications are insufficient… make more significant changes."

In contrast, REVOLVE looks at repeated model responses. If the response itself doesn’t evolve meaningfully across iterations, we prompt the model to revise more significantly. This is achieved with the following prompt: "Additionally, consider how the responses to this variable have changed across previous iterations: <PAST_ITERATIONS>{past_values}</PAST_ITERATIONS>. Make sure future responses reflect a meaningful, gradual evolution based on these past iterations, encouraging thoughtful progress rather than drastic shifts."

The two methods differ mainly in where they look for signals: M-TextGrad focuses on feedback patterns, while REVOLVE monitors the model’s own output history. We find this shift helps the model better track its progress and avoid getting stuck.

Q3. New experiments on the DSPy baseline.

We thank the reviewer for the suggestion.

We’ve now included DSPy as a baseline in our prompt optimization experiments. To adapt it to our setting, we used GPT-4o to generate few-shot exemplars for LLaMA 3.1 8B.

The results are shown below:

On GSM8K, DSPy performs similarly to other methods, which we believe is due to task saturation. On the Object Counting task, however, DSPy underperforms REVOLVE. We hypothesize this is because DSPy’s pipeline-level optimization and demonstration tuning are less adaptive in iterative feedback settings. REVOLVE benefits from tracking how responses evolve, which helps guide the optimization more consistently.

We’ve added these results to the manuscript and appreciate the helpful suggestion.

Q4. Inconsistency usage of "Revolve" and "REVOLVE."

We thank the reviewer for pointing this out. We have standardized “REVOLVE” in the manuscript.